DART: Articulated Hand Model with Diverse Accessories and Rich

  Textures (NeurIPS 2022) ([original](https://dart2022.github.io/)) ([raw](?raw))

Abstract

Hand, the bearer of human productivity and intelligence, is receiving much attention due to the recent fever of 3D digital avatars. Among different hand morphable models, MANO has been widely used in various vision & graphics tasks. However, MANO disregards textures and accessories, which largely limits its power to synthesize photorealistic & lifestyle hand data. In this paper, we extend MANO with more Diverse Accessories and Rich Textures, namelyDART. DART is comprised of 325 exquisite hand-crafted texture maps which varies in appearance, and covers different kinds of blemishes, make-ups and accessories. We also provide the Unity GUI which allows people to render hands with user-specific settings, e.g pose, camera, background, lighting, and DART's textures. In this way, we generate large-scale (800K), diverse, and high-fidelity hand images, paired with perfect-aligned 3D labels, called DARTset. Experiments demonstrate its superiority in generalization and diversity. As a great complement for existing datasets, DARTset could boost hand pose estimation & surface reconstruction tasks. DART and Unity software is publicly available for research purpose.

Comparison between DART and MANO basic topology.

In DART, we remould the standard template hand mesh of MANO, which has 778 vertices and 1,538 faces, to a wrist-enhanced template mesh of 842 vertices and 1,666 faces. The pose parameter that drive the template hand mesh in MANO can be used as a direct placement for DART without any modifications.

How to use DART tool in generating your own data?

Step1: clickDART GUI & Code and download Build_Hand.zip, unzip and execute Hand.exe.

Step2: Pose Editing: allow arbitrarily, illumination, accessory, skin color and other terms.

Step3: Exporting: rendered image with GT(mano pose, 2d/3d joint are put into the output.pkl)

2022.09.22 We put the postprocess folder into link. Please use that code for postprocessing the intermediate output yielded by DART's GUI!

DARTset Datasheet & Explanation

Dataset Structure

DARTset is composed of train and test. The folder of each is described as below.

Train set
* Train: 729,189 single hand frames, 25% of which are wearing accessories.

Test set
* Test (H): 288,77 single hand frames, 25% of which are wearing accessories.

Total set
* DARTset: 758,066 single hand frames. Noteworthy here, we conduct experiments on full 800K DARTset and filter out ~42,000 of images which left wrist unsealed on the final version.

Pose sampling we use spherical linear interpolation (Slerp) in pose and root rotation sampling. Among these hands, ~25% are assigned an accessory. In DARTset, basic UV map (skin tones, scars, moles, tattoos) and accessories are all uniformly sampled, the number of their corresponding renders are roughly equal.

Dataset Creation

Texture map (4096 x 4096) are all created manually by 3D artists. GUI and batch data generator is programmed by DART's authors.

Considerations for Using the Data

Additional Information

Acknowledgements

If you find our work useful in your research, please cite:


@inproceedings{gao2022dart,
title={{DART: Articulated Hand Model with Diverse Accessories and Rich Textures}},
author={Daiheng Gao and Yuliang Xiu and Kailin Li and Lixin Yang and Feng Wang and Peng Zhang and Bang Zhang and Cewu Lu and Ping Tan},
booktitle={Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track},
year={2022},
}

The website template was borrowed fromMichaël Gharbi.