Single Image to Simulation-Ready 3D Outfit with Diffusion Prior and Differentiable Physics (original) (raw)

ACM Transactions on Graphics (SIGGRAPH 2025)

1University of California, Los Angeles, 2University of Utah

*Equal Contributions

Dress-1-to-3 can reconstruct simulation-ready clothed humans from casually posed single view images.

Abstract

Recent advances in large models have significantly advanced image-to-3D reconstruction. However, the generated models are often fused into a single piece, limiting their applicability in downstream tasks. This paper focuses on 3D garment generation, a key area for applications like virtual try-on with dynamic garment animations, which require garments to be separable and simulation-ready. We introduce Dress-1-to-3, a novel pipeline that reconstructs physics-plausible, simulation-ready separated garments with sewing patterns and humans from an in-the-wild image. Starting with the image, our approach combines a pre-trained image-to-sewing pattern generation model for creating coarse sewing patterns with a pre-trained multi-view diffusion model to produce multi-view images. The sewing pattern is further refined using a differentiable garment simulator based on the generated multi-view images. Versatile experiments demonstrate that our optimization approach substantially enhances the geometric alignment of the reconstructed 3D garments and humans with the input image. Furthermore, by integrating a texture generation module and a human motion generation module, we produce customized physics-plausible and realistic dynamic garment demonstrations.

Pipeline

Starting with a single-view input image of a clothed human, we first derive an initial estimation of the sewing pattern. Additionally, we employ multi-view diffusion to generate orbital camera views, which serve as ground-truth 3D information for both human pose and garment shape. Next, we utilize differentiable simulation to sew and drape the pattern onto the posed human model, optimizing its shape and physical parameters in conjunction with geometric regularizers. Finally, the optimized garment shape provides a physically plausible rest shape in its static state and is readily animatable using a physical simulator.

Reconstruction Demos

DeepFashion

Input

Initial Pattern

Initial Fitting

Optimized Pattern

Optimized Fitting

Texture Generation

DeepFashion2 Customer

Input

Initial Pattern

Initial Fitting

Optimized Pattern

Optimized Fitting

Texture Generation

Flux Generated

Input

Initial Pattern

Initial Fitting

Optimized Pattern

Optimized Fitting

Texture Generation

Simulation Demos

Input

Simulation

Input

Simulation

Comparison

BibTeX

@article{li2025dress123,
      title={Dress-1-to-3: Single Image to Simulation-Ready 3D Outfit with Diffusion Prior and Differentiable Physics}, 
      Author = {Xuan Li and Chang Yu and Wenxin Du and Ying Jiang and Tianyi Xie and Yunuo Chen and Yin Yang and Chenfanfu Jiang},
      journal={arXiv preprint arXiv:2502.03449},
      year={2025}
}