Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots (original) (raw)
Abstract:We present Universal Manipulation Interface (UMI) -- a data collection and policy learning framework that allows direct skill transfer from in-the-wild human demonstrations to deployable robot policies. UMI employs hand-held grippers coupled with careful interface design to enable portable, low-cost, and information-rich data collection for challenging bimanual and dynamic manipulation demonstrations. To facilitate deployable policy learning, UMI incorporates a carefully designed policy interface with inference-time latency matching and a relative-trajectory action representation. The resulting learned policies are hardware-agnostic and deployable across multiple robot platforms. Equipped with these features, UMI framework unlocks new robot manipulation capabilities, allowing zero-shot generalizable dynamic, bimanual, precise, and long-horizon behaviors, by only changing the training data for each task. We demonstrate UMI's versatility and efficacy with comprehensive real-world experiments, where policies learned via UMI zero-shot generalize to novel environments and objects when trained on diverse human demonstrations. UMI's hardware and software system is open-sourced at this https URL.
Submission history
From: Zhenjia Xu [view email]
[v1] Thu, 15 Feb 2024 21:11:50 UTC (6,943 KB)
[v2] Mon, 19 Feb 2024 23:22:39 UTC (6,952 KB)
[v3] Wed, 6 Mar 2024 00:11:34 UTC (6,939 KB)