GitHub - NVIDIA/IsaacTeleop: The unified framework for sim & real robot teleoperation (original) (raw)

Isaac Teleop

The unified framework for high-fidelity ego-centric and robotics data collection.

Overview

Isaac Teleop: The unified standard for high-fidelity egocentric and robot data collection. It is designed to address the data bottleneck in robot learning by streamlining device integration; standardizing high-fidelity human demo data collection; and foster device & data interoperability.

Key Features

• Unified stack for sim & real teleoperation
• Standardized device interface
• Flexible retargeting framework

See the Ecosystem page for supported robotics stacks, devices, and retargeting details.

Teleoperation Use Cases

• Currently supported use cases
  • Use XR headsets for gripper / tri-finger hand manipulation
  • Use XR headsets with gloves for dex-hand manipulation
  • Seated full body loco-manipulation (Homie)
  • Tracking based full body loco-manipulation (Sonic)
  • Egocentric data collection (aka “no-robot”)
• Upcoming use cases
  • Teleoperate using only non-XR devices (e.g. gamepad, Gello, haply, etc.)
  • Teleoperate cloud based robotics simulations
  • Remote teleoperation with camera streaming to the desktop
  • Remote teleoperation with immersive camera streaming to XR headsets

Quick Start

Documentation

Our documentation page provides everything you need to get started, including detailed tutorials and step-by-step guides. Follow these links to learn more:

• Architecture
• Quick installation steps
• How to build from source

Install & Run Isaac Lab

Isaac Teleop is design to work side by side with NVIDIA Isaac Lab starting with Isaac Lab 3.0 Beta release.

To get started, please refer to Isaac Lab's Installation guide for more details. Then follow the CloudXR teleoperation in Isaac Lab to get started with Teleoperation in Sim.