Reinforcement Learning Scripts — Isaac Lab Documentation (original) (raw)

Reinforcement Learning Scripts#

We provide wrappers to different reinforcement libraries. These wrappers convert the data from the environments into the respective libraries function argument and return types.

RL-Games#

• Training an agent withRL-Games on Isaac-Ant-v0:
  Linux

install python module (for rl-games)

./isaaclab.sh -i rl_games

run script for training

./isaaclab.sh -p scripts/reinforcement_learning/rl_games/train.py --task Isaac-Ant-v0 --headless

run script for playing with 32 environments

./isaaclab.sh -p scripts/reinforcement_learning/rl_games/play.py --task Isaac-Ant-v0 --num_envs 32 --checkpoint /PATH/TO/model.pth

run script for playing a pre-trained checkpoint with 32 environments

./isaaclab.sh -p scripts/reinforcement_learning/rl_games/play.py --task Isaac-Ant-v0 --num_envs 32 --use_pretrained_checkpoint

run script for recording video of a trained agent (requires installing ffmpeg)

./isaaclab.sh -p scripts/reinforcement_learning/rl_games/play.py --task Isaac-Ant-v0 --headless --video --video_length 200
Windows
:: install python module (for rl-games)
isaaclab.bat -i rl_games
:: run script for training
isaaclab.bat -p scripts\reinforcement_learning\rl_games\train.py --task Isaac-Ant-v0 --headless
:: run script for playing with 32 environments
isaaclab.bat -p scripts\reinforcement_learning\rl_games\play.py --task Isaac-Ant-v0 --num_envs 32 --checkpoint /PATH/TO/model.pth
:: run script for playing a pre-trained checkpoint with 32 environments
isaaclab.bat -p scripts\reinforcement_learning\rl_games\play.py --task Isaac-Ant-v0 --num_envs 32 --use_pretrained_checkpoint
:: run script for recording video of a trained agent (requires installing ffmpeg)
isaaclab.bat -p scripts\reinforcement_learning\rl_games\play.py --task Isaac-Ant-v0 --headless --video --video_length 200

RSL-RL#

• Training an agent withRSL-RL on Isaac-Reach-Franka-v0:
  Linux

install python module (for rsl-rl)

./isaaclab.sh -i rsl_rl

run script for training

./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py --task Isaac-Reach-Franka-v0 --headless

run script for playing with 32 environments

./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/play.py --task Isaac-Reach-Franka-v0 --num_envs 32 --load_run run_folder_name --checkpoint model.pt

run script for playing a pre-trained checkpoint with 32 environments

./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/play.py --task Isaac-Reach-Franka-v0 --num_envs 32 --use_pretrained_checkpoint

run script for recording video of a trained agent (requires installing ffmpeg)

./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/play.py --task Isaac-Reach-Franka-v0 --headless --video --video_length 200
Windows
:: install python module (for rsl-rl)
isaaclab.bat -i rsl_rl
:: run script for training
isaaclab.bat -p scripts\reinforcement_learning\rsl_rl\train.py --task Isaac-Reach-Franka-v0 --headless
:: run script for playing with 32 environments
isaaclab.bat -p scripts\reinforcement_learning\rsl_rl\play.py --task Isaac-Reach-Franka-v0 --num_envs 32 --load_run run_folder_name --checkpoint model.pt
:: run script for playing a pre-trained checkpoint with 32 environments
isaaclab.bat -p scripts\reinforcement_learning\rsl_rl\play.py --task Isaac-Reach-Franka-v0 --num_envs 32 --use_pretrained_checkpoint
:: run script for recording video of a trained agent (requires installing ffmpeg)
isaaclab.bat -p scripts\reinforcement_learning\rsl_rl\play.py --task Isaac-Reach-Franka-v0 --headless --video --video_length 200

SKRL#

• Training an agent withSKRL on Isaac-Reach-Franka-v0:
  PyTorch
  Linux

install python module (for skrl)

./isaaclab.sh -i skrl

run script for training

./isaaclab.sh -p scripts/reinforcement_learning/skrl/train.py --task Isaac-Reach-Franka-v0 --headless

run script for playing with 32 environments

./isaaclab.sh -p scripts/reinforcement_learning/skrl/play.py --task Isaac-Reach-Franka-v0 --num_envs 32 --checkpoint /PATH/TO/model.pt

run script for playing a pre-trained checkpoint with 32 environments

./isaaclab.sh -p scripts/reinforcement_learning/skrl/play.py --task Isaac-Reach-Franka-v0 --num_envs 32 --use_pretrained_checkpoint

run script for recording video of a trained agent (requires installing ffmpeg)

./isaaclab.sh -p scripts/reinforcement_learning/skrl/play.py --task Isaac-Reach-Franka-v0 --headless --video --video_length 200
Windows
:: install python module (for skrl)
isaaclab.bat -i skrl
:: run script for training
isaaclab.bat -p scripts\reinforcement_learning\skrl\train.py --task Isaac-Reach-Franka-v0 --headless
:: run script for playing with 32 environments
isaaclab.bat -p scripts\reinforcement_learning\skrl\play.py --task Isaac-Reach-Franka-v0 --num_envs 32 --checkpoint /PATH/TO/model.pt
:: run script for playing a pre-trained checkpoint with 32 environments
isaaclab.bat -p scripts\reinforcement_learning\skrl\play.py --task Isaac-Reach-Franka-v0 --num_envs 32 --use_pretrained_checkpoint
:: run script for recording video of a trained agent (requires installing ffmpeg)
isaaclab.bat -p scripts\reinforcement_learning\skrl\play.py --task Isaac-Reach-Franka-v0 --headless --video --video_length 200
JAX
Warning
It is recommended to install JAX manually before proceeding to install skrl and its dependencies, as JAX installs its CPU version by default. For example, pip install -U "jax[cuda12]" can be used to install JAX for CUDA 12. Visit the skrl installation page for more details. Note that JAX GPU support is only available on Linux.

install python module (for skrl)

./isaaclab.sh -i skrl

install skrl dependencies for JAX

./isaaclab.sh -p -m pip install skrl["jax"]

run script for training

./isaaclab.sh -p scripts/reinforcement_learning/skrl/train.py --task Isaac-Reach-Franka-v0 --headless --ml_framework jax

run script for playing with 32 environments

./isaaclab.sh -p scripts/reinforcement_learning/skrl/play.py --task Isaac-Reach-Franka-v0 --num_envs 32 --ml_framework jax --checkpoint /PATH/TO/model.pt

run script for recording video of a trained agent (requires installing ffmpeg)

./isaaclab.sh -p scripts/reinforcement_learning/skrl/play.py --task Isaac-Reach-Franka-v0 --headless --ml_framework jax --video --video_length 200

• Training the multi-agent environment Isaac-Shadow-Hand-Over-Direct-v0 with skrl:
  Linux

install python module (for skrl)

./isaaclab.sh -i skrl

run script for training with the MAPPO algorithm (IPPO is also supported)

./isaaclab.sh -p scripts/reinforcement_learning/skrl/train.py --task Isaac-Shadow-Hand-Over-Direct-v0 --headless --algorithm MAPPO

run script for playing with 32 environments with the MAPPO algorithm (IPPO is also supported)

./isaaclab.sh -p scripts/reinforcement_learning/skrl/play.py --task Isaac-Shadow-Hand-Over-Direct-v0 --num_envs 32 --algorithm MAPPO --checkpoint /PATH/TO/model.pt
Windows
:: install python module (for skrl)
isaaclab.bat -i skrl
:: run script for training with the MAPPO algorithm (IPPO is also supported)
isaaclab.bat -p scripts\reinforcement_learning\skrl\train.py --task Isaac-Shadow-Hand-Over-Direct-v0 --headless --algorithm MAPPO
:: run script for playing with 32 environments with the MAPPO algorithm (IPPO is also supported)
isaaclab.bat -p scripts\reinforcement_learning\skrl\play.py --task Isaac-Shadow-Hand-Over-Direct-v0 --num_envs 32 --algorithm MAPPO --checkpoint /PATH/TO/model.pt

Stable-Baselines3#

• Training an agent withStable-Baselines3on Isaac-Cartpole-v0:
  Linux

install python module (for stable-baselines3)

./isaaclab.sh -i sb3

run script for training

note: we set the device to cpu since SB3 doesn't optimize for GPU anyway

./isaaclab.sh -p scripts/reinforcement_learning/sb3/train.py --task Isaac-Cartpole-v0 --headless --device cpu

run script for playing with 32 environments

./isaaclab.sh -p scripts/reinforcement_learning/sb3/play.py --task Isaac-Cartpole-v0 --num_envs 32 --checkpoint /PATH/TO/model.zip

run script for playing a pre-trained checkpoint with 32 environments

./isaaclab.sh -p scripts/reinforcement_learning/sb3/play.py --task Isaac-Cartpole-v0 --num_envs 32 --use_pretrained_checkpoint

run script for recording video of a trained agent (requires installing ffmpeg)

./isaaclab.sh -p scripts/reinforcement_learning/sb3/play.py --task Isaac-Cartpole-v0 --headless --video --video_length 200
Windows
:: install python module (for stable-baselines3)
isaaclab.bat -i sb3
:: run script for training
:: note: we set the device to cpu since SB3 doesn't optimize for GPU anyway
isaaclab.bat -p scripts\reinforcement_learning\sb3\train.py --task Isaac-Cartpole-v0 --headless --device cpu
:: run script for playing with 32 environments
isaaclab.bat -p scripts\reinforcement_learning\sb3\play.py --task Isaac-Cartpole-v0 --num_envs 32 --checkpoint /PATH/TO/model.zip
:: run script for playing a pre-trained checkpoint with 32 environments
isaaclab.bat -p scripts\reinforcement_learning\sb3\play.py --task Isaac-Cartpole-v0 --num_envs 32 --use_pretrained_checkpoint
:: run script for recording video of a trained agent (requires installing ffmpeg)
isaaclab.bat -p scripts\reinforcement_learning\sb3\play.py --task Isaac-Cartpole-v0 --headless --video --video_length 200

All the scripts above log the training progress to Tensorboard in the logs directory in the root of the repository. The logs directory follows the pattern logs/<library>/<task>/<date-time>, where <library>is the name of the learning framework, <task> is the task name, and <date-time> is the timestamp at which the training script was executed.

To view the logs, run:

Linux

execute from the root directory of the repository

./isaaclab.sh -p -m tensorboard.main --logdir=logs

Windows

:: execute from the root directory of the repository isaaclab.bat -p -m tensorboard.main --logdir=logs