In this version of IsaacTeleop, a plugin has been added for Foottroller (www.foottroller.com) that enables natural, hands-free locomotion control!
The plugin reads Foottroller’s raw inputs as gamepad axes and buttons and converts them into full locomotion commands via retargeting. You get smooth velocity control (X & Y), rotation, and robot height adjustment — all through intuitive foot gestures. Now you can walk, strafe, turn, and adjust height naturally with your feet, keeping both hands completely free for manipulation, grasping, and other tasks. Check out the demo video below to see how to use Foottroller for Robot locomotion control.
Foottroller Isaac Teleop: Natural Foot Gesture for hands-free locomotion (guide with slow motion) https://youtu.be/qGGIyA1QBXM
How to use: Install Isaac Sim 6.0 Install IsaacLab 3.0.0 beta-2
Clone this repo and build IsaacTeleop:
bash
rm -rf build install
cmake -B build -DISAAC_TELEOP_PYTHON_VERSION=3.12 # use the matching python version for IsaacLab
cmake --build build --target clang_format_fix
cmake --build build
cmake --install build
Install the new build to IsaacLab:
uv pip install "isaacteleop[retargeters,cloudxr,ui]" --find-links=./install/wheels/ --reinstall
Run the foottroller locomotion example to verify Foottroller inputs and retargeter function.
Start cloudXR: https://nvidia.github.io/IsaacTeleop/main/getting_started/quick_start.html
In a terminal run cloudXR server
python -m isaacteleop.cloudxr
Connect an XR headset
Connect Foottroller control box to the Linux PC via Bluetooth.
Set Foottroller control box to Dev mode by pressing the button closest to the two LED lights until both LED lights are on. This allows raw Foottroller measurements sent to the PC.
Set gamepad to zero dead zones for Foottroller axes.
Install jstest-gtk if not installed.
sudo apt install jstest-gtk
Open a new terminal and source the CloudXR environment variables:
source ~/.cloudxr/run/cloudxr.env
Run Foottroller plugin in the terminal
In build/src/plugins/foottroller/, run ./foottroller_plugin /dev/input/js0 # make sure /dev/input/js0 is the device path for Foottroller
Open another new terminal and source the CloudXR environment variables:
source ~/.cloudxr/run/cloudxr.env
Run the example
python examples/teleop/python/foottroller_locomotion_example.py
Generate locomotion controls with foot gestures using foot gestures as shown in the demo video
The unified framework for high-fidelity ego-centric and robotics data collection.
Isaac Teleop: The unified standard for high-fidelity egocentric and robot data collection. It is designed to address the data bottleneck in robot learning by streamlining device integration; standardizing high-fidelity human demo data collection; and foster device & data interoperability.
- Unified stack for sim & real teleoperation
- Standardized device interface
- Flexible retargeting framework
See the Ecosystem page for supported robotics stacks, devices, and retargeting details.
- Currently supported use cases
- Use XR headsets for gripper / tri-finger hand manipulation
- Use XR headsets with gloves for dex-hand manipulation
- Seated full body loco-manipulation (Homie)
- Tracking based full body loco-manipulation (Sonic)
- Egocentric data collection (aka “no-robot”)
- Upcoming use cases
- Teleoperate using only non-XR devices (e.g. gamepad, Gello, haply, etc.)
- Teleoperate cloud based robotics simulations
- Remote teleoperation with camera streaming to the desktop
- Remote teleoperation with immersive camera streaming to XR headsets
Our documentation page provides everything you need to get started, including detailed tutorials and step-by-step guides. Follow these links to learn more:
Isaac Teleop is design to work side by side with NVIDIA Isaac Lab starting with Isaac Lab 3.0 Beta release.
To get started, please refer to Isaac Lab's Installation guide for more details. Then follow the CloudXR teleoperation in Isaac Lab to get started with Teleoperation in Sim.