keyboard teleoperation enhance and docs update.

README.md

@@ -17,13 +17,13 @@ This repository provides teleoperation functionality in [IsaacLab](https://isaac
 
 Please refer to our [documentation](https://lightwheelai.github.io/leisaac/) to learn how to use this repository. Follow these links to learn more about:
 
-LeIsaac is now the official imitation-learning (IL) simulation playground integrated into LeRobot’s EnvHub. It provides fast, scalable Isaac-based environments designed for imitation learning, control, and policy evaluation. It is featured in LeRobot: [LeIsaac × LeRobot EnvHub](https://huggingface.co/docs/lerobot/en/envhub_leisaac)
-
 - [Installation and Setup](https://lightwheelai.github.io/leisaac/docs/getting_started/installation)
 - [Extra Features](https://lightwheelai.github.io/leisaac/docs/features)
 - [Policy Inference](https://lightwheelai.github.io/leisaac/docs/getting_started/policy_support)
 - [Available Robots](https://lightwheelai.github.io/leisaac/resources/available_robots), [Environments](https://lightwheelai.github.io/leisaac/resources/available_env) and [Policy](https://lightwheelai.github.io/leisaac/resources/available_policy)
 
+LeIsaac is now the official imitation-learning (IL) simulation playground integrated into LeRobot’s EnvHub. It provides fast, scalable Isaac-based environments designed for imitation learning, control, and policy evaluation. It is featured in LeRobot: [LeIsaac × LeRobot EnvHub](https://huggingface.co/docs/lerobot/en/envhub_leisaac)
+
 ## Acknowledgements 🙏
 
 We gratefully acknowledge [IsaacLab](https://github.com/isaac-sim/IsaacLab) and [LeRobot](https://github.com/huggingface/lerobot) for their excellent work, from which we have borrowed some code.

docs/README.md

@@ -1,6 +1,6 @@
-# Leisaac Docs Website
+# LeIsaac Docs Website
 
-Leisaac docs website is built using [Docusaurus](https://docusaurus.io/), a modern static website generator.
+LeIsaac docs website is built using [Docusaurus](https://docusaurus.io/), a modern static website generator.
 
 Make sure your working dir is `leisaac/docs`.
 
@@ -43,4 +43,4 @@ GIT_USER=<Your GitHub username> yarn deploy
 If you are using GitHub pages for hosting, this command is a convenient way to build the website and push to the `gh-pages` branch.
 
 >[!NOTE] 
-> The Leisaac repository is already configured with GitHub Actions. You only need to modify the relevant documents, verify the results locally, and then submit a PR; manual deployment is not necessary.
+> The LeIsaac repository is already configured with GitHub Actions. You only need to modify the relevant documents, verify the results locally, and then submit a PR; manual deployment is not necessary.

docs/docs/docs/getting_started/teleoperation.md

@@ -49,6 +49,10 @@ python scripts/environments/teleoperation/teleop_se3_agent.py \
 
 </details>
 
+::::tip
+We support multiple devices for teleoperation. See [here](/resources/available_devices) for more devices and usage instructions.
+::::
+
 ## Operating Instructions
 
 If the calibration file does not exist at the specified cache path, or if you launch with `--recalibrate`, you will be prompted to calibrate the SO101Leader.  Please refer to the [documentation](https://huggingface.co/docs/lerobot/so101#calibration-video) for calibration steps.

docs/docs/docs/introduction.md

@@ -11,7 +11,7 @@ slug: /
   style={{ width: '100%', maxWidth: '960px', borderRadius: '8px' }}
 />
 
-Leisaac provides teleoperation functionality in [IsaacLab](https://isaac-sim.github.io/IsaacLab/main/index.html) using the SO101Leader ([LeRobot](https://github.com/huggingface/lerobot)), including data collection, data conversion, and subsequent policy training.
+LeIsaac provides teleoperation functionality in [IsaacLab](https://isaac-sim.github.io/IsaacLab/main/index.html) using the SO101Leader ([LeRobot](https://github.com/huggingface/lerobot)), including data collection, data conversion, and subsequent policy training.
 
 - 🤖 We use the SO101 Follower robot (and other related robot) in IsaacLab and provide practical teleoperation methods.
 - 🔄 Ready-to-use scripts convert HDF5 data into the LeRobot dataset format.

docs/docs/resources/available_devices.md

@@ -0,0 +1,49 @@
+# Available Devices
+
+::::tip
+We support several teleoperation devices in simulation, but for the best control experience we recommend using the SO101 Leader.
+::::
+
+## Single-Arm SO101 Follower
+
+You can control the Single-Arm SO101 Follower in simulation with SO101 Leader or keyboard.
+
+### SO101-Leader
+
+Run with `--teleop_device=so101leader` to enable the SO101 Leader; once the port is configured you can start operating immediately.
+
+When you move the physical SO101 Leader, the simulated manipulator mirrors the same motion.
+
+
+### Keyboard
+
+Run with `--teleop_device=keyboard` to enable keyboard control.
+
+| target frame                                               | translation control                          | rotation control                         |
+| :--------------------------------------------------------: | :------------------------------------------: | :--------------------------------------: |
+| ![target frame](/img/devices/teleop_info/target_frame.jpg) | ![trans](/img/devices/teleop_info/trans.jpg) | ![rot](/img/devices/teleop_info/rot.jpg) |
+
+Keyboard teleoperation relies on target-frame-based control. Within the target-frame (gripper link) coordinate system, key signals always command the movement of that frame, so the operator only needs to focus on the desired pose of the link rather than sending joint-level commands. Gripper joints are still handled separately.
+
+We split target-frame control into translation and rotation. Translation covers forward/backward, up/down, and left/right motions of the target frame. Because of the manipulator structure, the left/right action is effectively a rotation around the base rather than a pure translation. Rotation control includes pitch up/down and rotation around the wrist link.
+
+The keyboard mapping is as follows:
+
+| input key | description                                                                   |
+| :-------: | :---------------------------------------------------------------------------- |
+| `W` / `S` | Forward/backward, aligned with the red arrows in the translation diagram      |
+| `A` / `D` | Left/right, aligned with the green arrows in the translation diagram          |
+| `Q` / `E` | Up/down, aligned with the blue arrows in the translation diagram              |
+| `J` / `L` | Rotate (yaw) left/right, aligned with the blue arrows in the rotation diagram |
+| `I` / `K` | Rotate (pitch) up/down, aligned with the green arrows in the rotation diagram |
+| `U` / `O` | Gripper open/close                                                            |
+
+
+## Bi-Arm SO101 Follower
+
+Because keyboard control becomes complicated for dual arms, the Bi-Arm SO101 Follower in simulation currently supports only the Bi-SO101 Leader.
+
+### Bi-SO101-Leader
+
+Run with `--teleop_device=bi-so101leader` to enable the Bi-SO101 Leader. After configuring both `left_arm_port` and `right_arm_port`, you can operate immediately, and the two simulated arms will reproduce the real bi-arm behavior.
+

docs/docusaurus.config.js

@@ -10,7 +10,7 @@ import {themes as prismThemes} from 'prism-react-renderer';
 
 /** @type {import('@docusaurus/types').Config} */
 const config = {
-  title: 'Leisaac Document',
+  title: 'LeIsaac Document',
   tagline: 'LeIsaac Docs: Quickstart, Guides, and Resources',
   favicon: 'img/favicon.ico',
 

docs/sidebars.js

@@ -107,6 +107,7 @@ const sidebars = {
   resources: [
     'resources/available_robots',
     'resources/available_env',
+    'resources/available_devices',
     'resources/available_policy',
   ],
 };

scripts/environments/teleoperation/teleop_se3_agent.py

@@ -54,7 +54,7 @@
 from isaaclab_tasks.utils import parse_env_cfg
 from isaaclab.managers import TerminationTermCfg, DatasetExportMode
 
-from leisaac.devices import Se3Keyboard, SO101Leader, BiSO101Leader
+from leisaac.devices import SO101Keyboard, SO101Leader, BiSO101Leader
 from leisaac.enhance.managers import StreamingRecorderManager, EnhanceDatasetExportMode
 from leisaac.utils.env_utils import dynamic_reset_gripper_effort_limit_sim
 
@@ -164,7 +164,7 @@ def main():  # noqa: C901
 
     # create controller
     if args_cli.teleop_device == "keyboard":
-        teleop_interface = Se3Keyboard(env, sensitivity=0.25 * args_cli.sensitivity)
+        teleop_interface = SO101Keyboard(env, sensitivity=args_cli.sensitivity)
     elif args_cli.teleop_device == "so101leader":
         teleop_interface = SO101Leader(env, port=args_cli.port, recalibrate=args_cli.recalibrate)
     elif args_cli.teleop_device == "bi-so101leader":

source/leisaac/leisaac/devices/__init__.py

@@ -1,3 +1,3 @@
 from .device_base import DeviceBase
 from .lerobot import SO101Leader, BiSO101Leader
-from .keyboard import Se3Keyboard
+from .keyboard import SO101Keyboard

source/leisaac/leisaac/devices/action_process.py

@@ -19,15 +19,16 @@ def init_action_cfg(action_cfg, device):
             scale=1.0,
         )
     elif device in ['keyboard']:
-        action_cfg.arm_action = mdp.RelativeJointPositionActionCfg(
+        action_cfg.arm_action = mdp.DifferentialInverseKinematicsActionCfg(
             asset_name="robot",
-            joint_names=["shoulder_pan", "shoulder_lift", "elbow_flex", "wrist_flex", "wrist_roll"],
-            scale=1.0,
+            joint_names=["shoulder_lift", "elbow_flex", "wrist_flex", "wrist_roll"],
+            body_name="gripper",
+            controller=mdp.DifferentialIKControllerCfg(command_type="pose", ik_method="dls", use_relative_mode=True),
         )
         action_cfg.gripper_action = mdp.RelativeJointPositionActionCfg(
             asset_name="robot",
-            joint_names=["gripper"],
-            scale=0.7,
+            joint_names=["shoulder_pan", "gripper"],
+            scale=1.0,
         )
     elif device in ['bi-so101leader']:
         action_cfg.left_arm_action = mdp.JointPositionActionCfg(
@@ -72,7 +73,7 @@ def init_action_cfg(action_cfg, device):
         action_cfg.gripper_action = mdp.RelativeJointPositionActionCfg(
             asset_name="robot",
             joint_names=["gripper"],
-            scale=0.7,
+            scale=1.0,
         )
     else:
         action_cfg.arm_action = None
@@ -107,7 +108,7 @@ def preprocess_device_action(action: dict[str, Any], teleop_device) -> torch.Ten
     if action.get('so101_leader') is not None:
         processed_action = convert_action_from_so101_leader(action['joint_state'], action['motor_limits'], teleop_device)
     elif action.get('keyboard') is not None:
-        processed_action = torch.zeros(teleop_device.env.num_envs, 6, device=teleop_device.env.device)
+        processed_action = torch.zeros(teleop_device.env.num_envs, 8, device=teleop_device.env.device)
         processed_action[:, :] = action['joint_state']
     elif action.get('bi_so101_leader') is not None:
         processed_action = torch.zeros(teleop_device.env.num_envs, 12, device=teleop_device.env.device)

source/leisaac/leisaac/devices/keyboard/__init__.py

@@ -1 +1 @@
-from .se3_keyboard import Se3Keyboard
+from .so101_keyboard import SO101Keyboard