Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
f870f10
Add data auto-generate module.
Papaercold Jan 27, 2026
e52df2e
Add data auto-generate module.
Papaercold Jan 27, 2026
ae15ddc
Add data auto-generate module.
Papaercold Jan 27, 2026
4fea004
Merge branch 'LightwheelAI:main' into auto-data-generation
Papaercold Jan 27, 2026
a245712
Add data auto-generate module.
Papaercold Jan 27, 2026
38df5fc
Add data auto-generate module.
Papaercold Jan 27, 2026
83b9e0b
Add auto_terminate.
Papaercold Jan 28, 2026
3b78d34
Add auto_terminate.
Papaercold Jan 28, 2026
47d8504
Add auto_terminate.
Papaercold Jan 28, 2026
2e448cf
Add Description.
Papaercold Jan 28, 2026
0e5f008
State Machinecode refactoring.
Papaercold Jan 28, 2026
9750b3d
State Machinecode refactoring.
Papaercold Jan 28, 2026
34be002
State Machinecode refactoring.
Papaercold Jan 28, 2026
d590d27
State Machinecode refactoring.
Papaercold Jan 28, 2026
b8c97a9
State Machinecode refactoring.
Papaercold Jan 28, 2026
4567af7
State Machinecode refactoring.
Papaercold Jan 28, 2026
6ad6606
Add State Machine code.
Papaercold Jan 28, 2026
e694585
Apply pre-commit fixes (black/isort/pyupgrade) for several files
Papaercold Feb 18, 2026
5074699
Apply pre-commit fixes (black/isort/pyupgrade) for pick_orange.py
Papaercold Feb 18, 2026
18da4fa
Merge branch 'LightwheelAI:main' into auto-data-generation
Papaercold Feb 18, 2026
9e73433
Change structure..
Papaercold Feb 18, 2026
310aee6
Create StateMacchine Class.
Papaercold Feb 18, 2026
fb8304d
Refactor code.
Papaercold Feb 18, 2026
9328529
Fix bugs.
Papaercold Feb 19, 2026
a5338d6
Delete redundant files
Papaercold Feb 19, 2026
e1c8729
Delete redundant files.
Papaercold Feb 19, 2026
c4f8111
Change PickOrangeStateMachine
Papaercold Feb 19, 2026
e42fc85
Change PickOrangeStateMachine
Papaercold Feb 19, 2026
002133f
Change PickOrangeStateMachine
Papaercold Feb 19, 2026
2460e14
Change PickOrangeStateMachine
Papaercold Feb 19, 2026
a748872
Change PickOrangeStateMachine
Papaercold Feb 19, 2026
c07cd7f
Change PickOrangeStateMachine
Papaercold Feb 19, 2026
9f0bdcd
Add state_machine/fold_cloth.py
Papaercold Feb 21, 2026
656fa77
Fix bugs
Papaercold Feb 21, 2026
30f8061
Add state_machine/replay.py
Papaercold Feb 21, 2026
2c8a938
Add readme
Papaercold Feb 21, 2026
f81ba70
fix bugs
Papaercold Feb 21, 2026
d1c3cef
fix bugs
Papaercold Feb 21, 2026
e81f2dd
fix bugs
Papaercold Feb 21, 2026
246a48f
fix bugs
Papaercold Feb 21, 2026
733de37
fix bugs
Papaercold Feb 21, 2026
1bde9b0
Change documents
Papaercold Feb 21, 2026
1342c18
Change bi_arm_cfg
Papaercold Feb 22, 2026
c7d8cca
Add RL module - 1st version.
Papaercold Mar 2, 2026
5dc5b8d
Change documents.
Papaercold Mar 2, 2026
8d1c810
Change bash.
Papaercold Mar 2, 2026
f34cb6e
Delete RL part.
Papaercold Mar 3, 2026
db99a6c
Refactor
Papaercold Mar 6, 2026
f26debf
Change format
Papaercold Mar 6, 2026
f4d40b7
Refactor
Papaercold Mar 6, 2026
88d5c41
Change Isaaclab version==2.3.2
Papaercold Mar 6, 2026
87ced23
Change Isaaclab version==2.3.0
Papaercold Mar 6, 2026
65484c5
Change documents.
Papaercold Mar 6, 2026
5155f6a
Fix bugs.
Papaercold Mar 6, 2026
1d3a912
Change format.
Papaercold Mar 6, 2026
4faee0c
Change format.
Papaercold Mar 6, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
103 changes: 103 additions & 0 deletions docs/docs/docs/features/state_machine.md
Comment thread
EverNorif marked this conversation as resolved.
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
# State Machine Data Generation

The state machine module provides automated data collection for manipulation tasks without human teleoperation. It runs a scripted policy and records demonstrations to HDF5 datasets.

## Recording

```shell
python scripts/datagen/state_machine/generate.py \
--task LeIsaac-SO101-PickOrange-v0 \
--num_envs 1 \
--device cuda \
--enable_cameras \
--record \
--dataset_file ./datasets/pick_orange.hdf5 \
--num_demos 50
```

<details>
<summary><strong>Parameter descriptions for generate.py</strong></summary>

- `--task`: Task environment name to run, e.g., `LeIsaac-SO101-PickOrange-v0`. See [here](/resources/available_env) for available tasks.

- `--num_envs`: Number of parallel simulation environments, usually `1`.

- `--device`: Computation device, such as `cpu` or `cuda` (GPU).

- `--enable_cameras`: Enable camera sensors to collect visual data.

- `--seed`: Seed for the environment. Defaults to current timestamp if not set.

- `--record`: Enable data recording; saves demonstrations to an HDF5 file.

- `--dataset_file`: Path to save the recorded dataset, e.g., `./datasets/pick_orange.hdf5`.

- `--resume`: Resume recording from an existing dataset file.

- `--num_demos`: Number of successful demonstrations to record. Set to `0` for infinite.

- `--step_hz`: Environment stepping rate in Hz (default: `60`).

- `--quality`: Enable quality render mode.

- `--use_lerobot_recorder`: Record directly in LeRobot Dataset format instead of HDF5.

- `--lerobot_dataset_repo_id`: HuggingFace dataset repository ID (format: `username/repository_name`). Required when `--use_lerobot_recorder` is set.

- `--lerobot_dataset_fps`: Dataset frame rate when using LeRobot recorder (default: `30`).

</details>

::::tip
Grasp success rate depends heavily on orange spawn positions. Adjusting the spawn positions in the task's environment config file (e.g. moving oranges closer to the robot base) can significantly improve success rate.
::::

## Replay

After recording, you can replay the collected demonstrations in simulation:

```shell
python scripts/datagen/state_machine/replay.py \
--task LeIsaac-SO101-PickOrange-v0 \
--dataset_file ./datasets/pick_orange.hdf5 \
--task_type so101_state_machine \
--select_episodes 0 \
--device cuda \
--enable_cameras \
--replay_mode action
```

<details>
<summary><strong>Parameter descriptions for replay.py</strong></summary>

- `--task`: Task environment name to run, e.g., `LeIsaac-SO101-PickOrange-v0`.

- `--num_envs`: Number of parallel simulation environments, usually `1`.

- `--device`: Computation device, such as `cpu` or `cuda` (GPU).

- `--enable_cameras`: Enable camera sensors to visualize during replay.

- `--dataset_file`: Path to the recorded dataset, e.g., `./datasets/pick_orange.hdf5`.

- `--replay_mode`: Replay mode — `action` replays IK pose targets, `state` replays joint positions.

- `--task_type`: State machine device type used during recording, e.g., `so101_state_machine` or `bi_so101_state_machine`. Inferred from task name if not set.

- `--select_episodes`: List of episode indices to replay. Leave empty to replay all episodes.

- `--step_hz`: Environment stepping rate in Hz (default: `60`).

</details>

## Adding a New Task

1. Implement a `StateMachineBase` subclass in `source/leisaac/leisaac/datagen/state_machine/`.
2. Register it in `TASK_REGISTRY` inside `scripts/datagen/state_machine/generate.py`:

```python
TASK_REGISTRY = {
"LeIsaac-SO101-PickOrange-v0": (PickOrangeStateMachine, "so101_state_machine"),
"LeIsaac-MY-NewTask-v0": (MyNewStateMachine, "so101_state_machine"),
}
```
1 change: 1 addition & 0 deletions docs/docs/docs/introduction.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ slug: /
LeIsaac provides teleoperation functionality in [IsaacLab](https://isaac-sim.github.io/IsaacLab/main/index.html) using the SO101Leader ([LeRobot](https://github.com/huggingface/lerobot)), including data collection, data conversion, and subsequent policy training.

- 🤖 We use the SO101 Follower robot (and other related robot) in IsaacLab and provide practical teleoperation methods.
- 🦾 State machine scripted policies enable fully automated data collection without human teleoperation.
- 🔄 Ready-to-use scripts convert HDF5 data into the LeRobot dataset format.
- 🧠 Simulation data is used to fine-tune [GR00T N1.5](https://github.com/NVIDIA/Isaac-GR00T) and deploy the policy on real hardware. And more policies will be supported.

Expand Down
6 changes: 6 additions & 0 deletions docs/sidebars.js
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,12 @@ const sidebars = {
link: { type: 'doc', id: 'docs/features/lerobot_recorder' },
items: [],
},
{
type: 'category',
label: 'State Machine Data Generation',
link: { type: 'doc', id: 'docs/features/state_machine' },
items: [],
},
],
},
'docs/trouble_shooting',
Expand Down
Loading