Repository navigation
Data auto-generation part using a simple state machine #127
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
EverNorif
merged 56 commits into
LightwheelAI:main
from
Papaercold:auto-data-generation
Mar 10, 2026
Merged
Changes from all commits
Commits
Show all changes
56 commits
Select commit
Hold shift + click to select a range
f870f10
Add data auto-generate module.
Papaercold e52df2e
Add data auto-generate module.
Papaercold ae15ddc
Add data auto-generate module.
Papaercold 4fea004
Merge branch 'LightwheelAI:main' into auto-data-generation
Papaercold a245712
Add data auto-generate module.
Papaercold 38df5fc
Add data auto-generate module.
Papaercold 83b9e0b
Add auto_terminate.
Papaercold 3b78d34
Add auto_terminate.
Papaercold 47d8504
Add auto_terminate.
Papaercold 2e448cf
Add Description.
Papaercold 0e5f008
State Machinecode refactoring.
Papaercold 9750b3d
State Machinecode refactoring.
Papaercold 34be002
State Machinecode refactoring.
Papaercold d590d27
State Machinecode refactoring.
Papaercold b8c97a9
State Machinecode refactoring.
Papaercold 4567af7
State Machinecode refactoring.
Papaercold 6ad6606
Add State Machine code.
Papaercold e694585
Apply pre-commit fixes (black/isort/pyupgrade) for several files
Papaercold 5074699
Apply pre-commit fixes (black/isort/pyupgrade) for pick_orange.py
Papaercold 18da4fa
Merge branch 'LightwheelAI:main' into auto-data-generation
Papaercold 9e73433
Change structure..
Papaercold 310aee6
Create StateMacchine Class.
Papaercold fb8304d
Refactor code.
Papaercold 9328529
Fix bugs.
Papaercold a5338d6
Delete redundant files
Papaercold e1c8729
Delete redundant files.
Papaercold c4f8111
Change PickOrangeStateMachine
Papaercold e42fc85
Change PickOrangeStateMachine
Papaercold 002133f
Change PickOrangeStateMachine
Papaercold 2460e14
Change PickOrangeStateMachine
Papaercold a748872
Change PickOrangeStateMachine
Papaercold c07cd7f
Change PickOrangeStateMachine
Papaercold 9f0bdcd
Add state_machine/fold_cloth.py
Papaercold 656fa77
Fix bugs
Papaercold 30f8061
Add state_machine/replay.py
Papaercold 2c8a938
Add readme
Papaercold f81ba70
fix bugs
Papaercold d1c3cef
fix bugs
Papaercold e81f2dd
fix bugs
Papaercold 246a48f
fix bugs
Papaercold 733de37
fix bugs
Papaercold 1bde9b0
Change documents
Papaercold 1342c18
Change bi_arm_cfg
Papaercold c7d8cca
Add RL module - 1st version.
Papaercold 5dc5b8d
Change documents.
Papaercold 8d1c810
Change bash.
Papaercold f34cb6e
Delete RL part.
Papaercold db99a6c
Refactor
Papaercold f26debf
Change format
Papaercold f4d40b7
Refactor
Papaercold 88d5c41
Change Isaaclab version==2.3.2
Papaercold 87ced23
Change Isaaclab version==2.3.0
Papaercold 65484c5
Change documents.
Papaercold 5155f6a
Fix bugs.
Papaercold 1d3a912
Change format.
Papaercold 4faee0c
Change format.
Papaercold File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,103 @@ | ||
| # State Machine Data Generation | ||
|
|
||
| The state machine module provides automated data collection for manipulation tasks without human teleoperation. It runs a scripted policy and records demonstrations to HDF5 datasets. | ||
|
|
||
| ## Recording | ||
|
|
||
| ```shell | ||
| python scripts/datagen/state_machine/generate.py \ | ||
| --task LeIsaac-SO101-PickOrange-v0 \ | ||
| --num_envs 1 \ | ||
| --device cuda \ | ||
| --enable_cameras \ | ||
| --record \ | ||
| --dataset_file ./datasets/pick_orange.hdf5 \ | ||
| --num_demos 50 | ||
| ``` | ||
|
|
||
| <details> | ||
| <summary><strong>Parameter descriptions for generate.py</strong></summary> | ||
|
|
||
| - `--task`: Task environment name to run, e.g., `LeIsaac-SO101-PickOrange-v0`. See [here](/resources/available_env) for available tasks. | ||
|
|
||
| - `--num_envs`: Number of parallel simulation environments, usually `1`. | ||
|
|
||
| - `--device`: Computation device, such as `cpu` or `cuda` (GPU). | ||
|
|
||
| - `--enable_cameras`: Enable camera sensors to collect visual data. | ||
|
|
||
| - `--seed`: Seed for the environment. Defaults to current timestamp if not set. | ||
|
|
||
| - `--record`: Enable data recording; saves demonstrations to an HDF5 file. | ||
|
|
||
| - `--dataset_file`: Path to save the recorded dataset, e.g., `./datasets/pick_orange.hdf5`. | ||
|
|
||
| - `--resume`: Resume recording from an existing dataset file. | ||
|
|
||
| - `--num_demos`: Number of successful demonstrations to record. Set to `0` for infinite. | ||
|
|
||
| - `--step_hz`: Environment stepping rate in Hz (default: `60`). | ||
|
|
||
| - `--quality`: Enable quality render mode. | ||
|
|
||
| - `--use_lerobot_recorder`: Record directly in LeRobot Dataset format instead of HDF5. | ||
|
|
||
| - `--lerobot_dataset_repo_id`: HuggingFace dataset repository ID (format: `username/repository_name`). Required when `--use_lerobot_recorder` is set. | ||
|
|
||
| - `--lerobot_dataset_fps`: Dataset frame rate when using LeRobot recorder (default: `30`). | ||
|
|
||
| </details> | ||
|
|
||
| ::::tip | ||
| Grasp success rate depends heavily on orange spawn positions. Adjusting the spawn positions in the task's environment config file (e.g. moving oranges closer to the robot base) can significantly improve success rate. | ||
| :::: | ||
|
|
||
| ## Replay | ||
|
|
||
| After recording, you can replay the collected demonstrations in simulation: | ||
|
|
||
| ```shell | ||
| python scripts/datagen/state_machine/replay.py \ | ||
| --task LeIsaac-SO101-PickOrange-v0 \ | ||
| --dataset_file ./datasets/pick_orange.hdf5 \ | ||
| --task_type so101_state_machine \ | ||
| --select_episodes 0 \ | ||
| --device cuda \ | ||
| --enable_cameras \ | ||
| --replay_mode action | ||
| ``` | ||
|
|
||
| <details> | ||
| <summary><strong>Parameter descriptions for replay.py</strong></summary> | ||
|
|
||
| - `--task`: Task environment name to run, e.g., `LeIsaac-SO101-PickOrange-v0`. | ||
|
|
||
| - `--num_envs`: Number of parallel simulation environments, usually `1`. | ||
|
|
||
| - `--device`: Computation device, such as `cpu` or `cuda` (GPU). | ||
|
|
||
| - `--enable_cameras`: Enable camera sensors to visualize during replay. | ||
|
|
||
| - `--dataset_file`: Path to the recorded dataset, e.g., `./datasets/pick_orange.hdf5`. | ||
|
|
||
| - `--replay_mode`: Replay mode — `action` replays IK pose targets, `state` replays joint positions. | ||
|
|
||
| - `--task_type`: State machine device type used during recording, e.g., `so101_state_machine` or `bi_so101_state_machine`. Inferred from task name if not set. | ||
|
|
||
| - `--select_episodes`: List of episode indices to replay. Leave empty to replay all episodes. | ||
|
|
||
| - `--step_hz`: Environment stepping rate in Hz (default: `60`). | ||
|
|
||
| </details> | ||
|
|
||
| ## Adding a New Task | ||
|
|
||
| 1. Implement a `StateMachineBase` subclass in `source/leisaac/leisaac/datagen/state_machine/`. | ||
| 2. Register it in `TASK_REGISTRY` inside `scripts/datagen/state_machine/generate.py`: | ||
|
|
||
| ```python | ||
| TASK_REGISTRY = { | ||
| "LeIsaac-SO101-PickOrange-v0": (PickOrangeStateMachine, "so101_state_machine"), | ||
| "LeIsaac-MY-NewTask-v0": (MyNewStateMachine, "so101_state_machine"), | ||
| } | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.