Skip to content

feat(data): optionally export arm-local YAM end-effector poses - #17

Open
sawhney17 wants to merge 1 commit into
amazon-far:mainfrom
sawhney17:feat/arm-local-ee-pose-export
Open

feat(data): optionally export arm-local YAM end-effector poses#17
sawhney17 wants to merge 1 commit into
amazon-far:mainfrom
sawhney17:feat/arm-local-ee-pose-export

Conversation

@sawhney17

Copy link
Copy Markdown

Summary

  • Adds an opt-in --derive-ee-poses flag to export_mcap.py and export_hf_task.py.
  • Writes a separate end_effector_poses.npz sidecar with observed/commanded left/right grasp-site poses, aligned timestamps, and a validity mask.
  • Leaves states_actions.bin, videos, metadata shape, and all legacy outputs unchanged when the flag is off.
  • Uses the official bundled I2RT YAM model and records model SHA-256, MuJoCo version, frame, units, quaternion convention, alignment method, and mask assignments.
  • Emits only arm-local poses. It does not guess the transform between physical arm bases.
  • Keeps missing, malformed, or non-finite joint samples as NaN with an unset validity bit; legacy zero-padding is never passed to FK.

Why

Issue #13 asks for an official FK procedure for raw episodes whose EEF pose fields are empty. I independently audited all 40 ungated raw MCAPs in the pinned Voxel51 preview:

  • 314,081 of 641,593 exact arm records have an empty pose field.
  • All 314,081 retain six finite joints and are recoverable arm-locally.
  • The YAM chain validates against 327,512 intact records.
  • Intact episodes expose two right-base offsets, -0.61 m and -0.80 m. Hard-coding the common offset creates a 19 cm error on 16,118 valid right-arm records, so this patch intentionally refuses a universal shared-frame transform.

Reproducible audit, independent cross-check, recovery artifact, and interactive proof:
https://github.com/sawhney17/abc-geometry-recovery

Data contract

The optional NPZ uses schema abc.end_effector_poses.v1. It contains four (N, 7) float64 arrays in [x, y, z, qw, qx, qy, qz] order, timestamp_ns, and a uint8 valid_mask. Bits 0-3 map to left state, right state, left action, and right action.

Validation

  • 7 focused tests pass in a minimal Python 3.12 environment.
  • Literal public left/right fixtures match the bundled official model to 1e-12.
  • Missing and malformed topics remain NaN and unmarked.
  • Causal alignment and all four validity bits are covered.
  • Sidecar publication is atomic and metadata is JSON-serializable.
  • Historical three-item export jobs remain compatible; wrapper forwarding is opt-in only.

Related to #13. This addresses the FK/export portion only; it does not claim to solve the camera calibration requests in that issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant