How to Add an Already-Retargeted Animation to HumanMotionLibrary in Isaac Sim 6.0.1?

[shajidHossainHemal]

Hi everyone,

I’m using Isaac Sim 6.0.1 with the HumanMotionLibrary to control animations for human characters.

The default HumanMotionLibrary contains several predefined animations/actions such as idle, walk, sit, pick up, etc.

I already have a custom animation that has been retargeted to the same Human skeleton/rig used by the HumanMotionLibrary. I would like to add this animation to the existing HumanMotionLibrary so that I can trigger it as a custom action.

What is the correct way to import/register an already-retargeted .usd animation into the HumanMotionLibrary and then trigger it through the Behavior Agent API.

If anyone has an example Python script or a step-by-step workflow for doing this in Isaac Sim 6.0.1, I would really appreciate it.

Thanks!

[lhuang-svg]

There's a supported path for this in 6.0.1 — custom actions are registered into the HumanMotionLibrary (HML) by ImportCustomActionCommand, then triggered by name via agent.custom_action(...). It isn't in the docs yet, so here are the details from the shipped omni.anim.behavior.core extension. I verified the whole flow below on a stock 6.0.1 install.

Important: even though your clip is already on the HML skeleton, still go through ImportCustomActionCommand rather than hand-placing the .usd. The command does more than retarget: it normalizes units/FPS/up-axis, applies BehaviorSchema.BehaviorActionAPI (this is what makes the action name resolvable), writes the ActionStart/ActionLoop/ActionEnd annotations, rebuilds the CustomActions.usda index, and wires the payload + animationSkelBinding:sourceSkeleton under the motion library. Just pass a skeleton_asset containing the same skeleton — retargeting is then effectively identity and your joint data is preserved. I tested exactly this (source clip authored against the HML's own 81-joint skeleton, skeleton_asset = the HumanMotionLibrary asset) and it imports and plays cleanly.

GUI route: select the HumanMotionLibrary prim in the Stage window → right-click → Behavior Library → Import Custom Action. Fill in Task Name, Import Actor/Skeleton (the skeleton your clip is already on), Import Motion/Animation (your .usd).

Python route:

  import omni.kit.commands
  from pxr import Sdf
  
  motion_library_path = Sdf.Path("/World/Humans/HumanMotionLibrary")
  
  omni.kit.commands.execute(
      "ImportCustomActionCommand",
      motion_library_prim_path=motion_library_path,
      rig_name="Human",
      # Asset containing the skeleton your clip is authored against.
      # Already retargeted -> use the same skeleton the HML uses.
      skeleton_asset="<path>/MyCharacterOrSkeleton.usd",
      action_name="MyAction",
      animation_asset_urls=["<path>/MyRetargetedAnim.usd"],
      action_type="Custom",          # or "Idle" to add it to the idle pool
      # Optional per-clip frame ranges; omit to infer from the layer time range.
      # action_start_ranges=[(0, 20)],
      # action_loop_ranges=[(20, 80)],
      # action_end_ranges=[(80, 100)],
  )

The clip is written to the user custom-actions folder (default ~/Documents/NVIDIA/Omniverse/Behavior/Human/CustomActions//.usd, overridable via the /persistent/exts/omni.anim.behavior.ui/userCustomActionFolder setting) and payloaded in at /CustomActions//. Newly created agents pick up anything already in that folder automatically.

After import you can confirm it landed: the clip prim has BehaviorActionAPI with behaviorActionName == "MyAction" and children ActionStart / ActionLoop / ActionEnd / MyAction.

Triggering it through the Behavior Agent API:

  import omni.anim.behavior.core as bh
  
  behavior_system = bh.acquire_interface()
  agent = behavior_system.get_agent("/World/Humans/Human/SkelRoot")   # SkelRoot path
  
  print(list(agent.get_custom_action_names()))   # -> ['MyAction']
  
  task_id = agent.custom_action("MyAction", duration=1.0)   # 0.0 -> indefinite
  assert task_id != bh.BEHAVIOR_TASK_ID_INVALID
  # agent.get_task_status(task_id) -> RUNNING, then SUCCEEDED when the duration elapses
  # agent.get_action_task_id() -> task_id while it is the active full-body action

custom_action(action, root_animation, duration) also takes bh.BehaviorRootAnimation (IGNORE_ROOT_ANIMATION, USE_ROOT_ANIMATION, APPLY_ROOT_ANIMATION_TO_HIPS) if your clip has root motion. It plays at the agent's current location/facing and interrupts whatever full-body action is running.

Gotchas that actually bit me while testing:

• The scene needs a NavMeshVolume. Without one, get_agent() returns None indefinitely even with the timeline playing and the agent prim present. Add a nav mesh volume covering the area (and a floor), then play.
• get_agent() also returns None until the timeline is actually playing and the agent has been created — poll a few frames rather than calling it once.
• cancel_task(task_id) ends the action but the status then reads SUCCEEDED, not a distinct cancelled value — test for != RUNNING.
• An unknown action name or a negative duration returns BEHAVIOR_TASK_ID_INVALID (-1) instead of raising, so check the return value.
• If you edit files in the CustomActions folder directly, right-click the HML prim → Behavior Library → Refresh Custom Actions (or omni.kit.commands.execute("RefreshCustomActionsCommand")) to rebuild the index. That only rebuilds CustomActions.usda — it doesn't do the schema/annotation authoring the import command does, so it's not a substitute for importing.

Reference implementation: exts/omni.anim.behavior.core/omni/anim/behavior/core/scripts/command.py (ImportCustomActionCommand) and .../tests/custom_action/test_custom_action.py, which is a complete working example of import → play → trigger → cancel.