Make CI tests pass - #187
Merged
tomasz-lewicki merged 1 commit intoAug 11, 2026
Merged
Conversation
…/ isaacgym / mujoco) The isaacsim single-gpu pytest lane hangs for ~2h44m on every PR: all 11 camera-sensor tests (new in #170) boot IsaacSim, write their PASS sentinel, then deadlock on process exit in Kit's carbOnPluginShutdown tearing down the omni.syntheticdata/OmniGraph render-product graph a TiledCamera creates. The subprocess never exits, so run_harness's 700s timeout SIGKILLs it and a real PASS is reported as a spurious timeout. Because the pytest matrix is continue-on-error, this bloats CI wall-clock (and previously blocked the docker-build :latest gate) without turning the lane red. Root causes and fixes (each verified on GPU hardware): 1. Teardown deadlock (the 700s hang): each camera *_assert.py now os._exit()s after the sentinel write, bypassing the atexit teardown -- the pattern behavior_assert / scene_spawn_assert already use. 2. IsaacSim camera didn't follow a moving body: render inline via sim.step(render=render_now) so RTX sensors track current poses even at NO_GUI_OR_RENDERING, instead of returning the stale first frame. 3. IsaacSim multi-env misalignment: reconcile self.env_origins to self.scene.env_origins so sim.env_origins means the same thing on every backend. 4. Un-actuated-robot drift flake: harnesses re-pin the root, hold joints rigid at the spawn pose, read-back-verify, and step a few frames before capture. Also adds a ruff PLC0415 per-file-ignore for tests/simulators/*_assert.py (deferred torch/isaaclab imports are inherent to IsaacSim). This revives the fix from #175 (opened but never landed), rebased onto current main; verified green end-to-end on the A10G CI in that PR (isaacsim single-gpu succeeded in 31m). Co-authored-by: Juan G. Bonilla <65906260+juan-g-bonilla@users.noreply.github.com>
tomasz-lewicki
changed the title
juan-g-bonilla
approved these changes
Aug 11, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix GPU tests