Supported single-RTX-LiDAR path for replaying a 4-second measured MID360 ray sequence in Isaac Sim 6.0.1?

[junminkyung]

Following up on #685, I would like to confirm the supported way to replay a measured MID360 sequence with one native RTX LiDAR instance per physical sensor.

Environment

• Isaac Sim: 6.0.1-rc.7+release.42383.32955d8d.gl
• LiDAR extension: omni.sensors.nv.lidar 3.0.0; experimental RTX Python API
• Ubuntu 22.04.5 LTS; GeForce RTX 3090 Ti (24 GB), single rendering GPU
• Driver 580.159.04; nvidia-smi CUDA compatibility display: 13.0 (not a claim about the bundled CUDA runtime/toolkit version)

Target

Replay 800,000 recorded azimuth/elevation pairs over 4 seconds, at 200,000 scheduled rays/s: 20,000 slots every 100 ms, with 5 microsecond firing offsets. This is a finite measured sequence replayed cyclically, not a claim to reproduce an indefinitely non-repeating manufacturer scan model. We need to preserve the recorded direction order and acquisition timing, rather than only match the visual coverage.

What currently works

Four co-located immutable RTX LiDAR instances, each holding ten 20,000-ray emitter states, provide a working workaround. All four compute continuously; the collector selects one bank per second using native timestamps. We pre-compensate the ROTARY tick rotation in the authored directions, before ray intersections.

In a stationary mounted-robot scene, the selected output passed a 10-second ROS capture: 100 consecutive frames, 20,000 slots/frame including nulls, 100 ms header intervals, and 5 microsecond point offsets. Maximum valid-ray direction difference from the input sequence was approximately 0.000051 degrees. This validates the replay pipeline, not real-sensor material/noise fidelity. Moving-robot integration is not validated.

Two separate constraints

1. State progression: in a small four-ray, 40-state ROTARY test, with scanRateBaseHz=1, patternFiringRateHz=10 and stateResolutionStep=1, only states 0–9 were observed repeatedly across one-second scan boundaries. Two-state tests did advance, but their period divides the ten ticks per scan. This is our observation under that configuration, not a claim that all configurations are limited to ten states.
2. Attribute payload: storing four 32-bit arrays for 800,000 rays already requires approximately 12.21 MiB before overhead; our five-array form is approximately 15.26 MiB. The approximately 5 MiB transfer limit discussed in Non Repetitive LiDAR support #685 prevents simply authoring the entire sequence into one prim. Splitting the same total into more states would not reduce that payload. The installed documentation marks emitterStatesFile as unsupported.

Questions

• Is there a supported configuration that advances the emitter-state sequence across native scan boundaries while keeping the original 5 microsecond firing offsets? A small working 6.0.1 example would help distinguish a configuration error from a limitation.
• Is there a supported fixed-size emitter-block update/streaming API, or an external emitter-state file/model interface, that avoids the attribute payload limit without recreating the sensor every update?
• If such updates are supported, what is the documented output-buffer invalidation and reacquisition contract? We do not want to rely on private render-texture controls or stale GMO views.

Our intended deployment has four physical LiDARs, so the current workaround would require sixteen RTX instances. Before further optimizing that workaround, we would appreciate confirmation of the intended single-instance path, or confirmation that this requires a feature request/custom model interface.

[zwdoescode]

Hi @junminkyung we recently released Isaac Sim 6.1. Could you please check if the latest release address your issues?

[junminkyung]

Hi @zwdoescode, thank you very much for the update.

I have not yet reproduced these tests on Isaac Sim 6.1.0, so I cannot confirm at this time whether the native RTX LiDAR emitter-state issue has been addressed.

Since posting this discussion, I have extended the measured MID360 replay from the original 4-second sequence to a 622.9-second native PCAP-derived sequence containing 124,578,624 recorded directions. However, the current implementation uses a custom Warp ray-casting backend with one logical sensor per physical LiDAR.

This avoids the RTX emitter-state progression and attribute-payload limitations, but it is not a supported single-instance RTX LiDAR solution. Therefore, it does not confirm that the original RTX issue has been resolved.

I will reproduce both the minimal two-state test and the 40-state scan-boundary test on Isaac Sim 6.1.0 and share the results in this discussion.

If available, could you please also let us know whether Isaac Sim 6.1.0 includes any specific changes related to emitter-state progression across scan boundaries, external emitter-state data, or runtime emitter updates? Any relevant release notes, API examples, or issue references would be greatly appreciated.

Thank you again for following up.

[Egoist0319]

Hi @junminkyung, thank you for documenting this in detail. I am encountering a closely related problem while simulating a Livox Avia non-repetitive scan pattern.

My setup uses Isaac Sim 6.0.1 and a four-second sequence containing approximately 960,000 recorded rays. Because of the attribute payload limit, I divided each 100 ms scan spatially across four co-located RTX LiDAR prims and merged their GMO outputs.

I originally authored 40 emitter states on each solid-state OmniLidar. All states and their azimuth, elevation, fireTimeNs, channelId, and bank arrays pass SensorChecker validation. However, the runtime always renders only s001.

I also reduced the test to a single OmniLidar with two states and 32 emitters, using clearly separated -20 and +20 degree azimuth patterns. Consecutive full outputs still used only s001, including when reading GMO through a Replicator writer without LidarSensor. An Isaac Sim collaborator reproduced this behavior on a local 6.0.1-rc.7 source build and created NVIDIA internal ticket 6758724.

Your observation that ROTARY emitter states advance within one native scan but restart at the next scan boundary appears closely related, although my test uses scanType=solidState and does not advance beyond s001.

Have you had a chance to test this on Isaac Sim 6.1? In particular:

• Does the state sequence now continue across native scan boundaries?
• Is the four time-banked RTX LiDAR workaround still required?
• Have you found any supported single-instance or streaming approach?

Any update would be very helpful. I would also be happy to compare minimal profiles or runtime observations.

[zwdoescode]

@Egoist0319 Is it the issue you talked about #818?

[Egoist0319]

yes

[junminkyung]

Hi @Egoist0319, thank you very much for sharing such a detailed reproduction.

Your solid-state result appears closely related to my ROTARY emitter-state observation. Your case may be even more restrictive, since the runtime remains on s001 and does not appear to advance at all.

I have not yet rerun the native RTX tests on Isaac Sim 6.1.0, so I cannot confirm whether the issue tracked in #818 and NVIDIA internal ticket 6758724 has been resolved.

In Isaac Sim 6.0.1, my four time-banked RTX LiDAR workaround was still required. Since then, I have moved the longer measured replay to a custom Warp ray-casting backend. It now replays a 622.9-second native PCAP-derived sequence containing 124,578,624 recorded directions, using one logical sensor per physical LiDAR.

This approach avoids the RTX emitter-state and attribute-payload limitations, but it is not a supported single-instance RTX LiDAR solution. Therefore, I believe the original native RTX question still remains open.

I will reproduce both my minimal two-state test and the 40-state scan-boundary test on Isaac Sim 6.1.0. It would also be very helpful to compare your solid-state profile with my ROTARY profile so that we can determine whether the behavior differs by scan type.

I will share my 6.1.0 results in this discussion once the tests are complete. Thank you again for providing your findings.