Infrared Camera Support

[reza-shahriari]

Hi Isaac-Sim team,
I hope this message finds you well. I'm a user of ISAAC Sim and have been really impressed with the quality and capabilities of the software.
I wanted to request the addition of infrared camera support as a feature. This would be incredibly valuable for robotics simulations involving thermal sensing, night vision applications, and various autonomous systems that rely on IR sensors.
Best regards,
Reza

[vick-yu]

Thank you for the suggestion! Could you provide more details about the specific infrared camera models or types you’re interested in simulating with Isaac Sim? For example, are you looking to simulate thermal imaging sensors, near-infrared cameras, or particular hardware brands? Knowing your use case or target hardware would help us better understand your requirements and explore more realistic or relevant simulation features.

Looking forward to your input!

[reza-shahriari]

While all of these features have merit, thermal imaging would likely provide the greatest value. I'm curious whether the IR camera could support configurable parameters such as field of view, similar to what we have with standard TV cameras.
Another highly desirable capability would be adjustable wavelength sensing. This would enable us to simulate various infrared bands including LWIR (Long-Wave Infrared), SWIR (Short-Wave Infrared), and other spectral ranges

[Alphakyl]

Since you asked for some specific models:

I think the FLIR ADK (which I believe is a FLIR Boson 640 sensor) would be a good model to include in Isaac sim, since it is designed for autonomous driving applications. It has also been included in some public robotics datasets such as Novel Sensors for Autonomous Driving (Charmichael et al. 2024).

Alternatively, the FLIR Hadron which is being deployed on small drones with AI integration.

Both the aforementioned units are LWIR.

[reza-shahriari]

it will be nice if we have specific model but will be even better if we able to change the configs (LWIR , SWIR, sensor size etc. ) then we can test different cameras and find the best for our usage

[vick-yu]

Thanks for the additional details, these are helpful for scoping

[whittenator]

This would be amazing!

[vick-yu]

Thanks again for all the additional context and concrete examples around thermal IR support, this is very helpful for us as we scope things out.

To better understand your needs and align with our internal planning, could you share a bit more about your intended workflow?

Are you primarily interested in a camera that visualizes thermal information you already have (for example, from external simulation tools or recorded datasets), or do you need Isaac Sim to perform a full heat‑transfer‑based thermal simulation of the environment and objects?

For your target use cases (e.g., robotics, autonomous driving, drones), are you mostly focused on LWIR thermal cameras like the FLIR ADK/Boson 640 or Hadron, or do you also need support for other bands such as SWIR/near‑IR in the same workflows?

If you have any representative datasets, sensor specs, or public benchmarks you’d like to match (beyond the ones you already mentioned), pointers to those would help us understand the level of realism and fidelity you’re aiming for.

These details will help us differentiate between a sensor‑modeling/visualization feature versus a more physically detailed thermal simulation pipeline, and plan accordingly.

[vick-yu]

Hi @reza-shahriari,

We have an update on thermal/IR camera simulation.

The Kit version shipping with Isaac Sim 6.0 GA includes Sensor Processing Graph (SPG), a Kit-level framework for building custom sensor models on top of the RTX renderer. This is the right tool for implementing thermal/IR cameras. Here's the approach:

1. Encode temperature as emission
Assign an OmniPBR material to each surface with emission enabled, using the emissive color channel to encode the object's temperature value.

2. Capture the thermal signal
Use the PtSelfIllumination AOV (render variable) to capture only the self-illumination (emission) component -- this gives you a clean per-pixel temperature readout without contamination from reflected or bounced light. Alternatively, render HdrColor with max bounces set to 0
to suppress global illumination.

3. Implement your sensor model in SPG
Feed the AOV output into a Sensor Processing Graph to implement your specific sensor characteristics -- NETD noise, spectral response curves, vignetting, or any other LWIR/MWIR sensor behavior. SPG lets you chain processing stages using Python or Warp kernels.

This gives you full control over the sensor model while leveraging the RTX renderer for physically-based emission.

[reza-shahriari]

veryyyyy well
Thanks for that!
ill give it a try

[vick-yu]

Hi @reza-shahriari — just a heads-up that Isaac Sim 6.0 GA was officially released last Thursday, so the SPG-based thermal camera approach described is now available in the stable release. Have you had a chance to try it out? Let us know if you run into any questions.

[reza-shahriari]

I actually installed the pre-release version.
I couldn’t find a way to implement the SPG despite trying many times.
I completed steps 1 and 2, but I am not familiar enough with Isaac Sim to create the SPG.
It would be great if you could share a tutorial or explain the process in more detail so I can try again.
Thanks again!

[vick-yu]

Could you see if https://docs.omniverse.nvidia.com/kit/docs/omni.rtx.spg/latest/Overview.html is helpful?

[vick-yu]

Hi @reza-shahriari — thanks for your patience and for engaging so thoroughly on this request! We shared the SPG-based thermal camera approach available in Isaac Sim 6.0 GA along with the official SPG documentation, but we haven't heard back on whether that resolved your implementation questions. If you're still running into issues, please open a new discussion and reference this one so we have full context. We're happy to help further!

[reza-shahriari]

Sorry for being late
And yes! I have tried these Nice features and I have a simple IR simulation!
I'll clean the code base and share in the GitHub, hope many people come and help to improve it

A big problem I wanted to deal with was this, in real life different points of a single object have different temperature so different colours in the IR view, but in other simulation I had tried before I could not find a good way to have different colours and temps in a single object which I did in the Issac Sim's IR camera

[reza-shahriari]

this is the github repo
https://github.com/reza-shahriari/TCIsaacSim/tree/master
could be nice if any one can help with it