fix(headless): Fix use-after-free crash when replay ends in headless mode in debug build

[bobtista]

Summary

• Fix use-after-free crash in headless replay caused by ParticleSystemManager::reset() deleting all particle systems while DrawModules still hold raw pointers to them. Use update() instead, which only cleans up finished systems.
• Set DX8Wrapper_IsWindowed to false in headless mode so ignoringAsserts() works correctly during shutdown after TheGlobalData has been destroyed.

Test plan

• Run headless replay to completion (-replay test.rep -headless -ignoreAsserts)
• Verify exit code 0, no crash dumps, no assertion popups
• Verify no EXCEPTION DUMP in debug log

[greptile-apps]

Greptile Overview

Greptile Summary

Fixes a use-after-free crash that occurs during headless replay mode in debug builds by addressing two related issues in the shutdown sequence.

The first fix changes GameClient::updateHeadless() to call ParticleSystemManager::update() instead of reset(). The original reset() method immediately deletes all particle systems, but DrawModules (like W3DTruckDraw, W3DTankDraw) hold raw pointers to these particle systems. When DrawModules are later destroyed during shutdown, they attempt to access the already-freed particle system memory, causing a crash. The update() method only cleans up finished particle systems, leaving active ones intact until they're properly cleaned up.

The second fix ensures DX8Wrapper_IsWindowed is set to false in headless mode during initialization. This ensures ignoringAsserts() returns true throughout the entire process lifetime, including during shutdown after TheGlobalData has been destroyed. Without this, assertion dialogs could appear during the shutdown sequence in headless mode, which is problematic for automated testing.

Confidence Score: 5/5

• This PR is safe to merge with minimal risk
• Both fixes are well-understood, targeted, and address specific crash scenarios. The change from reset() to update() is a conservative fix that prevents immediate deletion while still cleaning up finished particle systems. The DX8Wrapper_IsWindowed initialization fix is a simple flag setting that ensures consistent behavior in headless mode. The comprehensive inline comments explain the rationale clearly.
• No files require special attention

Important Files Changed

Filename	Overview
GeneralsMD/Code/GameEngine/Source/GameClient/GameClient.cpp	Changed ParticleSystemManager from reset() to update() to prevent use-after-free crash with DrawModule raw pointers
GeneralsMD/Code/Main/WinMain.cpp	Set DX8Wrapper_IsWindowed to false in headless mode to suppress assertion dialogs during shutdown

Sequence Diagram

sequenceDiagram
    participant Main as WinMain
    participant GD as GlobalData
    participant DX8 as DX8Wrapper
    participant GC as GameClient
    participant PSM as ParticleSystemManager
    participant DM as DrawModules

    Note over Main,DX8: Startup (WinMain.cpp)
    Main->>GD: Check m_headless
    alt headless mode
        Main->>DX8: Set DX8Wrapper_IsWindowed = false
        Note over DX8: Enables ignoringAsserts()<br/>throughout process lifetime
    else windowed mode
        Main->>Main: initializeAppWindows()
    end

    Note over GC,DM: Headless Replay Loop
    loop Each frame
        GC->>PSM: updateHeadless()
        PSM->>PSM: update() (NEW)
        Note over PSM: Only deletes finished<br/>particle systems
        Note over PSM,DM: DrawModules' raw pointers<br/>remain valid
    end

    Note over GC,DM: Shutdown
    Main->>GC: Cleanup game
    GC->>DM: Destroy DrawModules
    Note over DM: Safe: particle systems<br/>still exist if active
    GC->>PSM: Final cleanup

Loading

[xezon]

Fast shutdown is a common way to exit apps, but it is not a good sign to do this to avoid problems here. Where is the problem exactly?

[bobtista]

Fast shutdown is a common way to exit apps, but it is not a good sign to do this to avoid problems here. Where is the problem exactly?

Did a bunch of testing and found the crash was:
The crash was caused by GameClient::updateHeadless() calling TheParticleSystemManager->reset() each frame during headless replay playback.

1. updateHeadless() calls TheParticleSystemManager->reset() every frame
2. reset() deletes ALL ParticleSystems immediately (memory filled with DEADBEEF)
3. DrawModules (W3DTruckDraw, W3DTankDraw, W3DTankTruckDraw) still hold raw pointers to those ParticleSystems (m_dustEffect, m_dirtEffect, m_powerslideEffect)
4. When replay ends and game shuts down, DrawModule destructors call tossEmitters()
5. tossEmitters() calls m_dustEffect->destroy() on already-freed memory
6. CRASH when ParticleSystem::destroy() tries to set m_isDestroyed = true at offset 0x338 on DEADBEEF

Pushed two fixes - first, we use update() instead of reset(), which leaves active particles intact.

Second, I was getting memory leak debug asserts even with ignoreAsserts enabled. It seems that:
During shutdown, TheGlobalData is destroyed before the memory leak report assertion fires. The ignoringAsserts() function checks TheGlobalData->m_headless and TheGlobalData->m_debugIgnoreAsserts, but both fail when TheGlobalData is null. Additionally, DX8Wrapper_IsWindowed defaults to true and is never changed in headless mode (DX8 is never initialized), so the !DX8Wrapper_IsWindowed check also fails.

The second fix is to set DX8Wrapper_IsWindowed = false at startup in headless mode. There is no DX8 window in headless mode, so the variable should reflect that. The existing !DX8Wrapper_IsWindowed check in ignoringAsserts() then handles the entire process lifetime, including shutdown.

[Mauller]

After the changes in #2235 to use and search for ParticleID's instead of holding pointers, is this valid anymore?
Checking since your comment talks about the altered draw modules that now use ID's to lookup their particle systems.