Fix : RecursionError when using pipe.enable_sequential_cpu_offload()

[JVSCHANDRADITHYA]

This PR fixes Issue #8

This PR fixes a RecursionError that occurs when using enable_sequential_cpu_offload() with the FP8 wrapper.

Problem

The FP8 wrapper modifies module dtype inside the forward pass using .to()
When combined with accelerate hooks, this leads to recursive forward calls and a crash.

Fix

• Removed .to() calls from forward
• Avoided mutating model weights during execution
• Ensured original forward is correctly referenced
• Added guard to prevent double wrapping

Result

• Works correctly with sequential CPU offload
• No recursion errors during inference

Tested with:

• enable_model_cpu_offload (works)
• enable_sequential_cpu_offload (works)
• Checked external dependency for autocast_model_forward() to avoid ImportErrors (None Found)

Validation

• Reproduced RecursionError with enable_sequential_cpu_offload
• Verified successful inference after fix
• Tested with both CPU offload modes

ATTACHMENTS :

• Result with pipe.enable_sequential_cpu_offload()

• check for ext. dependency

[JVSCHANDRADITHYA]

Hi @sam-motamed and @ttchengab,

Wanted to check if this PR aligns with the intended approach... Will make changes if needed.
Thanks!