Skip to content

release(v0.5.0): shared host runtime, buffer safety, and debugging - #1613

Closed
elibol wants to merge 105 commits into
NVIDIA:release/v0.5.xfrom
elibol:release/v0.5.0-host-debug-integration
Closed

release(v0.5.0): shared host runtime, buffer safety, and debugging#1613
elibol wants to merge 105 commits into
NVIDIA:release/v0.5.xfrom
elibol:release/v0.5.0-host-debug-integration

Conversation

@elibol

@elibol elibol commented Oct 22, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

cuTile Rust v0.5.0 integration covering:

  • Shared CUDA contexts, streams, allocations, and async runtime across Tile and SIMT.
  • Safer buffer casts and ownership, tensor shape validation, and correct slice offsets.
  • Improved source locations for generated and inlined code, consistent device debug modes, and CUTILE_JIT_* overrides.
  • Correct binding layouts with libclang 22, FP8 capability checks, and benchmark smoke coverage.

These changes are already included in release/v0.5.x; this PR currently has no remaining file diff.

Superseded by #1744, from elibol:release/v050-tile-features into main.

Arthur031221 and others added 26 commits October 4, 2026 04:44
@Arthur031221 @elibol
fix: keep generated cuTile locations inside their containing functions
7738999 
Use captured module spans for source locations and the current function
definition for generated types, arrays and wrapper calls. Add regressions
using the assertion kernel from NVIDIA#1379 and a helper in another file.

Co-authored-by: Melih Elibol <1884921+elibol@users.noreply.github.com>
Signed-off-by: Arthur031221 <levi74108520963@gmail.com>
@elibol
fix(cutile): attribute generated entry setup to the user signature
77816f6 
Signed-off-by: Melih Elibol <elibol@users.noreply.github.com>
@elibol
cuda-core: safe element casts require AnyBitPattern; bool is DeviceCopy
f39c959 
DeviceBuffer::cast_elem, DeviceBuffer::cast_chunks, and Tensor::reinterpret
reinterpreted device bytes as any element type of the right size and
alignment, which let a u8 buffer become a DeviceBuffer<bool> or Tensor<bool>
holding values other than 0 and 1 (NVIDIA#1506). The casts now require
bytemuck::AnyBitPattern of their target, so bool is rejected at compile time,
and bool implements DeviceCopy: a typed buffer holds valid values by
construction, so bool buffers can be allocated, filled, launched, and read
back like any other element type. cuda-core re-exports bytemuck, implements
Pod for the DType newtypes, and enables half's bytemuck feature for f16 and
bf16.

The atomic wrappers in cuda-device are not Copy, so they cannot be
AnyBitPattern. DeviceBuffer gains wrap_elem and peel_elem, bounded on
bytemuck::TransparentWrapper, to view a scalar buffer as its atomic wrapper
and back. cuda-device implements TransparentWrapper for its atomic types and
Pod for its vector types, and the examples that cast into atomics use the new
methods. Every example lock gains the bytemuck dependency of cuda-device.

Signed-off-by: Melih Elibol <1884921+elibol@users.noreply.github.com>
@GiGiKoneti
fix(cutile-compiler): attribute short-circuit boolean constants to ex…
f468474 
…pression span

Short-circuit boolean operations (`a && b`, `a || b`) generate deciding
constants when the left-hand operand decides the outcome. Previously,
`compile_bool_constant` created constants from synthesized `bool` type
spans, resolving to fallback module-header locations.

Pass the logical expression's span through `compile_bool_constant` and the
shared constant builder (`compile_constant_from_exact_bounds`), ensuring
the resulting constant operation receives the exact expression line while
preserving inlined helper scopes and call chains.

Add regression test in `debug_info.rs` verifying source line attribution
and inline caller provenance for both direct and inlined expressions.

Part of NVIDIA#1475.

Signed-off-by: GiGiKoneti <gigikoneti@gmail.com>
@tpn
Use NVVM debug metadata merge behavior for native LTO
afeb8b8 
Signed-off-by: Trent Nelson <trent@trent.me>
@almightychang
refactor(cutile): select debug or optimized device code as one Codege…
b76010f 
…nMode

`tileiras` accepts `--device-debug` only at optimization level 0, but
`CompileOptions` kept the level and the debug flag as independent fields.
Debug builds default to full debug, so adding only `.opt_level(3)` produced
`--opt-level 3 --device-debug`, which `tileiras` rejects at JIT time.

Replace `opt_level`, `device_debug`, and `lineinfo` with
`codegen: CodegenMode`, where `Debug` owns level 0 and
`Optimized { level, lineinfo }` covers the rest. Every value now maps to
flags `tileiras` accepts. `debug_info()` and `opt_level()` select a mode,
and the last call wins. The level-0 default rule in
`TileirasOptions::from_compile_options` is gone, and the frontend reads the
same mode to decide check hoisting.

Remove the `device_debug(bool)` and `lineinfo(bool)` setters, which
`debug_info()` covers. The cache key and entry header bytes are unchanged
for every mode; a test pins them against the previous encoding.

Signed-off-by: almightychang (Sam, Joochul Chang) <almightychang@icloud.com>
@almightychang
feat(cutile): add CUTILE_JIT_* run-time compile overrides
b461498 
Nothing could switch an already-built application to a debug or sanitizer
build without changing its code. Add three process-wide variables, read
once per process:

- CUTILE_JIT_DEBUG=none|line|full acts like `debug_info()`
- CUTILE_JIT_OPT_LEVEL=0|1|2|3 acts like `opt_level()`
- CUTILE_JIT_SANITIZE=memcheck|none acts like `sanitize_memcheck()`

Each applies after the code's options, as if its setter were called last.
`CompileOptions::with_jit_env_overrides()` applies them once before both
compile stages, so the frontend's check placement and the `tileiras` flags
always come from the same options. Invalid values, and full debug together
with an explicit level, return a JIT error naming the variable.

This continues NVlabs/cutile-rs#237, which applied the overrides only to
the `tileiras` flags and left the frontend hoisting checks under
`CUTILE_JIT_OPTIMIZATION=debug`.

Signed-off-by: almightychang (Sam, Joochul Chang) <almightychang@icloud.com>
@elibol
Merge main into bool and element-cast safety fix
37c540e 
Signed-off-by: Melih Elibol <1884921+elibol@users.noreply.github.com>
@elibol
Fix atomic buffer representation and zeroed initialization safety
f42ed4f 
Signed-off-by: Melih Elibol <1884921+elibol@users.noreply.github.com>
@elibol
Refresh remaining SIMT example locks for bytemuck
787e241 
Signed-off-by: Melih Elibol <1884921+elibol@users.noreply.github.com>
@elibol
Align DeviceCopy derive diagnostics with the validity contract
228652d 
Signed-off-by: Melih Elibol <1884921+elibol@users.noreply.github.com>
@elibol
fix: enforce Tile argument contracts and pinned host alignment
2788c6c 
Signed-off-by: Melih Elibol <1884921+elibol@users.noreply.github.com>
@elibol
ci: validate interop stack against the v0.5.0 API boundary
98c023d 
Signed-off-by: Melih Elibol <1884921+elibol@users.noreply.github.com>
@elibol
refactor(cuda-core): share host handles and allocations across Tile a…
70f5676 
…nd SIMT

Signed-off-by: Melih Elibol <1884921+elibol@users.noreply.github.com>
@elibol
fix(cuda-core): complete context migration in Tile consumers
f0f61e2 
Signed-off-by: Melih Elibol <1884921+elibol@users.noreply.github.com>
@elibol
docs: order shared host imports consistently
8002432 
Signed-off-by: Melih Elibol <1884921+elibol@users.noreply.github.com>
@elibol
refactor: share the async host runtime across Tile and SIMT
80d9e7a 
Signed-off-by: Melih Elibol <1884921+elibol@users.noreply.github.com>
@elibol
fix(cuda-async): bound the GEMM output scalar for shared launches
fcfe932 
Signed-off-by: Melih Elibol <1884921+elibol@users.noreply.github.com>
@elibol
fix(cuda-macros): marshal async slice arguments through the shared la…
a9f81c5 
…uncher

Signed-off-by: Melih Elibol <1884921+elibol@users.noreply.github.com>
@tpn
Preserve inline scopes in line-table debug info
ac90d9e 
Retain the source-scope map when line tables are enabled so native
debug metadata includes inline call chains without variable debug info.

Signed-off-by: Trent Nelson <trent@trent.me>
@Arthur031221
test: pin the locations of generated cuTile constants
a450156 
Check the file, definition line, subprogram and caller chain of the
generated constants in the assertion kernel and in a nested cross-file
helper, in Line and Full, and that the same locations are present in the
serialized bytecode. A user literal in the same kernel must keep its own
line. An opt-in test checks the cubin line table with nvdisasm and DWARF.

Signed-off-by: Arthur031221 <levi74108520963@gmail.com>
@almightychang
test(cutile): isolate CUTILE_JIT_* cases and compare modes across pro…
765cb1f 
…cesses

Every override case now runs in a child process that inherits no
CUTILE_JIT_* setting and is selected by a dedicated marker variable. The
cases cover explicit none, line with an opt level, full, an invalid value,
the full/opt-level conflict, and the read-once rule.

The cache-key check compares keys reported by independent processes, each
under its own overrides, so a key that dropped the effective mode would
let two distinct modes collide. A GPU test runs the normal launcher
against one shared disk store and checks the kernel result, the check
placement, and whether stage 2 came from tileiras or the store.

Signed-off-by: almightychang (Sam, Joochul Chang) <almightychang@icloud.com>
@nihalpasham
fix(cuda-core): preserve shared read access on empty and peeled Devic…
eec8776 
…eBuffers

Signed-off-by: nihalpasham <nihalp@nvidia.com>
@nihalpasham
fix(cuda-async,cuda-host): preserve concurrent read permissions on De…
c68eee0 
…viceBox::from_raw_parts and update docs

Signed-off-by: nihalpasham <nihalp@nvidia.com>
@elibol
fix(cutile): retain source origins for generated code and host checks (…
08b9d85 
…NVIDIA#1602)

Preserve source locations and inline caller chains for const-generic arrays,
generated reduction bodies, and checks relocated to the host. Keep deduplicated
check origins private to the compiler and render them through the existing
diagnostic API.

Part of NVIDIA#1475. The remaining frontend items and debugger/backend fixes remain
tracked separately; this does not close the full source-location audit.

Validated with the CPU/assembler suites, source-location regressions, live
CUDA-GDB tests, GPU host-check enforcement, host/Tile semver checks, and all
25 active CI checks on 825de3e.

Signed-off-by: Melih Elibol <1884921+elibol@users.noreply.github.com>
(cherry picked from commit 9718b43)
Signed-off-by: Melih Elibol <1884921+elibol@users.noreply.github.com>
@elibol
ci: validate pull requests targeting release branches
b8c263e 
Signed-off-by: Melih Elibol <1884921+elibol@users.noreply.github.com>
elibol added 2 commits October 30, 2026 11:59
@elibol
Merge atomic example compatibility fix
95a51d7 
Signed-off-by: Melih Elibol <1884921+elibol@users.noreply.github.com>
@elibol
Merge atomic example compatibility fix
c6c5a86 
Signed-off-by: Melih Elibol <1884921+elibol@users.noreply.github.com>
@elibol

elibol commented Oct 30, 2026

Copy link
Copy Markdown
Collaborator Author

/ok to test c6c5a86

elibol added 11 commits October 30, 2026 12:38
@elibol
Merge main after entry-wrapper debug fix; preserve both changelog ent…
c5ea583 
…ries

Signed-off-by: Melih Elibol <1884921+elibol@users.noreply.github.com>
@elibol
Merge refreshed safe-cast prerequisite after debug fix lands
c16fa3f 
Signed-off-by: Melih Elibol <1884921+elibol@users.noreply.github.com>
@elibol
Merge refreshed shared-core prerequisite after debug fix lands
eddfe14 
Signed-off-by: Melih Elibol <1884921+elibol@users.noreply.github.com>
@elibol
Record merged entry-wrapper debug fix in release candidate ancestry
4458979 
Signed-off-by: Melih Elibol <1884921+elibol@users.noreply.github.com>
@elibol
Record accepted safe-cast squash merge without changing the reviewed …
ba76442 
…tree

Signed-off-by: Melih Elibol <1884921+elibol@users.noreply.github.com>
@elibol
Record accepted safe-cast ancestry through shared core
ede4267 
Signed-off-by: Melih Elibol <1884921+elibol@users.noreply.github.com>
@elibol
Record accepted safe-cast merge in release candidate ancestry
5270acc 
Signed-off-by: Melih Elibol <1884921+elibol@users.noreply.github.com>
@elibol
Record accepted shared-core squash merge without changing the reviewe…
21bc9bb 
…d async tree

Signed-off-by: Melih Elibol <1884921+elibol@users.noreply.github.com>
@elibol
Record accepted shared-core merge in release candidate ancestry
10c5e9c 
Signed-off-by: Melih Elibol <1884921+elibol@users.noreply.github.com>
@elibol
Merge accepted shared async runtime into v0.5.0 candidate
5676dc0 
Signed-off-by: Melih Elibol <1884921+elibol@users.noreply.github.com>
@elibol
Integrate accepted slice-offset, FP8 and benchmark fixes for v0.5.0
dfd6867 
Merge main through NVIDIA#1541, NVIDIA#1683 and NVIDIA#1634. Resolve workflow conflicts using the accepted NVIDIA#1580 copy-pr-bot triggers.

Signed-off-by: Melih Elibol <1884921+elibol@users.noreply.github.com>
@elibol

elibol commented Oct 30, 2026

Copy link
Copy Markdown
Collaborator Author

/ok to test dfd6867

elibol added 2 commits October 30, 2026 16:31
@elibol
fix(cuda-bindings): resolve complete structs with libclang 22
8b38fb8 
Signed-off-by: Melih Elibol <1884921+elibol@users.noreply.github.com>
@elibol
Integrate libclang 22 binding layout fix for v0.5.0 SWQA
54be19d 
Signed-off-by: Melih Elibol <1884921+elibol@users.noreply.github.com>
@elibol

elibol commented Oct 30, 2026

Copy link
Copy Markdown
Collaborator Author

/ok to test 54be19d

elibol added 2 commits October 30, 2026 16:42
@elibol
fix(ci): align example rustc-hash pins after bindgen upgrade
0818e44 
Signed-off-by: Melih Elibol <1884921+elibol@users.noreply.github.com>
@elibol
Integrate example lockfile alignment for v0.5.0 CI
2c01bfb 
Signed-off-by: Melih Elibol <1884921+elibol@users.noreply.github.com>
@elibol

elibol commented Oct 30, 2026

Copy link
Copy Markdown
Collaborator Author

/ok to test 2c01bfb

elibol added 3 commits October 30, 2026 17:14
@elibol
fix(ci): run bindgen regression only on compiler-equipped runners
3f726ca 
Signed-off-by: Melih Elibol <1884921+elibol@users.noreply.github.com>
@elibol
Merge accepted tensor-shape validation into final v0.5.0 fix
b216c9d 
Signed-off-by: Melih Elibol <1884921+elibol@users.noreply.github.com>
@elibol
Integrate final binding-test CI fix and accepted tensor-shape validation
e16b73a 
Signed-off-by: Melih Elibol <1884921+elibol@users.noreply.github.com>
@elibol

elibol commented Oct 31, 2026

Copy link
Copy Markdown
Collaborator Author

/ok to test e16b73a

@elibol
Integrate accepted v0.5.0 fixes and deduplicate release changelog
d2c8120 
Signed-off-by: Melih Elibol <1884921+elibol@users.noreply.github.com>
@elibol

elibol commented Oct 31, 2026

Copy link
Copy Markdown
Collaborator Author

/ok to test d2c8120

@elibol elibol mentioned this pull request Oct 31, 2026
Open
6 tasks
@elibol elibol changed the title release(v0.5.0): integrate shared host runtime and source-location fixes release(v0.5.0): shared host runtime, buffer safety, and debugging Nov 4, 2026
@elibol elibol closed this Nov 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@nihalpasham nihalpasham Awaiting requested review from nihalpasham
@kkraus14 kkraus14 Awaiting requested review from kkraus14
@rparolin rparolin Awaiting requested review from rparolin
@binarybana binarybana Awaiting requested review from binarybana
@cryos cryos Awaiting requested review from cryos
@roivanov roivanov Awaiting requested review from roivanov

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

cutile rust v0.5.0

Development

Successfully merging this pull request may close these issues.

6 participants

@elibol @Arthur031221 @GiGiKoneti @tpn @almightychang @nihalpasham