Skip to content

Shared-memory groupby hangs when multi-column packing exceeds the single-column compatibility guard #23829

Description

@VaggelisGian

The shared-memory hash groupby can hang forever when the host-side compatibility check passes but the device-side packing cannot fit all aggregation columns in one pass. This is a follow-on to #17853: the guard added by PR #17851 checks only the single-column data footprint, while the kernel needs room for every column's data plus its validity mask plus 16-byte alignment padding.

Mechanism

is_shared_memory_compatible (cpp/src/groupby/hash/compute_single_pass_aggs.cu:26-36) accepts the shared-memory path when data_buffer_size >= size * GROUPBY_CARDINALITY_THRESHOLD for each column individually. The kernel's packing step calculate_columns_to_aggregate (cpp/src/groupby/hash/compute_shared_memory_aggs.cu:121-153) instead needs, for all columns together, sum_i(round_up16(width_i*C) + round_up16(C)) bytes where C is the block cardinality (up to 128). When that sum does not fit, the loop breaks without advancing and the driver while (col_end < num_cols) at line 302 has no progress condition, so it re-runs the identical break point forever; any single spinning block hangs the whole kernel.

For int64 aggregations at cardinality 128 one column needs round16(8*128) + round16(128) = 1152 bytes, but the guard only requires avail - offsets >= 1024. Any configuration with per-column slack in [1024, 1152) passes the host check and then spins.

Reproduced on RTX 5060 Ti (sm_120), libcudf 26.10 nightly

One int64 key column with all-distinct keys plus k int64 value columns aggregated with SUM:

rows k (number of value columns)
27648 868-870 hang, neighbors ok
23040 1082-1086 hang, neighbors ok
18432 1402-1406 hang, neighbors ok
13824 1934 hangs

Each hanging config deterministically times out (60 s); configs just outside the narrow windows return normally, matching the computed window boundary exactly. A cuda-gdb backtrace of a live hang shows the main thread blocked in cudaFree inside compute_single_pass_aggs, i.e. the launched single_pass_shmem_aggs_kernel never retires.

The same window arithmetic predicts hangs on H100/B200 at different occupancy points (for example 6 int64 aggregations at blocksPerSM=16 on 228KB parts).

Suggested fix

Tighten the host check to the true worst-case total the packer needs at cardinality 128: require sum_i(round_up16(width_i*128) + round_up16(128)) <= data_buffer_size (falling back to the existing global-memory path otherwise). That is conservative for blocks whose cardinality is below 128 but eliminates the spin.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions