ATen op coverage testing framework for Neuron using the NeuronOpInfo shape registry.
Tests all ~1362 target ATen ops for coverage, performance, and correctness on Neuron. Uses an explicit shape registry — every op either has registered shapes or is marked pending_registry.
tests/vertical/
├── README.md
├── __init__.py
├── conftest.py # Aggregates coverage CSV, session hooks, shape-selection opts
├── internal/ # NeuronOpInfo framework
│ ├── __init__.py
│ ├── target_ops.py # Curated list of ~1362 target ATen ops
│ ├── neuron_shape_registry.py # Op-specific shape registry (CSV/JSON backed)
│ ├── neuron_shape_defaults_generated.py # Auto-generated shapes + smoke test subsets
│ ├── neuron_op_info.py # NeuronOpInfo extending PyTorch OpInfo
│ ├── neuron_op_db.py # build_neuron_op_db() merging all sources
│ ├── generate_op_shape_registry.py # Tool: cache keys → op_shape_registry.json
│ ├── test_generate_op_shape_registry.py # Unit tests for the parser
├── accuracy/ # Op-accuracy suite (normwise relative error vs. a flat 1-ULP budget)
│ ├── conftest.py # ROW_FIELDS CSV aggregation, --dtypes/--seed-base
│ ├── budgets.py # The threshold and which groups it applies to
│ ├── formats.py # NumericFormat: unit roundoff + reference upcast
│ ├── metrics.py # norms, relative-error conversion, cell id, seeding
│ ├── test_metrics.py # Phase 1: the error model, CPU-only, no device
│ └── test_accuracy.py # Phase 2: on-device sweep, one row per cell
├── test_target_ops_coverage.py # Eager coverage + perf + correctness
├── test_target_ops_compile_coverage.py # torch.compile coverage + perf + correctness
├── test_dynamic_aten_ops.py # dynamic-shape NKI kernels perf (compared to static) + correctness
# Run all eager coverage tests in parallel (recommended)
python -m pytest tests/vertical/test_target_ops_coverage.py -n 32 --tb=no -q
# Run all compile coverage tests in parallel
python -m pytest tests/vertical/test_target_ops_compile_coverage.py -n 32 --tb=no -q
# Run a single op (eager)
python -m pytest "tests/vertical/test_target_ops_coverage.py::TestTargetOpsCoverage::test_op[aten::linear[linalg]]" -v -s
# Run a single op (compile)
python -m pytest "tests/vertical/test_target_ops_compile_coverage.py::TestTargetOpsCompileCoverage::test_op_compile[aten::linear[linalg]]" -v -s
# Run ops matching a pattern
python -m pytest tests/vertical/test_target_ops_coverage.py -v -k "relu or gelu"
# Run all dynamic NKI kernels
python -m pytest tests/vertical/test_dynamic_aten_ops.py -n 8 --tb=no -q
# Run a single op
python -m pytest "tests/vertical/test_dynamic_aten_ops.py::TestDynamicAtenOps::test_op[aten::mm]" -v -s
python -m pytest "tests/vertical/test_dynamic_aten_ops.py::TestDynamicAtenOps::test_op[aten::_grouped_mm]" -v -s
# Run ops matching a pattern
python -m pytest tests/vertical/test_dynamic_aten_ops.py -v -k "silu or mm"
# Limit each op to its N largest shapes (default 10)
python -m pytest "tests/vertical/test_dynamic_aten_ops.py::TestDynamicAtenOps::test_op[aten::mm]" --max-shapes-per-op 2 -v -s
neuron-profile artifacts (ntff + NEFFs) are written under aten_profiles/<cell>/ in the working directory (override with AOP_PROFILE_DIR); the path is recorded per row in the CSV's profile_rel_path column.
Measures each op's normwise relative error, expressed in units of the input format's unit
roundoff (unit = 2**-precision; bf16 → 2**-8). The check is
max_rel_err_units <= budget_units, where
max_rel_err_units = (max|T-R| / max|R|) / unit
The budget is one flat constant: 2.0 units = 1 full ULP = the output format's machine
epsilon (bf16 → 2**-7).
Only formats coarser than fp32 are gated. At fp32 the accumulator is the input format,
so accumulation rather than rounding sets the scale and no constant bounds it — a correct
fp32 reduction over 4096 values drifts ~64 units. Those cells report no_gate.
# Phase 1 -- the error model only. No device, unit test for accuracy related classes.
python -m pytest tests/vertical/accuracy/test_metrics.py -q
# Phase 2 -- on-device sweep against a model's traced op+shape list. --max-shapes-per-op 0
# sweeps every registered shape; the dtype sweep defaults to both bf16 and fp32.
# qwen3_235b_ep64_tp4 is traced BOTH ways (eager + compiled); one run writes one CSV per
# path: op_accuracy_eager_<ts>.csv and op_accuracy_compiled_<ts>.csv.
python -m pytest tests/vertical/accuracy/test_accuracy.py \
--traced-model qwen3_235b_ep64_tp4 --max-shapes-per-op 0 -n 32 --tb=no -q
# One dtype only
python -m pytest tests/vertical/accuracy/test_accuracy.py \
--traced-model qwen3_235b_ep64_tp4 --max-shapes-per-op 0 --dtypes bfloat16 -n 32 --tb=no -q
# A single op, verbose
python -m pytest "tests/vertical/accuracy/test_accuracy.py::TestOpAccuracy::test_op[aten::bmm]" \
--traced-model qwen3_235b_ep64_tp4 --max-shapes-per-op 0 -s --log-cli-level=INFO
# Include shapes above the shared 512Mi-element guard (NeuronOpInfo._MAX_TENSOR_ELEMS).
# No entry in qwen3_235b_ep64_tp4 hits it -- the largest tensor arg in the set is the 64-expert
# MoE weight stack at 64x1536x4096 = 402M elements -- so this is a knob for future sets.
NEURON_MAX_TENSOR_ELEMS=700000000 python -m pytest tests/vertical/accuracy/test_accuracy.py \
--traced-model qwen3_235b_ep64_tp4 --max-shapes-per-op 0 -n 32 --tb=no -q--traced-model is required — with no model the sweep fails loudly. It does two things —
registers that set's shapes, and restricts the run to them. Both are needed: registering alone
would leave the op's builtin shapes in the pool as well, so the sweep would report more cells
than the set has entries and would not be purely model-shaped. --model-shapes-only excludes
builtin but cannot separate two model sets registered together; the traced model selects
exactly one.
--max-shapes-per-op caps the shapes swept per op: 0 sweeps all registered shapes; a
positive n keeps only the n largest-tensor shapes per op. --dtypes defaults to both
bf16 and fp32; pass a comma-separated subset to narrow it.
This gates on both metrics. Each (op, dtype) is its own item; it fails if any cell is
over/invalid on the rel-err budget (bf16 only — fp32 has no rel-err verdict, no_gate) or
on the Frobenius ratio (every dtype). Every cell still emits a CSV row regardless. A cell
with an empty verdict — a device/lowering error (failed_lowering, …), no_input_sample,
unresolved, no_reference — is shown as other in the panel (never folded into a pass) but
is not gated: op runnability/lowering is owned by the vertical coverage suites
(test_target_ops_*); this accuracy suite only judges onboarded ops that actually run.
Runs as a first-class trn2 check in PR Checks (pull.yml, peer to the ATen/CPP suites) and
in Dependency Guard (dependency-guard.yml, blocks stable-image promotion). Each job runs
one command: --traced-model qwen3_235b_ep64_tp4 --max-shapes-per-op 0 over both dtypes.
NEURON_ACCURACY_OUTPUT_DIR=outputs/accuracy puts the
per-path CSV(s) + a step_summary.md metrics panel under the uploaded artifact; the panel is
posted to the GitHub job summary via the append-step-summary action. It has two head-level
metrics — relative error (gates bf16) and Frobenius ratio (gates every dtype) — each
split by dtype into a Tests (ops+shapes) and an Ops table over
pass/fail/xfail/skip/xpass/other, plus per-(metric, dtype) failing-cell tables.
| Option | Default | Description |
|---|---|---|
--dtypes |
bfloat16,float32 |
Comma-separated input dtypes to sweep. fp32 has no rel-err verdict (no_gate) but is gated on the Frobenius ratio |
--seed-base |
0 |
Re-roll every cell's input draw while keeping cells independent |
One row per (op, shapes, dtype, path) cell, aggregated into one CSV per capture path (op_accuracy_eager_<timestamp>.csv / op_accuracy_compiled_<timestamp>.csv; a single unsplit run writes op_accuracy_<timestamp>.csv). unit
is carried alongside the measurement so a verdict can be re-read from the row without
re-running the op, and arg_shapes so anything shape-derived (reduction width, element count)
stays recomputable without re-running it either.
| Column | Description |
|---|---|
cell_id |
16-hex-char id of (op, arg_shapes, dtype). Excludes the seed — the seed is derived from it |
op / op_group |
Full ATen op name; TARGET_ATEN_OP_GROUP category |
op_mode |
Which capture mode saw the op name in the capture: eager_only / compile_only / both_modes. Empty for subsets that declare no mode partition. Distinct from op_group, which sets the budget |
dtype |
Input dtype the cell was measured at |
path |
Capture path the cell came from (eager / compiled); the split key for the per-path CSVs |
arg_shapes |
JSON of the registry entry used. The source for any shape-derived figure — the suite no longer extracts a reduction width itself |
unit |
The input format's unit roundoff 2**-precision = 1/2 ULP (bf16 → 0.00390625). Multiply max_rel_err_units by this to get the raw relative error |
max_rel_err_units |
The measurement. Normwise relative error, L-inf: (max|T-R| / max|R|) / unit |
budget_units |
The threshold. 2.0 for arithmetic, 0.0 for bit-exact groups, empty when not gated |
relative_norm_verdict |
within, over, invalid, or empty when status != ok |
max_abs_err / max_abs_ref |
The two raw absolute scalars max_rel_err_units is computed from |
frobenius_abs_err_base / frobenius_ratio / frobenius_ratio_verdict |
Frobenius-ratio metric: denominator ‖cpu_low − ref_high‖, the ratio ‖trn_low − ref_high‖ / ‖cpu_low − ref_high‖, and its verdict (within/over/invalid, gate 1.1) |
seed |
RNG seed, derived from cell_id + --seed-base |
gates |
JSON of the shape-selection knobs in effect (max_shapes_per_op, shape_registry, traced_model, ...) |
status |
See below |
error_cause_raw |
First 200 chars of the failure, when there is one |
| Verdict | Meaning |
|---|---|
within |
max_rel_err_units <= budget_units — the worst element is inside one ULP of the fp32 reference |
over |
Exceeded the budget. Includes inf, which is deliberate: an error against an exact-zero reference, or a bit-exact op that did not reproduce exactly |
invalid |
max_rel_err_units is nan — the output held a nan, or both sides overflowed to inf. Not a comparison, so not counted either way |
| Status | Meaning |
|---|---|
ok |
Cell measured; read relative_norm_verdict |
no_gate |
Measured, but the dtype is fp32 or finer, where a flat rounding bound is not the right instrument. Stays countable rather than passing silently |
excluded |
random group — two correct RNG implementations disagree elementwise |
no_input_sample |
The op's schema could not be satisfied from the registry entry |
no_reference |
The CPU fp32 reference could not run, or its output arity disagreed with the device's |
unresolved |
Op name did not resolve to a callable |
failed_lowering |
Device allocation or execution failed (see error_cause_raw) |
Distributed all-gather benchmark. Each cell is one (config, shape) swept in both dynamic (NKI kernel, NEURON_DYN_NKI_COLLECTIVES=1) and torch (native) modes, each in its own cold torchrun job. Configs are defined in _CONFIGS: tp (contiguous group, nproc=4) and fsdp (stride-TP group, nproc=64).
# Run all cells (both configs). fsdp launches 64 ranks, so run serially (no -n):
python -m pytest tests/vertical/test_dynamic_collectives.py -v -s
# Run only the tp config (nproc=4) in parallel; each xdist worker owns a disjoint
# block of AG_CORE_BLOCK cores. Keep -n * AG_CORE_BLOCK within the logical-core
# budget (64 on TRN2 at LNC=2):
AG_CORE_BLOCK=4 python -m pytest tests/vertical/test_dynamic_collectives.py -k tp -n 4 -v -s
# Run a single cell
python -m pytest "tests/vertical/test_dynamic_collectives.py::test_dynamic_collectives[all_gather-tp-8192x4096-bfloat16]" -v -s -n 1
# or directly as a driver (launches the torchruns itself):
NEURON_LOGICAL_NC_CONFIG=2 AG_CSV=/tmp/ag.csv python3 tests/vertical/test_dynamic_collectives.py
neuron-profile artifacts (ntff + NEFFs) are written under c10d_profiles/<h>x<w>/<mode>/ in the working directory (override with AG_PROFILE_DIR).
Results are aggregated into:
- Eager:
target_ops_coverage_<timestamp>.csv - Compile:
target_ops_compile_coverage_<timestamp>.csv
| Column | Description |
|---|---|
op_name |
Full ATen op name (e.g., aten::add.Tensor) |
op_group |
Category: elementwise, linalg, nn, reduction, indexing, etc. |
status |
supported, missing, failed_lowering, pending_registry |
supported_by |
registered, composite, variant_covered, unknown |
input_source |
shape_registry_model, shape_registry_default, pytorch_db, pending_registry |
input_element_count |
Total input element count of the slowest-executing shape (empty when no shape ran) |
neuron_primitive_ops |
ATen ops executed on Neuron |
fallback_primitive_ops |
ATen ops that fell back to CPU |
time_to_first_ms |
Time to first result (ms) |
exec_avg_ms / exec_min_ms / exec_max_ms |
Execution time stats |
error |
Error message if failed |
last_updated |
Timestamp |
correctness_test |
pass, fail, or empty |
Per-op rows are merged into one timestamped dynamic_aten_ops_<timestamp>.csv. Each row is one (op, shape, dtype) cell profiled in both dynamic (NKI kernel) and static (default lowering) modes.
| Column | Description |
|---|---|
op_name |
Full ATen op name (e.g., aten::add.Tensor) |
input_shape |
Tensor-arg dims joined with x, args with _ (scalar args skipped) |
dtype |
Float dtype of the swept sample (e.g., bfloat16) |
input_size_mb |
Total input size (sum of input-tensor bytes) in MiB |
dynamic_time_to_first_ms / static_time_to_first_ms |
Cold time-to-first (compile + load + execute), per mode |
dynamic_e2e_time_us / static_e2e_time_us |
Warm wall-clock latency incl. host overhead, per mode |
e2e_ratio |
dynamic_e2e / static_e2e (>1 ⇒ dynamic slower) |
dynamic_device_time_us / static_device_time_us |
On-device wall-clock time (neuron-profile total_time), per mode |
device_ratio |
dynamic_device / static_device |
correctness_test |
pass / fail — device output vs CPU reference |
last_updated |
Timestamp |
error |
Error message, or an informational note (e.g. fell back when the dynamic kernel declined and the op ran via the default lowering) |
Per-cell rows are merged into one timestamped dynamic_collectives_<timestamp>.csv. Each row is one (config, shape) all-gather cell profiled in both dynamic (NKI kernel) and torch (native) modes. Tiling columns (regime, *_tile_h, n_*, unroll) are mode-independent and taken from the dynamic worker's on-device plan.
| Column | Description |
|---|---|
op_name |
Collective op (all_gather) |
total_mib |
Gathered size in MiB (world * h * w * elem_size / 2^20) |
input_h / input_w |
Per-rank shard shape (h = seq, w = last dim) |
replica_group |
Ranks in this cell's all-gather group (== replica_groups[0]); stride-TP for fsdp (e.g. [0, 4, ..., 60]), contiguous otherwise |
regime |
Tiling regime name, or native-fallback when the plan is ineligible |
large_tile_h / medium_tile_h / small_tile_h |
Element tile sizes (pow2) the NEFF is baked with |
n_large / n_medium / n_small / n_rem |
Tile counts on the flat numel (n_rem = single small tail, 0 or 1) |
remainder_row_count |
Leftover elements below one small tile |
unroll |
Unroll variant (1/2/4) auto-picked from the tile count; 0 = native |
dynamic_e2e_us / torch_e2e_us |
Warm wall-clock latency incl. host overhead, per mode |
e2e_ratio |
dynamic_e2e / torch_e2e (>1 ⇒ dynamic slower) |
dynamic_device_us / torch_device_us |
On-device wall-clock time (neuron-profile total_time), per mode |
device_ratio |
dynamic_device / torch_device |
dynamic_cc_us / torch_cc_us |
On-device collective time (neuron-profile cc_op_time), per mode |
dynamic_bus_bw_gbps / torch_bus_bw_gbps |
Bus bandwidth (GiB/s) derived from device time (busbw = algbw * (n-1)/n) |
correctness_test |
pass / fail — gathered output vs locally-reconstructed reference |
last_updated |
Timestamp |
- Shape registry (
internal/neuron_shape_registry.py) — explicit shapes per op - PyTorch op_db — for base ops and
.default/.outoverloads only pending_registry— no input source, op is skipped
Register in internal/neuron_shape_registry.py or internal/neuron_shape_defaults_generated.py.
Entries added at the top level of register_all_defaults join the builtin pool every suite
sees; to add shapes that only one sweep picks up, put them in a smoke test subset instead (see
Accuracy smoke test subsets).
SHAPE_REGISTRY.register(
"aten::my_op",
arg_shapes={"self": (256, 4096), "weight": (4096, 4096), "bias": None},
source="manual",
)For ops needing specific dtypes:
SHAPE_REGISTRY.register(
"aten::my_index_op",
arg_shapes={
"self": (256, 4096),
"index": {"shape": (64,), "dtype": "long", "low": 0, "high": 256},
"mask": {"shape": (256, 4096), "dtype": "bool"},
},
source="manual",
)Scope. A subset is a small frozen op set for accuracy smoke tests. This is not a model onboarding framework. Do not add a model here to get accuracy coverage for that model. The first subset comes from a Qwen3-235B EP64/TP4 capture, You can repurpose or extend the subset, but the intent is one small frozen subset.
The op+shape list of a subset is checked in to
internal/neuron_shape_defaults_generated.py, alongside the builtin defaults, so there is
one registry source rather than a second file to keep in sync. Each subset contributes four
module-level names:
| Name | Purpose |
|---|---|
SMOKE_SUBSET_SHAPES[<name>] |
The register function. <name> is what --traced-model accepts |
SMOKE_SUBSET_OPS[<name>] |
The frozenset of ops the subset actually traced |
SMOKE_SUBSET_OP_MODE[<name>] |
Optional. {op: mode} for subsets that partition by capture mode; drives the op_mode CSV column |
<NAME> |
The source tag its entries carry, so SHAPE_REGISTRY.get(op, model=<NAME>) works |
qwen3_235b_ep64_tp4 — 328 unique entries over 89 ops, from an end-to-end 64-rank capture of
qwen3_235b_6layers_ep64_nle1_tp4_neuron_ep (EP64 / 64 experts / top-8, TP4 × FSDP16,
SelectiveAC, seq 2048) on trn3.
The ops capture kit is on origin branch opcap/qwen3-235b-ep64, that branch's tests/vertical/tools/opcap/README.md has the procedure, the regeneration steps and the known gaps:
git fetch origin opcap/qwen3-235b-ep64
git show 16c27852:tests/vertical/tools/opcap/README.mdThe subset is categorized on two independent axes.
Axis 1 — op_mode, per op. Which capture mode ever saw the op name at all:
| mode | meaning | ops |
|---|---|---|
eager_only |
seen in eager dispatch, never in a post-AOT graph | 33 |
compile_only |
post-AOT graphs only — introduced by AOT, never in model source | 10 |
both_modes |
seen in eager dispatch and in a post-AOT graph | 46 |
The full per-op attribution is in tools/opcap/README.md on the pinned
opcap/qwen3-235b-ep64 commit.
Axis 2 — capture path, per shape. Which capture path actually observed that exact call,
recorded per cell during capture and carried on the entry as OpShapeEntry.paths (eager /
compiled, or both). This is the split key for the per-path CSVs: a cell traced on both paths is
measured once per path, and accuracy/conftest.py routes the results to
op_accuracy_eager_<ts>.csv and op_accuracy_compiled_<ts>.csv accordingly. A run whose entries
carry no path at all (builtin shapes, no --traced-model) is unsplit and writes the single
op_accuracy_<ts>.csv.
One constraint: a shape cannot have a path the op was never seen on, so an eager_only op's entries
are all eager and a compile_only op's are all compiled. A both_modes op is a mix — some shapes
were only seen in compile and some shape only in eager. The path allows us to track back where the
specific op shape is traced from.
--max-shapes-per-op 0 is right for this set because the op shape registry is already pruned
with top-10 largest tensor cap, which leaves only 328 entries, and aten::bmm is declared in QWEN3_235B_EP64_TP4_NO_SHAPE_OPS, because the captured entries shapes is over the cap, so it
reports one no_input_sample in result CSV.
Registering a subset is opt-in and off by default. register_all_defaults(r, smoke_subsets=())
takes an allowlist, and --traced-model is the only thing that fills it. That makes the gate
fail safe: adding a subset to the generated module cannot change what any existing suite
measures until a run names it. Without the gate the new entries would join the shared
op-keyed pool and compete in the top-N-by-size cap in NeuronOpInfo._registry_entries,
silently displacing shapes the coverage suites measure today. An unknown --traced-model name
raises loudly.
Use SMOKE_SUBSET_OPS[<name>] — not SHAPE_REGISTRY.all_ops(model=<name>) — when you need the
list of ops a subset traced. propagate_to_compatible_overloads copies entries and their
source tag onto schema-compatible .out / .default overloads, so all_ops(model=)
reports those synthesized overloads too (more ops rather than the actually traced).
The qwen3_235b_ep64_tp4 block in internal/neuron_shape_defaults_generated.py is generated output of
tests/vertical/tools/opcap/gen_ep64_registry.py --model qwen3_235b_ep64_tp4, which lives
on origin branch opcap/qwen3-235b-ep64.
# commit 7b73b5dc5eb4e7f68dfdc925701d3d528dbc747f
git fetch origin opcap/qwen3-235b-ep64
git show 7b73b5dc:tests/vertical/tools/opcap/README.mdsha256 (first 16) of the inputs and generators, so a re-derivation can be checked against the
block without trusting the branch pointer (paths relative to tests/vertical/tools/opcap/):
| sha256[:16] | file |
|---|---|
3b0c326890d2a731 |
captures/qwen3_235b_ep64_tp4/opcap_shapes.json |
2902ae97407c846a |
captures/qwen3_235b_ep64_tp4/opcap_ops.json |
fcc04d5752454cfe |
gen_ep64_registry.py |
112d7e62bb36da88 |
gen_shape_registry.py |
To reproduce: check that commit out, python gen_ep64_registry.py --model qwen3_235b_ep64_tp4 -o /tmp/block.py, ruff format --line-length 100 /tmp/block.py, and diff against the block — it is
byte-for-byte identical. Nothing inside the block is hand-edited; regenerate and re-splice, then
update this pin. The capture procedure, the known gaps and the per-op mode attribution live in
that branch's tests/vertical/tools/opcap/README.md.
-
Run a model to generate compilation artifacts:
export TORCH_NEURONX_PRESERVE_COMPILATION_ARTIFACTS=1 torchrun --nproc-per-node=64 run_benchmarks.py --block qwen3_torchtitan --preset qwen3-8b-fsdp -
Generate JSON from the cache keys in the artifacts:
python -m tests.vertical.internal.generate_op_shape_registry \ /tmp/neuron_backend/neuronx-cc -o op_shape_registry.jsonOr from a cache key text file (one key per line):
python -m tests.vertical.internal.generate_op_shape_registry \ --cache-keys aggregate_cache_keys.txt -o op_shape_registry.json -
Run coverage tests with the generated shapes:
python -m pytest tests/vertical/test_target_ops_coverage.py -n 64 \ --shape-registry op_shape_registry.json --model-shapes-only --tb=no -q
The model test suite runs the HuggingFace and torchtitan models in CI. Each job
writes its Neuron compiler cache keys to a daily S3 folder. A generated function,
register_cache_key_shapes in internal/neuron_shape_defaults_generated.py,
holds the model shapes that this feed gives, one union per op, capped per op.
The shapes load by default, so a plain run tests the model shapes too. Turn them
off with --no-cache-key-shapes to isolate a regression:
python -m pytest tests/vertical/test_target_ops_coverage.py -n 64 \
--no-cache-key-shapes --tb=no -qRegenerate the shapes offline. Do not fetch S3 at test time. The tool rewrites
only the block between the sentinel comments in the generated file. The S3
location has no default because the bucket moves. Pass it with
--bucket/--prefix, or set CACHE_KEY_SHAPES_BUCKET/CACHE_KEY_SHAPES_PREFIX.
# From S3 (needs a read-only profile for the bucket):
python tests/vertical/internal/refresh_cache_key_shapes.py \
--bucket <bucket> --prefix <prefix> --profile <aws-profile>
# From a local folder of *_cache_keys.txt files:
python tests/vertical/internal/refresh_cache_key_shapes.py --from-dir <dir>Review the diff and commit the regenerated file.
| Option | Description |
|---|---|
--shape-registry <path> |
JSON shape registry file to load into SHAPE_REGISTRY, tagged model_trace. For a one-off trace that isn't checked in |
--no-cache-key-shapes |
Skip the committed cache-key model shapes (tagged cache_keys). The shapes load by default |
--traced-model <names> |
Comma-separated smoke test subset(s) (keys of SMOKE_SUBSET_SHAPES). Registers them and restricts the run to them. Omit it and none are registered. Stricter than --model-shapes-only, which only excludes builtin and so still mixes every registered subset together |
--model-shapes-only |
For ops with model-traced shapes, skip builtin/default shapes |
--max-shapes-per-op <n> |
Cap per op to the top-N entries by max tensor element count. 0 (or negative) disables the cap — sweep all shapes registered for the op |
from internal.neuron_shape_defaults_generated import (
SMOKE_SUBSET_OPS,
QWEN3_235B_EP64_TP4,
register_all_defaults,
)
from internal.neuron_shape_registry import SHAPE_REGISTRY, NeuronShapeRegistry
# The process-wide registry holds builtin defaults, plus whatever --traced-model named.
SHAPE_REGISTRY.models() # -> [] with no --traced-model; ["qwen3_235b_ep64_tp4"] with it
# A private registry with one subset registered on top of the builtin defaults.
reg = NeuronShapeRegistry()
register_all_defaults(reg.register, smoke_subsets=[QWEN3_235B_EP64_TP4])
# that subset's entries only, and the ops it traced (see the caveat above)
reg.get("aten::linear", model=QWEN3_235B_EP64_TP4)
SMOKE_SUBSET_OPS[QWEN3_235B_EP64_TP4]
# Load a one-off trace from JSON. Source defaults to the filename stem.
SHAPE_REGISTRY.load_json("llama7b_shapes.json")
# Load from / save to consolidated CSV
SHAPE_REGISTRY.load_csv("op_shapes.csv")
SHAPE_REGISTRY.save_csv("op_shapes.csv")| Status | Meaning |
|---|---|
supported |
Op runs on Neuron without CPU fallback |
missing |
Op not implemented on Neuron |
failed_lowering |
Op registered but execution fails (real Neuron bug) |
| Input Source | Meaning |
|---|---|
shape_registry_model |
Inputs from model-traced shapes (via --traced-model or --shape-registry) |
shape_registry_default |
Inputs from builtin/default shape registry entries |
pytorch_db |
Inputs from PyTorch's native op_db |
pending_registry |
No valid inputs available — needs shapes registered to test |
Note: When input_source=pending_registry, the status is inferred from dispatch classification (registered → supported, composite → supported, unknown → missing) rather than from actual execution.