mutate breaks semantics: peephole treats memory.grow, table.grow and trapping instructions as pure #2767

With --preserve-semantics, the peephole mutator only rewrites subtrees whose DFG nodes all have the same color, and it switches to a new color around effectful instructions. A new color is started only after call, local.tee, stores, global.set and table.set. memory.grow and table.grow are pushed like pure values and never start a new color:

Compare with call, which calls new_color() after pushing its result:

Instructions that may trap (integer div/rem, loads) are also treated like pure values.

As a result, rules that reorder, drop or duplicate operands change the semantics:

• Reordering a memory.grow with memory.size or a load: all *-commutes rules, select-invert.
• Dropping an operand that may trap or grow: i32.or--1, i64.or--1, select-same-branches.
• Changing the number of memory.grows: i32.add-x-x, i64.add-x-x (both directions), i32.or-x-x, i64.or-x-x (both directions), and x => (and x x) in i32.and-x-x, i64.and-x-x.

((and x x) => x happens to be fine: memory.grow may always fail, so the original can also take the path where the first grow succeeds and the second fails, and then n & -1 = n gives the same result and the same memory size.)

Repro

Reordering (i32.add-commutes)

(module
  (memory 1 2)
  (func (export "f") (result i32)
    (i32.add (memory.size) (memory.grow (i32.const 1)))))

$ wasm-tools parse grow.wat -o grow.wasm
$ wasm-tools mutate --preserve-semantics -s 1048 grow.wasm -o grow-m.wasm
$ wasmtime run --invoke f grow.wasm
2
$ wasmtime run --invoke f grow-m.wasm
3

The mutant computes (i32.add (memory.grow (i32.const 1)) (memory.size)). The original can only return 2 (grow succeeds, 1 + 1) or 0 (grow fails, 1 + -1).

Dropping a trap (i32.or--1)

(module
  (func (export "f") (param i32) (result i32)
    (i32.or (i32.div_u (i32.const 1) (local.get 0)) (i32.const -1))))

$ wasm-tools parse trap.wat -o trap.wasm
$ wasm-tools mutate --preserve-semantics -s 177 trap.wasm -o trap-m.wasm
$ wasmtime run --invoke f trap.wasm 0
...
wasm trap: integer divide by zero
$ wasmtime run --invoke f trap-m.wasm 0
-1

The mutant is just i32.const -1.

Duplicating a grow (i32.and-x-x, x => (and x x))

(module
  (memory 1 3)
  (func (export "f") (result i32)
    (drop (memory.grow (i32.const 1)))
    (memory.size)))

$ wasm-tools parse dup.wat -o dup.wasm
$ wasm-tools mutate --preserve-semantics -s 379 dup.wasm -o dup-m.wasm
$ wasmtime run --invoke f dup.wasm
2
$ wasmtime run --invoke f dup-m.wasm
3

The mutant drops (i32.and (memory.grow (i32.const 1)) (memory.grow (i32.const 1))). The original can only end with 2 pages (grow succeeds) or 1 page (grow fails).

Suggested fix

• Call new_color() after MemoryGrow and TableGrow in dfg.rs, as for Call.
• For traps, either start a new color at instructions that may trap, or require that the dropped operand cannot trap in *.or--1 and select-same-branches. Reordering or duplicating trapping operands is fine once grows are handled, since both sides trap.

Version

• wasm-tools 1.261.0.