Skip to the content.

Origin. PLI Labs research archive, file v100-research/deepseek-v4.1-flash/RANGE_AUDIT.md. Published 2026-11-18 in the Volta V100 research repository. Edited for publication. Machine names, private file paths, network addresses, session names and the operator’s name were replaced with generic terms (the V100 server, [private path], [address], the owner). Links to files that are not published were turned into plain text. No figure and no finding was changed. Dates inside the note are its own; later results are in the build pages of this repository.

Lane L-REF, generated 2026-10-07 03:22Z by tools/ds41_ref/range_audit.py from golden.py range statistics. Reference: official deepseek-ai/DeepSeek-V4.1-Flash @ 2cba9e42, FP32 layer-streamed transcription (tools/ds41_ref), batch 1, world_size 1.

Summary and recommendations (L-REF)

Data: 10 runs (prompts x modes x index-K variants), every prefill token and 31 greedy decode steps; max over all of them. Both numerics modes agree to within a few percent.

  1. Residual stream: BF16, never FP16. Max 3.625e+05 (> 65,504). The extremes sit on BOS tokens — position 0 and also BOS tokens generated mid-sequence (p2_code continuation) — as on V4.
  2. hc_pre collapse (sublayer input before RMSNorm) reaches 4.617e+05: it must be computed and held in FP32 (or BF16) until the norm; only the normed output may be FP16.
  3. MoE combine (routed + shared) and its all-reduce stay FP32 (max 2.283e+04 > 8,192). TP4 partials peak lower but the summed value does not fit the rule.
  4. Attention sublayer output may go FP16 at P5 (max 441 « 8,192).
  5. Per-expert outputs (port order, weight after w2): max 1.502e+04 vs the 16,384 rule — allowed, but with only 8 % margin on four prompts. L-REF recommendation: keep the FP32 epilogue (a clamp would be a model change, §4.3).
  6. Engram wkv output, q.k logits and indexer scores are far from any FP16 limit (tables below).

Runs

prompt mode prompt tokens decode steps provisional
p0_smoke ref-exact 6 31 False
p0_smoke v100-semantic 6 31 False
p1_text ref-exact 85 31 False
p1_text v100-semantic 85 31 False
p2_code ref-exact 158 31 False
p2_code v100-semantic 158 31 False
p3_doc ref-exact+ownk 2046 31 False
p3_doc ref-exact 2046 31 False
p3_doc v100-semantic+ownk 2046 31 False
p3_doc v100-semantic 2046 31 False

ref-exact: max |x| per layer (all prompts, all steps)

Columns: stream = stream, str_attn = stream_attn, hcx_attn = hc.attn_x, hcx_ffn = hc.ffn_x, attn_o = attn.o, attn_out = attn.out, attn_tp4 = attn.out_tp4_partial, qk = attn.qk_logit, exp_w = moe.expert_out_max, exp_unw = moe.expert_out_unweighted_max, shared = moe.shared_out, moe_comb = moe.combine_f32, moe_tp4 = moe.out_tp4_partial, eng_wkv = engram.wkv_out, eng_add = engram.gate_add, idx = idx.score

layer stream str_attn hcx_attn hcx_ffn attn_o attn_out attn_tp4 qk exp_w exp_unw shared moe_comb moe_tp4 eng_wkv eng_add idx
0 9.31 1.92 0.66 1.84 3.5 10.1 4.16 17.3 9.88 13.2 7.22 10.3 8.27 — — —
1 77.5 9.25 5.03 3.38 3.66 9.38 3.55 18.4 43.8 37 2.45 43.9 19.1 104 5.57 —
2 68.5 72.5 44 53.8 2.3 5.81 2.93 11.1 17.6 30.6 2.31 19.7 14 — — 6.34
3 59.8 64 53 30.9 2.95 4.69 2.22 11.1 15.2 17.5 2.14 16.6 15.1 — — —
4 64.5 59 61 36.5 3 4 1.89 10.6 7.25 11.5 11.6 11.9 7.84 — — —
5 86.5 64.5 40.5 49 2.62 6.12 2.23 10.1 1.41 5.97 12.8 12.8 9.86 — — —
6 191 86.5 74.5 37 2.45 2.67 1.56 9.3 88 84.1 1.88 89.8 26.8 — — —
7 189 191 55.2 119 2.52 4.94 4.29 12.3 24 19.3 8.62 24.5 53.6 — — —
8 2.5e+03 189 84 59.8 3.17 30.8 25.6 15 1.16e+03 920 3.14 1.16e+03 386 — — 7.09
9 2.48e+03 2.5e+03 776 255 3.12 8.81 4 10.7 20.6 26.7 0.977 20.8 12.9 — — —
10 2.86e+03 2.48e+03 396 564 2.8 7.34 3.19 13.9 490 397 0.434 490 168 — — —
11 2.86e+03 2.86e+03 1.52e+03 1.78e+03 2.97 3.25 1.94 9.05 3.03 11.1 2.69 3.21 1.74 — — —
12 2.88e+03 2.86e+03 1.18e+03 470 4.25 8.81 3.53 16 6.94 8.63 0.832 6.99 4.87 — — —
13 2.94e+03 2.88e+03 1.82e+03 684 2.89 5 2.17 14.5 55.5 47.7 17 55.7 21.8 — — —
14 5.31e+03 2.94e+03 124 808 3.44 23.6 19.1 13 1.26e+03 974 21.8 1.26e+03 402 224 13.9 6.34
15 1.382e+04 5.31e+03 1.98e+03 5.41e+03 3.48 52.8 51.7 13.6 5.92e+03 5.32e+03 68.5 5.93e+03 1.7e+03 — — —
16 2.112e+04 1.382e+04 3.17e+03 1.96e+03 3.53 6.62 5 14.2 3.66e+03 2.67e+03 3.33 3.66e+03 1.07e+03 — — —
17 2.458e+04 2.112e+04 540 4.77e+03 4 3.33 2.34 17.8 1.71e+03 1.34e+03 1.24 1.71e+03 555 — — —
18 2.573e+04 2.458e+04 7.49e+03 1.318e+04 4.09 3.67 2.28 12.6 600 444 11.1 603 292 — — —
19 2.957e+04 2.573e+04 5.34e+03 6.59e+03 5.81 6.69 3.52 28.7 1.9e+03 1.56e+03 86.5 1.9e+03 521 — — —
20 5.376e+04 2.957e+04 1.306e+04 6.08e+03 4.91 5.09 3.62 12.3 1.907e+04 1.317e+04 12.8 1.908e+04 5.3e+03 — — 5.41
21 6.912e+04 5.376e+04 2.099e+04 4.25e+04 4.09 33.8 17.3 13.8 1.325e+04 1.002e+04 70.5 1.332e+04 3.45e+03 — — —
22 1.019e+05 6.912e+04 1.158e+04 4.531e+04 6.06 4.59 3.04 14 1.638e+04 1.165e+04 29.5 1.641e+04 4.51e+03 — — —
23 1.413e+05 1.019e+05 3.085e+04 1.92e+04 4.53 4.78 5.44 11.5 1.984e+04 1.338e+04 162 2e+04 5.1e+03 — — —
24 1.772e+05 1.413e+05 5.043e+04 5.658e+04 4.25 6.25 4.09 11.5 1.792e+04 1.258e+04 134 1.806e+04 4.79e+03 — — 5.28
25 2.109e+05 1.772e+05 4.966e+04 5.171e+04 4.19 6.53 3.5 12.2 1.69e+04 1.231e+04 199 1.709e+04 4.5e+03 — — —
26 2.335e+05 2.109e+05 4.096e+04 4.838e+04 3.86 8.81 5.05 16.1 1.114e+04 7.7e+03 446 1.154e+04 2.97e+03 — — —
27 2.499e+05 2.335e+05 1.495e+05 1.741e+04 4.81 6.06 4.56 12.9 1.107e+04 9.62e+03 248 1.132e+04 3.13e+03 — — —
28 2.949e+05 2.499e+05 9.984e+04 8.09e+04 3.83 13.7 7.58 14.2 2.202e+04 1.474e+04 756 2.24e+04 5.87e+03 — — 5.03
29 3.256e+05 2.949e+05 3.871e+05 7.168e+04 6.19 8.38 7.24 10.8 1.472e+04 1.021e+04 644 1.51e+04 3.88e+03 — — —
30 3.297e+05 3.256e+05 4.526e+05 7.834e+04 5.81 11.1 7.9 13 3.25e+03 4.89e+03 466 3.26e+03 862 — — —
31 3.359e+05 3.297e+05 2.785e+05 9.677e+04 5 7.75 4.53 13.4 4.96e+03 3.85e+03 178 4.97e+03 1.31e+03 — — —
32 3.359e+05 3.359e+05 3.727e+05 1.311e+05 3.69 8.25 3.77 12.2 196 1.46e+03 255 356 122 — — 5.78
33 3.4e+05 3.359e+05 1.679e+05 9.933e+04 5.06 27 11.2 13.5 6.24e+03 4.16e+03 776 7.02e+03 1.92e+03 — — —
34 3.42e+05 3.4e+05 3.174e+05 5.632e+04 4.16 22.9 10.4 14.8 2.64e+03 4.54e+03 215 2.66e+03 720 — — —
35 3.42e+05 3.42e+05 3.502e+05 6.016e+04 3.88 11.3 5.65 12.6 2.43e+03 2.54e+03 16.2 2.86e+03 763 — — —
36 3.584e+05 3.42e+05 2.112e+04 6.81e+04 5.16 32.8 20.6 41.9 1.114e+04 8.33e+03 900 1.149e+04 3.51e+03 — — 5.56
37 3.584e+05 3.564e+05 1.12e+04 5.888e+04 5.16 13.7 13.6 18.1 1.184e+04 7.89e+03 1.33e+03 1.275e+04 3.97e+03 — — —
38 3.584e+05 3.584e+05 3.83e+05 3.021e+04 4.31 35.2 17.1 86.7 4.02e+03 3.98e+03 434 4e+03 1.41e+03 — — —
39 3.584e+05 3.584e+05 1.193e+05 4.096e+04 36 484 355 102 1.453e+04 9.69e+03 2.82e+03 1.583e+04 4.47e+03 — — —

Overall maxima (layer, prompt:token position):

ref-exact: §4.4 rules

ref-exact+ownk: max |x| per layer (all prompts, all steps)

Columns: stream = stream, str_attn = stream_attn, hcx_attn = hc.attn_x, hcx_ffn = hc.ffn_x, attn_o = attn.o, attn_out = attn.out, attn_tp4 = attn.out_tp4_partial, qk = attn.qk_logit, exp_w = moe.expert_out_max, exp_unw = moe.expert_out_unweighted_max, shared = moe.shared_out, moe_comb = moe.combine_f32, moe_tp4 = moe.out_tp4_partial, eng_wkv = engram.wkv_out, eng_add = engram.gate_add, idx = idx.score

layer stream str_attn hcx_attn hcx_ffn attn_o attn_out attn_tp4 qk exp_w exp_unw shared moe_comb moe_tp4 eng_wkv eng_add idx
0 9.31 1.79 0.617 1.78 3.47 8.44 4.16 17.3 9.88 13.2 6.59 10.3 8.27 — — —
1 77.5 9.25 5.03 2.88 3.66 9.12 3.55 18.4 43.8 37 1.64 43.8 19.1 103 1.47 —
2 68.5 72.5 36.5 53.8 2.25 5.62 2.44 11 17.6 30.6 2.31 19.7 14 — — 5.78
3 59.8 64 53 30.9 2.95 4.69 2.1 11.1 15.2 17.5 2.14 16.6 15.1 — — —
4 64.5 56.2 61 36.5 3 4 1.78 10.3 1.55 2.14 11.6 11.9 7.84 — — —
5 86.5 64.5 34.2 49 2.62 5.94 2.18 10.1 0.949 2.18 12.8 12.8 9.86 — — —
6 86.5 86.5 74.5 12.1 2.45 2.67 1.56 9.3 1.38 2.5 1.87 1.66 1.83 — — —
7 86.5 86.5 55.2 119 2.52 4.94 4.29 12.3 24 19 8.62 24.5 53.6 — — —
8 290 86.5 84 14.4 3.17 30.8 25.6 15 123 112 3.12 123 66.6 — — 7.09
9 290 290 560 255 3.12 8.81 4 10.7 0.781 2.2 0.977 1.41 0.699 — — —
10 1.26e+03 290 396 215 2.8 7.34 3.19 13.9 484 393 0.434 484 168 — — —
11 1.26e+03 1.26e+03 1.5e+03 1.18e+03 2.97 3.25 1.94 9.05 0.973 1.91 2.69 3.21 1.02 — — —
12 1.26e+03 1.26e+03 1.17e+03 378 4.25 8.31 3.29 16 2.11 3.76 0.832 2.23 1.14 — — —
13 1.26e+03 1.26e+03 1.81e+03 318 2.89 5 2.17 12.8 11 23.4 17 16.6 15.6 — — —
14 1.57e+03 1.27e+03 84 808 3.44 22.1 16.3 13 165 132 21.1 165 61.9 224 13.9 6.34
15 1.318e+04 1.57e+03 152 1.9e+03 3.38 35.2 33 13.6 5.82e+03 4.03e+03 42.5 5.84e+03 1.68e+03 — — —
16 1.408e+04 1.318e+04 420 960 3.53 6.19 4.73 14.4 456 330 1.56 456 247 — — —
17 1.44e+04 1.408e+04 434 1.11e+03 4 3.33 2.34 17.8 165 132 1.24 165 50.5 — — —
18 1.491e+04 1.44e+04 692 1.5e+03 4.09 3.67 2.24 12.3 229 168 11.1 242 76.5 — — —
19 1.536e+04 1.491e+04 5.18e+03 3.63e+03 5.81 5.16 3.04 28.7 125 111 86.5 211 75.1 — — —
20 5.35e+04 1.536e+04 8.32e+03 5.38e+03 4.91 5.09 3.62 12.3 1.907e+04 1.314e+04 12.4 1.908e+04 5.3e+03 — — 5.41
21 6.861e+04 5.35e+04 1.338e+04 4.25e+04 4.06 33.8 17.3 13.8 1.318e+04 9.97e+03 70 1.325e+04 3.44e+03 — — —
22 1.014e+05 6.861e+04 1.133e+04 4.506e+04 4.75 4.59 3 14 1.638e+04 1.16e+04 28.4 1.641e+04 4.51e+03 — — —
23 1.413e+05 1.014e+05 2.048e+04 1.894e+04 4.53 4.53 5.14 11.5 1.984e+04 1.338e+04 162 2e+04 5.09e+03 — — —
24 1.772e+05 1.413e+05 2.918e+04 5.632e+04 4.25 4.41 3.97 11.5 1.792e+04 1.258e+04 128 1.806e+04 4.78e+03 — — 5.28
25 2.109e+05 1.772e+05 2.534e+04 5.146e+04 4.19 5.25 3.5 12.2 1.69e+04 1.231e+04 198 1.709e+04 4.49e+03 — — —
26 2.335e+05 2.109e+05 4.07e+04 4.838e+04 3.86 8.81 5.05 16.1 1.107e+04 7.66e+03 438 1.148e+04 2.97e+03 — — —
27 2.499e+05 2.335e+05 9.779e+04 1.741e+04 4.81 6.06 4.56 12.9 8e+03 9.62e+03 172 8.17e+03 2.07e+03 — — —
28 2.949e+05 2.499e+05 3.302e+04 5.606e+04 3.81 9.5 7.58 14.2 2.202e+04 1.474e+04 388 2.24e+04 5.86e+03 — — 5.03
29 3.256e+05 2.949e+05 3.85e+05 7.014e+04 6.16 8.38 7.06 10.8 1.472e+04 1.02e+04 388 1.51e+04 3.88e+03 — — —
30 3.297e+05 3.256e+05 4.526e+05 7.782e+04 5.81 9.69 7.57 11 3.2e+03 3.86e+03 172 3.21e+03 844 — — —
31 3.359e+05 3.297e+05 2.785e+05 9.626e+04 4.66 6.94 3.93 13.4 4.93e+03 3.48e+03 64.5 4.94e+03 1.31e+03 — — —
32 3.359e+05 3.359e+05 3.727e+05 5.658e+04 3.69 7.94 3.72 12.1 5.28 22.7 90 93.8 30.9 — — 5.78
33 3.4e+05 3.359e+05 9.779e+04 2.982e+04 4.62 14.7 5.83 13.5 4.02e+03 4.06e+03 266 4.02e+03 1.06e+03 — — —
34 3.42e+05 3.4e+05 3.174e+05 1.555e+04 4 16.4 7.71 14.8 2.64e+03 2.08e+03 83 2.66e+03 717 — — —
35 3.42e+05 3.42e+05 3.502e+05 5.99e+04 3.88 8.75 4 12.6 2.43e+03 2.54e+03 5.94 2.86e+03 761 — — —
36 3.584e+05 3.42e+05 2.112e+04 6.81e+04 5.16 32.2 19.4 41.9 1.114e+04 8.33e+03 572 1.149e+04 3.51e+03 — — 5.56
37 3.564e+05 3.543e+05 3.78e+03 5.862e+04 5.16 13.7 13.6 18.1 1.184e+04 7.89e+03 908 1.275e+04 3.97e+03 — — —
38 3.564e+05 3.564e+05 3.83e+05 3.008e+04 3.56 32.8 16.4 60.1 3.98e+03 2.66e+03 434 4e+03 1.41e+03 — — —
39 3.584e+05 3.564e+05 7.936e+04 4.096e+04 30 484 355 93 1.453e+04 9.69e+03 2.75e+03 1.583e+04 4.47e+03 — — —

Overall maxima (layer, prompt:token position):

ref-exact+ownk: §4.4 rules

v100-semantic: max |x| per layer (all prompts, all steps)

Columns: stream = stream, str_attn = stream_attn, hcx_attn = hc.attn_x, hcx_ffn = hc.ffn_x, attn_o = attn.o, attn_out = attn.out, attn_tp4 = attn.out_tp4_partial, qk = attn.qk_logit, exp_w = moe.expert_out_max, exp_unw = moe.expert_out_unweighted_max, shared = moe.shared_out, moe_comb = moe.combine_f32, moe_tp4 = moe.out_tp4_partial, eng_wkv = engram.wkv_out, eng_add = engram.gate_add, idx = idx.score

layer stream str_attn hcx_attn hcx_ffn attn_o attn_out attn_tp4 qk exp_w exp_unw shared moe_comb moe_tp4 eng_wkv eng_add idx
0 9.21 1.92 0.66 1.85 3.46 10.1 4.18 17.5 9.83 13.1 7.23 10.2 8.28 — — —
1 76.9 9.17 5.01 3.35 3.65 10.2 3.58 18.4 43.7 36.6 2.49 43.8 19.9 104 5.83 —
2 68.2 71.9 43.4 55.8 2.27 5.84 2.94 11 17.9 31 2.3 19.9 14.2 — — 5.91
3 59.8 63.9 54 31.4 2.95 4.74 2.25 11.2 15.6 17.5 2.13 16.8 15.5 — — —
4 65.8 58.4 62.1 36.9 3.02 4 1.87 10.6 7.25 11.4 11.4 11.7 7.64 — — —
5 87.6 65.7 42.6 49.5 2.64 6.05 2.17 9.99 1.4 5.92 12.9 13 9.64 — — —
6 193 87.5 76.2 36.7 2.45 2.66 1.6 9.26 88.8 84.5 1.99 90.7 26.7 — — —
7 191 192 55.7 120 2.62 4.94 4.29 12 23.7 19.1 8.37 24.1 52.9 — — —
8 2.52e+03 191 85.1 58.5 3.18 30.6 25.5 15 1.17e+03 925 3.16 1.17e+03 393 — — 7.24
9 2.5e+03 2.52e+03 787 258 3.13 8.8 4.03 10.9 20.6 26.5 0.971 20.8 12.8 — — —
10 2.88e+03 2.5e+03 403 573 2.79 7.4 3.12 14.1 489 397 0.435 490 169 — — —
11 2.88e+03 2.88e+03 1.55e+03 1.79e+03 2.92 3.26 1.97 9.08 2.95 10.9 2.66 3.18 1.74 — — —
12 2.9e+03 2.88e+03 1.19e+03 480 4.15 8.63 3.53 16 7.01 8.72 0.843 7.05 4.84 — — —
13 2.96e+03 2.9e+03 1.91e+03 694 2.9 5.16 2.16 14.7 55.5 47.7 17.1 55.7 21.7 — — —
14 5.35e+03 2.96e+03 125 814 3.43 23.4 19.6 13.2 1.27e+03 976 21.4 1.27e+03 407 224 13.9 6.46
15 1.379e+04 5.35e+03 2.02e+03 5.44e+03 3.47 53.6 52 15 5.91e+03 5.27e+03 67.3 5.93e+03 1.71e+03 — — —
16 2.114e+04 1.379e+04 3.2e+03 1.99e+03 3.51 6.66 5.22 13 3.68e+03 2.69e+03 3.33 3.68e+03 1.07e+03 — — —
17 2.447e+04 2.113e+04 541 4.88e+03 4.48 3.32 2.37 18.8 1.69e+03 1.32e+03 1.36 1.69e+03 545 — — —
18 2.567e+04 2.447e+04 7.53e+03 1.323e+04 4.12 3.52 2.3 12.1 612 453 11.4 616 299 — — —
19 2.953e+04 2.567e+04 5.26e+03 6.67e+03 5.88 6.37 3.24 30 1.94e+03 1.6e+03 87.1 1.94e+03 534 — — —
20 5.377e+04 2.953e+04 1.323e+04 6.05e+03 4.92 5.04 3.7 14.3 1.906e+04 1.308e+04 12.1 1.907e+04 5.3e+03 — — 5.32
21 6.943e+04 5.376e+04 2.11e+04 4.249e+04 4.12 34.9 18.2 13.6 1.348e+04 1.019e+04 71.8 1.355e+04 3.52e+03 — — —
22 1.034e+05 6.939e+04 1.145e+04 4.555e+04 6.06 4.58 3 14.4 1.699e+04 1.201e+04 29.9 1.702e+04 4.65e+03 — — —
23 1.444e+05 1.034e+05 3.117e+04 1.939e+04 4.39 4.61 5.34 11.4 2.041e+04 1.376e+04 163 2.057e+04 5.25e+03 — — —
24 1.823e+05 1.444e+05 5.286e+04 5.684e+04 4.2 6.36 4.2 11.4 1.89e+04 1.312e+04 134 1.903e+04 5.04e+03 — — 5.27
25 2.161e+05 1.823e+05 5.066e+04 5.189e+04 4.13 5.81 3.56 13.4 1.667e+04 1.211e+04 197 1.687e+04 4.44e+03 — — —
26 2.402e+05 2.161e+05 4.09e+04 4.849e+04 3.86 8.91 5.24 16.8 1.163e+04 8.04e+03 457 1.205e+04 3.11e+03 — — —
27 2.564e+05 2.402e+05 1.487e+05 1.764e+04 4.84 5.87 4.29 13.5 1.141e+04 9.89e+03 254 1.166e+04 3.23e+03 — — —
28 3.021e+05 2.564e+05 1.001e+05 8.168e+04 3.83 13.8 7.6 14.5 2.243e+04 1.502e+04 745 2.283e+04 5.97e+03 — — 4.99
29 3.313e+05 3.021e+05 3.942e+05 7.372e+04 6.17 8.57 7.3 10.9 1.429e+04 9.91e+03 649 1.464e+04 3.75e+03 — — —
30 3.349e+05 3.313e+05 4.617e+05 7.861e+04 5.79 11.4 8.37 12.9 3.13e+03 4.83e+03 456 3.14e+03 829 — — —
31 3.413e+05 3.349e+05 2.823e+05 9.561e+04 4.96 6.93 3.84 12.9 5.08e+03 3.82e+03 177 5.09e+03 1.34e+03 — — —
32 3.413e+05 3.411e+05 3.8e+05 1.32e+05 3.71 8.34 3.66 12.3 197 1.45e+03 250 351 119 — — 5.68
33 3.451e+05 3.413e+05 1.676e+05 1.013e+05 5 27.2 11.2 13.6 6.25e+03 4.22e+03 761 7.01e+03 1.91e+03 — — —
34 3.466e+05 3.451e+05 3.232e+05 5.636e+04 4.13 22.8 10.5 14.9 2.61e+03 4.89e+03 228 2.63e+03 709 — — —
35 3.464e+05 3.466e+05 3.542e+05 5.99e+04 3.7 11.7 5.71 12.9 2.42e+03 2.52e+03 15.6 2.85e+03 758 — — —
36 3.625e+05 3.464e+05 2.127e+04 7.074e+04 5.13 32.7 21 39.9 1.066e+04 9.12e+03 919 1.158e+04 3.25e+03 — — 5.86
37 3.607e+05 3.593e+05 1.142e+04 6.02e+04 5.34 10.4 10.5 18 1.191e+04 7.94e+03 1.33e+03 1.267e+04 4.42e+03 — — —
38 3.608e+05 3.599e+05 3.888e+05 3.061e+04 3.95 36.1 19 90.3 4.2e+03 4.91e+03 462 4.22e+03 1.26e+03 — — —
39 3.593e+05 3.601e+05 1.314e+05 4.15e+04 36 441 365 102 1.222e+04 8.15e+03 2.89e+03 1.219e+04 3.49e+03 — — —

Overall maxima (layer, prompt:token position):

v100-semantic: §4.4 rules

v100-semantic+ownk: max |x| per layer (all prompts, all steps)

Columns: stream = stream, str_attn = stream_attn, hcx_attn = hc.attn_x, hcx_ffn = hc.ffn_x, attn_o = attn.o, attn_out = attn.out, attn_tp4 = attn.out_tp4_partial, qk = attn.qk_logit, exp_w = moe.expert_out_max, exp_unw = moe.expert_out_unweighted_max, shared = moe.shared_out, moe_comb = moe.combine_f32, moe_tp4 = moe.out_tp4_partial, eng_wkv = engram.wkv_out, eng_add = engram.gate_add, idx = idx.score

layer stream str_attn hcx_attn hcx_ffn attn_o attn_out attn_tp4 qk exp_w exp_unw shared moe_comb moe_tp4 eng_wkv eng_add idx
0 9.21 1.78 0.617 1.77 3.25 8.46 4.18 17.5 9.83 13.1 6.64 10.2 8.28 — — —
1 76.9 9.17 5.01 2.88 3.65 9.18 3.58 18.4 43.2 36.6 1.66 43.3 19.9 103 1.46 —
2 68.2 71.9 36 55.8 2.27 5.54 2.51 11 17.9 31 2.3 19.9 14.2 — — 5.91
3 59.8 63.9 54 31.4 2.95 4.74 2.12 11.2 15.2 17.5 2.13 16.5 15 — — —
4 65.8 56.2 62.1 36.9 3.02 4 1.8 10.2 1.55 2.16 11.4 11.7 7.64 — — —
5 87.6 65.7 34.3 49.5 2.64 6.04 2.17 9.99 0.929 2.12 12.9 13 9.64 — — —
6 87.5 87.5 76.2 12.2 2.45 2.66 1.6 9.26 1.36 2.62 1.99 1.77 1.94 — — —
7 87.5 87.5 55.7 120 2.51 4.94 4.29 12 23.7 18.7 8.37 24.1 52.9 — — —
8 290 87.4 85.1 14.4 3.18 30.6 25.5 15 123 113 3.12 124 66.8 — — 7.24
9 290 290 561 258 3.13 8.8 4.03 11.2 0.774 2.1 0.971 1.42 0.708 — — —
10 1.27e+03 290 403 218 2.79 7.4 3.12 14.1 489 396 0.435 489 169 — — —
11 1.27e+03 1.27e+03 1.55e+03 1.19e+03 2.92 3.26 1.89 9.08 0.961 1.87 2.66 3.18 1.03 — — —
12 1.27e+03 1.27e+03 1.19e+03 385 4.15 8.45 3.27 16 2.08 3.75 0.843 2.2 1.13 — — —
13 1.27e+03 1.27e+03 1.83e+03 320 2.9 5.16 2.16 13 10.8 23 17.1 16.7 15.6 — — —
14 1.58e+03 1.28e+03 84.8 814 3.43 22.5 17.2 13.7 166 132 21.4 166 61.6 224 13.9 6.46
15 1.336e+04 1.58e+03 153 1.91e+03 3.35 35.5 33.3 13.5 5.9e+03 4.08e+03 43.3 5.91e+03 1.7e+03 — — —
16 1.43e+04 1.335e+04 425 968 3.51 6.66 4.75 14 475 345 1.55 475 260 — — —
17 1.463e+04 1.429e+04 443 1.11e+03 4.48 3.32 2.37 18.8 167 134 1.36 167 50.9 — — —
18 1.511e+04 1.463e+04 702 1.52e+03 4.12 3.52 2.3 12.1 231 170 11.4 245 76.9 — — —
19 1.553e+04 1.511e+04 5.26e+03 3.64e+03 5.88 5.17 2.83 30 124 111 87.1 212 74.9 — — —
20 5.313e+04 1.554e+04 8.46e+03 5.45e+03 4.92 5.04 3.7 14.3 1.881e+04 1.299e+04 11.7 1.882e+04 5.23e+03 — — 5.32
21 6.896e+04 5.311e+04 1.321e+04 4.205e+04 4.08 34.9 18.2 13.6 1.346e+04 1.018e+04 71.4 1.353e+04 3.51e+03 — — —
22 1.028e+05 6.892e+04 1.141e+04 4.49e+04 4.74 4.58 2.79 14.4 1.696e+04 1.199e+04 29.7 1.699e+04 4.64e+03 — — —
23 1.438e+05 1.028e+05 2.045e+04 1.898e+04 4.39 4.61 5.34 11.4 2.04e+04 1.375e+04 162 2.056e+04 5.24e+03 — — —
24 1.812e+05 1.438e+05 2.916e+04 5.622e+04 4.19 4.44 4.06 11.4 1.862e+04 1.306e+04 134 1.877e+04 4.97e+03 — — 5.19
25 2.149e+05 1.812e+05 2.526e+04 5.126e+04 4.13 5.33 3.56 13.4 1.662e+04 1.207e+04 196 1.682e+04 4.43e+03 — — —
26 2.389e+05 2.149e+05 4.042e+04 4.798e+04 3.86 8.91 5.24 16.8 1.161e+04 8.02e+03 455 1.204e+04 3.1e+03 — — —
27 2.544e+05 2.389e+05 9.961e+04 1.752e+04 4.84 5.87 4.29 13.5 7.56e+03 9.87e+03 180 7.74e+03 1.97e+03 — — —
28 3e+05 2.544e+05 3.343e+04 5.625e+04 3.83 9.88 7.6 14.5 2.243e+04 1.502e+04 397 2.283e+04 5.97e+03 — — 4.99
29 3.293e+05 3e+05 3.914e+05 7.048e+04 6.17 8.57 7.3 10.9 1.429e+04 9.91e+03 354 1.464e+04 3.75e+03 — — —
30 3.329e+05 3.293e+05 4.587e+05 7.861e+04 5.79 9.88 7.9 11.7 3.13e+03 3.75e+03 165 3.14e+03 829 — — —
31 3.393e+05 3.329e+05 2.808e+05 9.553e+04 4.75 6.93 3.84 12.9 5.07e+03 3.58e+03 68.1 5.08e+03 1.34e+03 — — —
32 3.393e+05 3.391e+05 3.774e+05 5.748e+04 3.71 7.96 3.66 12.3 5.08 21.9 85.2 88.9 29 — — 5.68
33 3.431e+05 3.393e+05 9.866e+04 2.983e+04 4.68 14.4 5.76 13.6 4.13e+03 4.21e+03 285 4.15e+03 1.09e+03 — — —
34 3.446e+05 3.431e+05 3.21e+05 1.59e+04 4.02 16.5 7.98 14.9 2.6e+03 2.05e+03 81.9 2.62e+03 708 — — —
35 3.443e+05 3.446e+05 3.526e+05 5.98e+04 3.7 8.94 3.89 12.9 2.41e+03 2.52e+03 5.85 2.84e+03 756 — — —
36 3.604e+05 3.443e+05 2.121e+04 7.074e+04 5.13 32.7 20 39.7 8.75e+03 9.12e+03 678 9.58e+03 2.81e+03 — — 5.62
37 3.586e+05 3.573e+05 3.74e+03 6.02e+04 5.34 10.4 10.5 18 1.191e+04 7.94e+03 884 1.267e+04 4.42e+03 — — —
38 3.588e+05 3.579e+05 3.866e+05 3.059e+04 3.56 33.8 17.1 64 4.2e+03 4.91e+03 382 4.21e+03 1.26e+03 — — —
39 3.572e+05 3.581e+05 7.989e+04 4.15e+04 28 441 365 95.5 1.222e+04 8.15e+03 2.78e+03 1.219e+04 3.49e+03 — — —

Overall maxima (layer, prompt:token position):

v100-semantic+ownk: §4.4 rules

Raw: [...]/range_audit.json; per-run stats: [private path].