Skip to the content.

Origin. PLI Labs research archive, file eight-card research/topk-full-20261110.md. Published 2026-11-18 in the Volta V100 research repository. Edited for publication. Machine names, private file paths, network addresses, session names and the operator’s name were replaced with generic terms (the V100 server, [private path], [address], the owner). Links to files that are not published were turned into plain text. No figure and no finding was changed. Dates inside the note are its own; later results are in the build pages of this repository.

Labels: MEASURED = from the receipt of a run, READ = from code or a log, INFERENCE = not measured. Receipt: logs/topk/topk-full-full-20261110T042244Z.json (pass 2 reported). Bench: stage/topk/bench_topk_full.py.

Answer

Live tokens 1K 4K 16K 64K 128K 256K 512K 1M
Stage 0, 3 layers (ms) 0.032 0.092 0.232 0.231 0.233 0.260 0.312 0.472
Stage 1, 5 layers (ms) 0.119 0.231 0.387 0.387 0.434 0.521 0.793 1.560
Total per round (ms) 0.152 0.323 0.618 0.618 0.667 0.780 1.106 2.031
Share of a 40 / 65 ms round 0.4 / 0.2 % 0.8 / 0.5 % 1.5 / 1.0 % 1.5 / 1.0 % 1.7 / 1.0 % 2.0 / 1.2 % 2.8 / 1.7 % 5.1 / 3.1 %

Microseconds per call (MEASURED, 1M capacity, 6 rows, median of 400 samples, each the mean of 10 calls)

Stage Live tokens n per row (max) nseg cap / live cap, engine path: median (min-max) k2 hist p1 k4 hist p2 k6 count k7 write k1+k3+k5 single live grid
0 1,024 512 65 / 1 10.8 (10.6-11.2) 1.9 1.9 2.0 1.9 4.6 1.5 8.6
0 4,096 2,048 65 / 1 30.6 (30.5-30.9) 5.6 5.6 4.6 8.6 6.5 12.1 30.0
0 16,384 8,192 65 / 1 77.3 (77.2-77.7) 15.5 14.6 13.6 27.4 6.5 38.0 77.6
0 65,536 32,768 65 / 4 77.1 (77.0-77.5) 15.5 14.6 13.5 27.3 6.5 141.0 75.2
0 131,072 65,536 65 / 8 77.5 (77.4-77.8) 15.6 14.8 13.8 27.3 6.5 295.1 75.9
0 262,144 131,072 65 / 16 86.5 (86.3-86.9) 19.7 20.1 14.0 27.4 6.5 586.1 85.0
0 524,288 262,144 65 / 32 104.1 (103.7-104.6) 25.5 27.1 14.7 31.3 6.5 1169.5 102.8
0 1,048,576 524,288 65 / 64 157.3 (151.2-160.1) 39.4 43.4 20.2 48.8 6.8 2345.0 156.5
1 1,024 1,024 129 / 1 23.9 (23.7-24.3) 4.1 4.1 3.1 5.5 6.5 7.8 22.2
1 4,096 4,096 129 / 1 46.3 (46.2-46.7) 9.0 8.7 7.7 15.0 6.5 20.4 46.0
1 16,384 16,384 129 / 2 77.3 (77.2-77.6) 15.5 14.6 13.6 27.4 6.5 71.1 75.5
1 65,536 65,536 129 / 8 77.3 (77.2-77.7) 15.6 14.7 13.7 27.3 6.5 289.9 75.7
1 131,072 131,072 129 / 16 86.8 (86.6-87.3) 19.7 20.1 14.2 27.5 6.5 576.6 85.2
1 262,144 262,144 129 / 32 104.1 (103.7-104.7) 25.5 27.0 14.6 31.5 6.5 1149.4 102.7
1 524,288 524,288 129 / 64 158.6 (155.3-160.8) 39.2 43.4 20.5 48.7 6.7 2309.6 157.3
1 1,048,576 1,048,576 129 / 128 311.9 (307.6-316.6) 78.4 86.2 40.3 102.3 7.2 5378.2 310.6
0, short path (width 16,384) 1K / 4K / 16K 512 / 2,048 / 8,192 - - - - - - - 1.5 / 12.2 / 37.6 -
1, short path (width 16,384) 1K / 4K / 16K 1,024 / 4,096 / 16,384 - - - - - - - 7.6 / 19.5 / 67.2 -

k1 = hist.zero_() memset, k3 and k5 = _seg_select (1.4 to 2.9 us each). The sum of k1 to k7 timed alone is within 2.6 us of the whole call. Short-path ranges are in the receipt (min-max within 1.0 us). 512K-capacity rows are in the receipt (the engine-path arm within 2.2 us of the 1M row at the same live length, every arm within 5.2 us).

Method

Not measured