Skip to the content.

Origin. PLI Labs research archive, file kernel-research/UPSCAN1105-ONECAT.md. Published 2026-11-18 in the Volta V100 research repository. Edited for publication. Machine names, private file paths, network addresses, session names and the operator’s name were replaced with generic terms (the V100 server, [private path], [address], the owner). Links to files that are not published were turned into plain text. No figure and no finding was changed. Dates inside the note are its own; later results are in the build pages of this repository.

Read-only audit complete. No missing upstream PR is a demonstrated drop-in speedup for this V4.1 GGUF + PP2×TP4 + MRV2 build. The strongest new 1Cat findings are batching/producer-layout patterns and a narrowly useful Engram memory reduction. The existing campaign already covers the large categories.

Ranked portable findings

All effects below distinguish upstream author measurements from forecasts for our build. Effort: S = 0.5–2 engineer days, M = 3–7, L = >1 week; includes port and validation, not GPU-window availability.

Rank here Idea / primary source KCAMP Already present? SM70 / target applicability Effort Evidence-based effect, limits
1 Widen every decode fast-path admission, share expert weights across rows, fuse row permutation: #1369 C1 + C2 New main Nov5, absent named series; our C1 already proposes same class Yes pattern; Qwen Flash-Next FP4/FP16/GDN kernels cannot serve IQ2_XXS V4.1 unchanged. PP independence requires layer-level port, MRV2 batch dispatch L AUTHOR MEASURED c4 39.7→28.53 ms = 28.1% less round time / 39.2% more throughput at fixed acceptance, c1 15.1→14.75 ms. Not a forecast for ours. Local plan forecasts 24-row round 1.3–1.6× six-row ⇒ aggregate 4/(1.3–1.6)=2.5–3.08×; author result supports targeting admission cliffs, not this numerical forecast
2 Producer emits consumer’s K16 layout, remove repack launches: #1367 S6 + C1 New main Nov5; not carried Yes design; target dmv/IQ2 pack formats differ, so first count actual repacks. Preserve exact rounding and ownership under overlapping MRV2 batches M AUTHOR MEASURED ~180 launches removed/GPU/c4; c4 20.34→20.13, 20.47→20.23, 21.85→21.51 ms (1.0–1.6% lower), c2 14.89→14.53 /16.23→15.71 (2.4–3.2%). The nominal ~1ms removed launch cost did not translate to ~1ms wall time. Our S6 0.6–1ms remains an estimate, not validated here
3 Decode-preserving full-chunk prefill, fixed+linear budget and graph ladder: #1288, #1290, #1295 C4 + C2 Already main≤6db9c81; Qwen profile does not prove our custom V4.1 route uses it Host algorithm portable; retain our fixed 8,188 boundaries and V4.1 compressor/replay semantics. Check routing before any change M–L AUTHOR MEASURED Qwen c4 1K aggregate107→316tok/s in1288 (2.95×), no comparable8K gain;1290 c4 GSM8K489→554 /455→524 (+13.3–15.2%), mixed120-token step186→135ms. Not additive; model-specific. Our 9.8tok/s interference measurement motivates C4 but no exact result can be predicted
4 Engram projection gather in 1,024-row blocks: issue#1342 New memory prerequisite for C0/C5; potentially S2 Not in named25 or published production patch set searched; base model.py:210–215 full gather verified; latest deployed allocator unprofiled Directly relevant to V41Engram IQ4_NL input: reduce all-gather intermediates, leave wkv full-row GEMM intact. PP stage0 only. Collective/gate copy order must be checked bitwise S–M Exact memory arithmetic: 2×8192×25600×2 bytes =800MiB; 1024-row blocks=100MiB; 700MiB/card stage0 saved, not all8cards. AUTHOR MEASURED TP4 offload0.32→0.28 lowers47.1→44.4ms but our experts already resident: expect 0 decode gain unless current memory constraint changes. Larger blocks/more collectives can hurt TTFT; no published isolated TTFT gain
5 Long-context fused index scoring; cap workspace by actual rows: #1347 S10 + C5 Not named series; fused/tiled custom V4.1 index scorers already local production Author explicitly says neither change reaches V4.1. Reuse FP32-head accumulator/tiling ideas, not cherry-pick-and-enable. Need custom compressed-KV layout + batched top-k M–L AUTHOR MEASURED V4 PP5 7.6–8.4ms/round less at178–336K, ~10.5–11.5%; workspace −540–560MiB/card for pass1024→512, TTFT+0.3%. V100 microbench132096keys0.748→0.387ms, 48.3% less scorer time. Not ours; current tiled scorer already removes much of this opportunity. S10 local1–3ms estimate must be reprofiled
6 Eliminate draft attention metadata host sync: issue#1361 S9 / New diagnostic Not named series; target DSpark custom port may bypass entire path SM70 host fix portable only if DSpark calls EAGLE metadata→Flash-V100. Issue concerns Qwen MTP; DSpark sparse/custom path not proven affected. Search actual call path before ranking as speed win S diagnosis, M if applicable AUTHOR MEASURED31.5→23.3ms/round (−26.0%),92.9→127.4tok/s (+37.1%; title says+28%, denominator differs); ~2.7ms/draft-step sync removed. QSA MTP control unchanged123.6tok/s. Our entire drafter~1.35ms/round on old profile, so cannot claim8ms gain for ours
7 Stop tail-length JIT specialization in mHC: issue#1326 New cold-TTFT item / S6 Not named patch; V4.1 custom mHC may not invoke generic kernel Generic Triton argument metadata change is Volta-compatible, but issue’s GLM mHC layout24 and generic triton.py differ from custom V4.1 S AUTHOR MEASURED fresh requests13.7/5.9/10.4→5.7/5.4/5.3s; steady~5.3s unchanged. Only remove observed JIT gaps, not steady throughput. Fixed chunk tails/prefix hits still need all specialization classes checked
8 Shape-qualified allreduce+norm fusion and topology selection: #1292, #1245 C1 + S6 1292 in base;1245 absent; our push-allreduce already exists SM70+TP4 NVLink pattern applies; Gemma norm≠V4.1 FP32 mHC, preserve sum order, no FP16 compression of FP32 residual.1245 examples TP2/TP8 not our TP4 guarantee M–L Local allreduce family~0.95ms/round is ceiling on eliminating whole family at old profile, not achievable forecast.1245 AUTHOR MEASURED8GPU mixed FP32 128KiB70.807→95.611us comparator (+35% throughput) but448/512KiB loses; demonstrate topology/payload crossover, not universal faster
9 mHC row reuse staging + grouped dense wo_a: #1325, #1273 S4/S6 Both carried Row reuse applies abstractly; generic GLM staging/native blockFP8 wo_a not custom V4.1 FP16 dense path M 1325 AUTHOR MEASURED GLM prefill+7–7.5%; same shape T2048 splits4 staging2011→741us, no V4 evidence.1273 grouping reuses native code but target dense needs own grouped-GEMM port. Do not count already-carried code as new gain
10 Explicit batch invariance and graph/clone lifetime ownership: #1358, #1359, issue#1331 C2 (required) Clone fixes absent series, top-k tie issue open Generic compile ownership applies if compile pass used; radix atomic tie bug only if our scorer uses that op. Kernel set equality ≠ deterministic tie rule S–M No speed promise. Required to make C1/C3 concurrent measurements meaningful.1331 AUTHOR OBSERVED1104/1152 planted-tie rows selected different sets across30calls; non-tie0. House exactness demands tie rule and actual-path tests

Scope and evidence

Complete tables are ONECAT-CENSUS.md (110 PRs, every state), ONECAT-COMMITS.md (68 main commits), ONECAT-ISSUES.md (117 all-date topic-title matches). Each PR row records number/title/link, author, state, created/updated/merged dates, source-stack membership and target/KCAMP applicability. Commit table records both author and committer date. Full body text and head/base/merge SHAs remain in upstream/onecat-evidence/pulls-window.json; all PRs, including outside the window, in pulls-all.json; all issues in issues-all.json; changed-file diffs for new candidates in pr-N-files.json. Fetches used public read-only REST APIs to exhaustion, not GitHub search’s1000-result limit. No repository/GPU inference or performance test ran.

Window starts 2026-10-28T00:00:00Z, current API cutoff during this scan2026-11-05. Main observed at 3c051b52ed (2026-11-05T14:45:28Z), while published baseline is 6db9c816cd. Main has27 subsequent commits including QR docs; many are PRs already explicitly carried. Some old PRs were only updated after cutoff (#387/#389) and are recorded without claiming new code. Apache-2.0 repository licence per published CREDITS; individual borrowed code/external Triton still needs respective source notice if ported.

Prior scans read: CONCBATCHOP1104/research-notes.md and philbert-prefill-inventory.md; LANES.md upstream entries. The latter already covered replay skip, PP overlap, Engram residency, fused attention/indexer, fixed-M padding, demand KV, layer-sliced mixed prefill and EP expert layout. They are not newly discovered speedups here.

The25 open-PR stack and version drift

The published SERIES includes exactly: 1344,1345,1328,1325,1321,1320,1294,1293,1300,1273,1338,1322,1319,1318,1317,1316,1315,1314,1309,1308,1307,1306,1305,1304,1282, plus main6db9c81 and base1277. #1282 incorporates1280; local hot-refill0048 ports issue1341. Therefore a new GitHub merged state is not a missing feature. Nov5 main now merges1338,1318,1316,1315,1314,1309,1308,1307,1306,1305,1304 that are already in the series.

Presence ≠ current PR head equivalence. #1282 and #1280 updated Nov3 after series assembly: FP32 scorer repair and released sm7x Triton wheel/docs. Raw PR body explicitly records these deltas. Our custom V4.1 scorer replaces much of the generic route, but compare blob-level manifests before claiming either missing or fixed. Stock Triton3.6 does not lower this route to Volta HMMA; carrying opt-in Python code alone does not supply the patched compiler. New wheel claim82.5TFLOPS vs8.6 stock on4096³ is a stand-alone GEMM measurement, not9.6× engine speed; cuBLAS94.4 on the same test. No install or compiler change was made.

Relevant issue disposition

Issue Author; state; created / last update (UTC) Target assessment
1361 squibloads; open;2026-11-04 /2026-11-04 Draft metadata CPU-bound issue; conditional route only, as ranked above
1342 squibloads; open;2026-11-02 /2026-11-02 Direct V4.1 TP4 Engram memory transient, portability strongest; no resident-expert decode speed promised
1341 squibloads; open;2026-11-02 /2026-11-02 Packed hot-ring stale rows on prefix hit; local0048 hot-refill already ports this. Keep prefix-after-long-answer vs prefix-after-short-answer exactness case
1336 squibloads; open;2026-10-31 /2026-10-31 Sampling clone+reference_rows.any host fence; GLM author−0.15ms/round (32.10→31.95), local coupled sharded sampler may bypass. Base1267 does not prove all fences gone
1335 squibloads; open;2026-10-31 /2026-10-31 EXL3 32slot grouped GEMV1.4–2.2× launch/+28–29% GLM prefill. IQ2 different decoder: expert-row reuse is C1/S3 pattern only, already earlier inventory
1334 squibloads; open;2026-10-31 /2026-10-31 Fused FP32 scorer1.27–1.34× vs cuBLAS, end-to-endGLM+0.6–1.6%; local fused V4.1 scorer already adopts idea
1333 squibloads; open;2026-10-31 /2026-10-31 Register-resident sparse MLA output2.5–2.9× kernel/+17% GLM prefill; model-specific, S1 fused attention already candidate; no fresh additive claim
1332 squibloads; open;2026-10-31 /2026-10-31 f16GEMM N%32tail wrong; verify exact shape admission for S4, no speed idea
1331 squibloads; open;2026-10-31 /2026-10-31 Topk atomic tie nondeterminism C2, as ranked
1330 squibloads; open;2026-10-31 /2026-10-31 GLM compressed kpool virtual-split zeroing bug; custom V4.1 arena distinct, add ownership check for C0 if shared generic zeroer reached
1327 squibloads; open;2026-10-31 /2026-10-31 Index scores fp16 overflow;1328 carried, custom fused scorer must keep FP32 head products. Generic fix does not prove custom path
1326 squibloads; open;2026-10-30 /2026-10-30 JIT tail specialization, conditional cold-TTFT opportunity
1323,1324 squibloads; open;2026-10-30 /2026-10-30 GLM MTP graph omission/capture pinning; custom DSpark graph route differs, diagnostic only
1285,1269,1262,1270 full dates/authors/states in issue census Qwen capacity/DFlash sampler/alignment/nativeMoE reports. Do not import262Kcapacity or corrupt-sampler workaround to custom DSpark blindly
652,597 full metadata in census;652closed,597open V100 custom-allreduce no-NVLink hang / oldV4tool+speed report. Our TP4 NVLink topology and V4.1 route differ; motivates collective real-surface tests, not blanket disable
852 historical;open;2026-10-03 update PP2TP4Qwen async OFF31→48tok/s, c16aggregate unchanged. Important negative evidence against assuming generic async C3 automatically faster; target MRV2 port needs own cadence test
1223,880,1070,1042 historical metadata in census Mixed latency/capture ladders/concurrency regressions; prefill starvation and non-overlap measurement caveats support C4, no clean new kernel
678 historical;open;2026-09-22 update NUMA per-rank binding measured+1.9%decode (locality72.6→98.2%); source-specific host topology, existing tuning not a new kernel

Other topic matches (including resolved historical bug reports and usage questions) are preserved without omissions in the117-row issue census. Issues674and1337 are umbrella reports; their underlying proposals are classified above or by PR census. No relevant issue body is used as proof of our runtime behavior.

What to carry into KCAMP

  1. C1 acceptance before code: enumerate every route for rows6/12/18/24 and real expert histograms; identify batch-size cliffs before optimizing FLOPS. Use1369 as design evidence and1368 as warning: removing four launches can lengthen the surviving launch enough to erase the gain, and b/a rounding changed.
  2. C3 is engine work, not #1294 again:1294 already overlaps eligible eager sends; it cannot make two complete decode microbatches independently in flight. Tie ownership of token history, speculative outcomes, PP output buffers and Engram/KV rows to a batch, then test host/GPU timelines. No new 1Cat PR in this window demonstrates exact V4.1 PP2 decode pipelining.
  3. Memory audit before C0/C5: trace Engram800MiB transient, per-pass sparse workspaces and per-request hot-ring allocation.1342 offers exact700MiB reduction on stage0 if that transient exists;1347’s generic540–560MiB saving does not apply automatically to our custom path.
  4. Retain exactness:1358/1359 compile clone fixes,1331ties,1341prefix rings and1368batch rounding show why C2 is a separate work item. No near-tie equivalence is silently promoted to bitwise identity.

Supplementary vLLM pointers from parent (#61464/#63956/#60618/#64442/#59131/#64458) were searched against the110PR bodies; none appears by upstream number. That is not a proof of absence of equivalent code: inspect source/diffs against1Cat baseline before porting.

Engram source confirmation: upstream base899e9ed33 model.py:210–215 runs full projected gather then per-row gate. The published series/engine patch files were searched for row-block gather equivalents and none was found. This establishes published-source opportunity, not latest deployed-memory measurement.

Issue-body completeness addendum

The final pass searched all207 non-PR issue titles and bodies, not just topic words in titles. It found 57additional matches beyond the117-title inventory (174candidate issues total;33nonmatches). Every extra match has author/state/dates and an explicit relevance disposition in ONECAT-ISSUES-BODY.md, with raw bodies preserved. Substantive additions: issue656probabilistic multi-request corruption despite normal acceptance;668/424batch-vs-request draft-limit ownership;728capture-time metadata freezing the wrong no-history attention route;1249/1238/632offload completion/preemption/per-group watermark races;265allocator options reset by partial settings updates. They reinforce C2/C0/C3/C5 acceptance and ownership design, do not establish target defects or new speedups, and do not change the ranked opportunities. All concern older generic/Qwen/TurboQuant/native-offload paths whose correspondence to custom V4.1 must be checked.