Origin. PLI Labs research archive, file
kernel-research/UPSCAN1105-ONECAT.md. Published 2026-11-18 in the Volta V100 research repository. Edited for publication. Machine names, private file paths, network addresses, session names and the operator’s name were replaced with generic terms (the V100 server,[private path],[address],the owner). Links to files that are not published were turned into plain text. No figure and no finding was changed. Dates inside the note are its own; later results are in the build pages of this repository.
Read-only audit complete. No missing upstream PR is a demonstrated drop-in speedup for this V4.1 GGUF + PP2×TP4 + MRV2 build. The strongest new 1Cat findings are batching/producer-layout patterns and a narrowly useful Engram memory reduction. The existing campaign already covers the large categories.
Ranked portable findings
All effects below distinguish upstream author measurements from forecasts for our build. Effort: S = 0.5–2 engineer days, M = 3–7, L = >1 week; includes port and validation, not GPU-window availability.
| Rank here | Idea / primary source | KCAMP | Already present? | SM70 / target applicability | Effort | Evidence-based effect, limits |
|---|---|---|---|---|---|---|
| 1 | Widen every decode fast-path admission, share expert weights across rows, fuse row permutation: #1369 | C1 + C2 | New main Nov5, absent named series; our C1 already proposes same class | Yes pattern; Qwen Flash-Next FP4/FP16/GDN kernels cannot serve IQ2_XXS V4.1 unchanged. PP independence requires layer-level port, MRV2 batch dispatch | L | AUTHOR MEASURED c4 39.7→28.53 ms = 28.1% less round time / 39.2% more throughput at fixed acceptance, c1 15.1→14.75 ms. Not a forecast for ours. Local plan forecasts 24-row round 1.3–1.6× six-row ⇒ aggregate 4/(1.3–1.6)=2.5–3.08×; author result supports targeting admission cliffs, not this numerical forecast |
| 2 | Producer emits consumer’s K16 layout, remove repack launches: #1367 | S6 + C1 | New main Nov5; not carried | Yes design; target dmv/IQ2 pack formats differ, so first count actual repacks. Preserve exact rounding and ownership under overlapping MRV2 batches | M | AUTHOR MEASURED ~180 launches removed/GPU/c4; c4 20.34→20.13, 20.47→20.23, 21.85→21.51 ms (1.0–1.6% lower), c2 14.89→14.53 /16.23→15.71 (2.4–3.2%). The nominal ~1ms removed launch cost did not translate to ~1ms wall time. Our S6 0.6–1ms remains an estimate, not validated here |
| 3 | Decode-preserving full-chunk prefill, fixed+linear budget and graph ladder: #1288, #1290, #1295 | C4 + C2 | Already main≤6db9c81; Qwen profile does not prove our custom V4.1 route uses it | Host algorithm portable; retain our fixed 8,188 boundaries and V4.1 compressor/replay semantics. Check routing before any change | M–L | AUTHOR MEASURED Qwen c4 1K aggregate107→316tok/s in1288 (2.95×), no comparable8K gain;1290 c4 GSM8K489→554 /455→524 (+13.3–15.2%), mixed120-token step186→135ms. Not additive; model-specific. Our 9.8tok/s interference measurement motivates C4 but no exact result can be predicted |
| 4 | Engram projection gather in 1,024-row blocks: issue#1342 | New memory prerequisite for C0/C5; potentially S2 | Not in named25 or published production patch set searched; base model.py:210–215 full gather verified; latest deployed allocator unprofiled | Directly relevant to V41Engram IQ4_NL input: reduce all-gather intermediates, leave wkv full-row GEMM intact. PP stage0 only. Collective/gate copy order must be checked bitwise | S–M | Exact memory arithmetic: 2×8192×25600×2 bytes =800MiB; 1024-row blocks=100MiB; 700MiB/card stage0 saved, not all8cards. AUTHOR MEASURED TP4 offload0.32→0.28 lowers47.1→44.4ms but our experts already resident: expect 0 decode gain unless current memory constraint changes. Larger blocks/more collectives can hurt TTFT; no published isolated TTFT gain |
| 5 | Long-context fused index scoring; cap workspace by actual rows: #1347 | S10 + C5 | Not named series; fused/tiled custom V4.1 index scorers already local production | Author explicitly says neither change reaches V4.1. Reuse FP32-head accumulator/tiling ideas, not cherry-pick-and-enable. Need custom compressed-KV layout + batched top-k | M–L | AUTHOR MEASURED V4 PP5 7.6–8.4ms/round less at178–336K, ~10.5–11.5%; workspace −540–560MiB/card for pass1024→512, TTFT+0.3%. V100 microbench132096keys0.748→0.387ms, 48.3% less scorer time. Not ours; current tiled scorer already removes much of this opportunity. S10 local1–3ms estimate must be reprofiled |
| 6 | Eliminate draft attention metadata host sync: issue#1361 | S9 / New diagnostic | Not named series; target DSpark custom port may bypass entire path | SM70 host fix portable only if DSpark calls EAGLE metadata→Flash-V100. Issue concerns Qwen MTP; DSpark sparse/custom path not proven affected. Search actual call path before ranking as speed win | S diagnosis, M if applicable | AUTHOR MEASURED31.5→23.3ms/round (−26.0%),92.9→127.4tok/s (+37.1%; title says+28%, denominator differs); ~2.7ms/draft-step sync removed. QSA MTP control unchanged123.6tok/s. Our entire drafter~1.35ms/round on old profile, so cannot claim8ms gain for ours |
| 7 | Stop tail-length JIT specialization in mHC: issue#1326 | New cold-TTFT item / S6 | Not named patch; V4.1 custom mHC may not invoke generic kernel | Generic Triton argument metadata change is Volta-compatible, but issue’s GLM mHC layout24 and generic triton.py differ from custom V4.1 | S | AUTHOR MEASURED fresh requests13.7/5.9/10.4→5.7/5.4/5.3s; steady~5.3s unchanged. Only remove observed JIT gaps, not steady throughput. Fixed chunk tails/prefix hits still need all specialization classes checked |
| 8 | Shape-qualified allreduce+norm fusion and topology selection: #1292, #1245 | C1 + S6 | 1292 in base;1245 absent; our push-allreduce already exists | SM70+TP4 NVLink pattern applies; Gemma norm≠V4.1 FP32 mHC, preserve sum order, no FP16 compression of FP32 residual.1245 examples TP2/TP8 not our TP4 guarantee | M–L | Local allreduce family~0.95ms/round is ceiling on eliminating whole family at old profile, not achievable forecast.1245 AUTHOR MEASURED8GPU mixed FP32 128KiB70.807→95.611us comparator (+35% throughput) but448/512KiB loses; demonstrate topology/payload crossover, not universal faster |
| 9 | mHC row reuse staging + grouped dense wo_a: #1325, #1273 | S4/S6 | Both carried | Row reuse applies abstractly; generic GLM staging/native blockFP8 wo_a not custom V4.1 FP16 dense path | M | 1325 AUTHOR MEASURED GLM prefill+7–7.5%; same shape T2048 splits4 staging2011→741us, no V4 evidence.1273 grouping reuses native code but target dense needs own grouped-GEMM port. Do not count already-carried code as new gain |
| 10 | Explicit batch invariance and graph/clone lifetime ownership: #1358, #1359, issue#1331 | C2 (required) | Clone fixes absent series, top-k tie issue open | Generic compile ownership applies if compile pass used; radix atomic tie bug only if our scorer uses that op. Kernel set equality ≠ deterministic tie rule | S–M | No speed promise. Required to make C1/C3 concurrent measurements meaningful.1331 AUTHOR OBSERVED1104/1152 planted-tie rows selected different sets across30calls; non-tie0. House exactness demands tie rule and actual-path tests |
Scope and evidence
Complete tables are ONECAT-CENSUS.md (110 PRs, every state), ONECAT-COMMITS.md (68 main commits), ONECAT-ISSUES.md (117 all-date topic-title matches). Each PR row records number/title/link, author, state, created/updated/merged dates, source-stack membership and target/KCAMP applicability. Commit table records both author and committer date. Full body text and head/base/merge SHAs remain in upstream/onecat-evidence/pulls-window.json; all PRs, including outside the window, in pulls-all.json; all issues in issues-all.json; changed-file diffs for new candidates in pr-N-files.json. Fetches used public read-only REST APIs to exhaustion, not GitHub search’s1000-result limit. No repository/GPU inference or performance test ran.
Window starts 2026-10-28T00:00:00Z, current API cutoff during this scan2026-11-05. Main observed at 3c051b52ed (2026-11-05T14:45:28Z), while published baseline is 6db9c816cd. Main has27 subsequent commits including QR docs; many are PRs already explicitly carried. Some old PRs were only updated after cutoff (#387/#389) and are recorded without claiming new code. Apache-2.0 repository licence per published CREDITS; individual borrowed code/external Triton still needs respective source notice if ported.
Prior scans read: CONCBATCHOP1104/research-notes.md and philbert-prefill-inventory.md; LANES.md upstream entries. The latter already covered replay skip, PP overlap, Engram residency, fused attention/indexer, fixed-M padding, demand KV, layer-sliced mixed prefill and EP expert layout. They are not newly discovered speedups here.
The25 open-PR stack and version drift
The published SERIES includes exactly: 1344,1345,1328,1325,1321,1320,1294,1293,1300,1273,1338,1322,1319,1318,1317,1316,1315,1314,1309,1308,1307,1306,1305,1304,1282, plus main6db9c81 and base1277. #1282 incorporates1280; local hot-refill0048 ports issue1341. Therefore a new GitHub merged state is not a missing feature. Nov5 main now merges1338,1318,1316,1315,1314,1309,1308,1307,1306,1305,1304 that are already in the series.
Presence ≠ current PR head equivalence. #1282 and #1280 updated Nov3 after series assembly: FP32 scorer repair and released sm7x Triton wheel/docs. Raw PR body explicitly records these deltas. Our custom V4.1 scorer replaces much of the generic route, but compare blob-level manifests before claiming either missing or fixed. Stock Triton3.6 does not lower this route to Volta HMMA; carrying opt-in Python code alone does not supply the patched compiler. New wheel claim82.5TFLOPS vs8.6 stock on4096³ is a stand-alone GEMM measurement, not9.6× engine speed; cuBLAS94.4 on the same test. No install or compiler change was made.
Relevant issue disposition
| Issue | Author; state; created / last update (UTC) | Target assessment |
|---|---|---|
| 1361 | squibloads; open;2026-11-04 /2026-11-04 | Draft metadata CPU-bound issue; conditional route only, as ranked above |
| 1342 | squibloads; open;2026-11-02 /2026-11-02 | Direct V4.1 TP4 Engram memory transient, portability strongest; no resident-expert decode speed promised |
| 1341 | squibloads; open;2026-11-02 /2026-11-02 | Packed hot-ring stale rows on prefix hit; local0048 hot-refill already ports this. Keep prefix-after-long-answer vs prefix-after-short-answer exactness case |
| 1336 | squibloads; open;2026-10-31 /2026-10-31 | Sampling clone+reference_rows.any host fence; GLM author−0.15ms/round (32.10→31.95), local coupled sharded sampler may bypass. Base1267 does not prove all fences gone |
| 1335 | squibloads; open;2026-10-31 /2026-10-31 | EXL3 32slot grouped GEMV1.4–2.2× launch/+28–29% GLM prefill. IQ2 different decoder: expert-row reuse is C1/S3 pattern only, already earlier inventory |
| 1334 | squibloads; open;2026-10-31 /2026-10-31 | Fused FP32 scorer1.27–1.34× vs cuBLAS, end-to-endGLM+0.6–1.6%; local fused V4.1 scorer already adopts idea |
| 1333 | squibloads; open;2026-10-31 /2026-10-31 | Register-resident sparse MLA output2.5–2.9× kernel/+17% GLM prefill; model-specific, S1 fused attention already candidate; no fresh additive claim |
| 1332 | squibloads; open;2026-10-31 /2026-10-31 | f16GEMM N%32tail wrong; verify exact shape admission for S4, no speed idea |
| 1331 | squibloads; open;2026-10-31 /2026-10-31 | Topk atomic tie nondeterminism C2, as ranked |
| 1330 | squibloads; open;2026-10-31 /2026-10-31 | GLM compressed kpool virtual-split zeroing bug; custom V4.1 arena distinct, add ownership check for C0 if shared generic zeroer reached |
| 1327 | squibloads; open;2026-10-31 /2026-10-31 | Index scores fp16 overflow;1328 carried, custom fused scorer must keep FP32 head products. Generic fix does not prove custom path |
| 1326 | squibloads; open;2026-10-30 /2026-10-30 | JIT tail specialization, conditional cold-TTFT opportunity |
| 1323,1324 | squibloads; open;2026-10-30 /2026-10-30 | GLM MTP graph omission/capture pinning; custom DSpark graph route differs, diagnostic only |
| 1285,1269,1262,1270 | full dates/authors/states in issue census | Qwen capacity/DFlash sampler/alignment/nativeMoE reports. Do not import262Kcapacity or corrupt-sampler workaround to custom DSpark blindly |
| 652,597 | full metadata in census;652closed,597open | V100 custom-allreduce no-NVLink hang / oldV4tool+speed report. Our TP4 NVLink topology and V4.1 route differ; motivates collective real-surface tests, not blanket disable |
| 852 | historical;open;2026-10-03 update | PP2TP4Qwen async OFF31→48tok/s, c16aggregate unchanged. Important negative evidence against assuming generic async C3 automatically faster; target MRV2 port needs own cadence test |
| 1223,880,1070,1042 | historical metadata in census | Mixed latency/capture ladders/concurrency regressions; prefill starvation and non-overlap measurement caveats support C4, no clean new kernel |
| 678 | historical;open;2026-09-22 update | NUMA per-rank binding measured+1.9%decode (locality72.6→98.2%); source-specific host topology, existing tuning not a new kernel |
Other topic matches (including resolved historical bug reports and usage questions) are preserved without omissions in the117-row issue census. Issues674and1337 are umbrella reports; their underlying proposals are classified above or by PR census. No relevant issue body is used as proof of our runtime behavior.
What to carry into KCAMP
- C1 acceptance before code: enumerate every route for rows6/12/18/24 and real expert histograms; identify batch-size cliffs before optimizing FLOPS. Use1369 as design evidence and1368 as warning: removing four launches can lengthen the surviving launch enough to erase the gain, and b/a rounding changed.
- C3 is engine work, not #1294 again:1294 already overlaps eligible eager sends; it cannot make two complete decode microbatches independently in flight. Tie ownership of token history, speculative outcomes, PP output buffers and Engram/KV rows to a batch, then test host/GPU timelines. No new 1Cat PR in this window demonstrates exact V4.1 PP2 decode pipelining.
- Memory audit before C0/C5: trace Engram800MiB transient, per-pass sparse workspaces and per-request hot-ring allocation.1342 offers exact700MiB reduction on stage0 if that transient exists;1347’s generic540–560MiB saving does not apply automatically to our custom path.
- Retain exactness:1358/1359 compile clone fixes,1331ties,1341prefix rings and1368batch rounding show why C2 is a separate work item. No near-tie equivalence is silently promoted to bitwise identity.
Supplementary vLLM pointers from parent (#61464/#63956/#60618/#64442/#59131/#64458) were searched against the110PR bodies; none appears by upstream number. That is not a proof of absence of equivalent code: inspect source/diffs against1Cat baseline before porting.
Engram source confirmation: upstream base899e9ed33 model.py:210–215 runs full projected gather then per-row gate. The published series/engine patch files were searched for row-block gather equivalents and none was found. This establishes published-source opportunity, not latest deployed-memory measurement.
Issue-body completeness addendum
The final pass searched all207 non-PR issue titles and bodies, not just topic words in titles. It found 57additional matches beyond the117-title inventory (174candidate issues total;33nonmatches). Every extra match has author/state/dates and an explicit relevance disposition in ONECAT-ISSUES-BODY.md, with raw bodies preserved. Substantive additions: issue656probabilistic multi-request corruption despite normal acceptance;668/424batch-vs-request draft-limit ownership;728capture-time metadata freezing the wrong no-history attention route;1249/1238/632offload completion/preemption/per-group watermark races;265allocator options reset by partial settings updates. They reinforce C2/C0/C3/C5 acceptance and ownership design, do not establish target defects or new speedups, and do not change the ranked opportunities. All concern older generic/Qwen/TurboQuant/native-offload paths whose correspondence to custom V4.1 must be checked.