Skip to the content.

Most of what makes a V100 serve a 2026 model is other people’s work. This page says whose, by public handle, with a link to the work itself. Authors and states of pull requests were read from GitHub with gh pr view on 2026-11-18. If we have missed or misdescribed a contribution, please open an issue and we will correct it.

The model and its weights

Who What Licence
DeepSeek DeepSeek-V4.1-Flash-1017: the model, its official MXFP4 experts, the DSpark speculative drafter, the Engram design MIT
NVIDIA DeepSeek-V4.1-Flash-1017-NVFP4: the NVFP4 pack whose experts the NVFP4 builds serve, and its accuracy table MIT
ISTA-DASLab DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF: the 2-bit quantization Volta sPLIt serves, and the non-expert tensors of every build MIT
ggml ggml: the GGUF container and the IQ quantization formats MIT
Phil Tompkins (philbert440) The EXL3 pack and engine our first eight-card build ran on see the repositories

The engine: 1Cat-vLLM

1CatAI/1Cat-vLLM (Apache-2.0) is the vLLM fork that keeps Volta and Turing cards serving current models. Every build here runs on it. The work of its authors that these builds use directly:

Pull request or issue Author State (2026-11-18) What it is How these builds use it
#1277 yangzhuxinyzx merged 2026-10-27 DeepSeek-V4.1-Flash on four V100s: 512K packed key-value storage, faster prompt reading, the model’s kernels The base of every build here. Volta sPLIt is its port to two boards
#1482 yangzhuxinyzx merged 2026-11-11 The official checkpoint in four stages of four cards on sixteen V100s, with batched decode Not in our engine. The upstream reference for several requests per step
#274, #308, #344 yangzhuxinyzx merged 2026-08 The gated DSpark verifier; decode hot paths for two stages of four cards; quality gates on the fast paths (DeepSeek-V4) The speculative decoding path and the two-stage precedent
#183 yangzhuxinyzx merged 2026-08-03 Probabilistic DSpark drafting Tested here as a variant; lost
#1292 yangzhuxinyzx merged 2026-10-28 The fused four-card push all-reduce Our all-reduce change admits this model’s message size to it
#742 Peuqui merged 2026-10-03 The skinny QPN expert backend for NVFP4 and MXFP4 Every NVFP4 expert in the NVFP4 builds runs through it
#1263 Peuqui merged 2026-10-28 Do not resend a request’s token history every decode step Our keep-batch lever is its port to the old runner (-24.8 % of a round at 128K)
#1221 Peuqui open The DSpark speculator for Model Runner V2 The path our Model Runner V2 port builds on
#1280, #1282, #1347 Peuqui closed; open; open Long-prefill index and sparse attention work; tensor-core Triton prefill routes; a fused decode index scorer Ideas behind our fused prefill index scorer
#1362 Peuqui merged 2026-11-16 Skip the embedding and head on pipeline stages that do not hold them The precedent for the twelve-card build’s middle-stage memory change
#874, #1222, #1273, #1294, #1218 Peuqui merged Index, grouped GEMM, send overlap and draft-vocabulary work on the same path Read before every plan. The Volta sPLIt patch series carries 1Cat’s main branch and then-open pull requests by their authors, unchanged
#1243 rivetphilbot open DeepSeek-V4.1-Flash on eight V100s with EXL3 experts, CUDA graphs, DSpark and 1M context The engine of our first eight-card build
#1157 kezboard233 merged 2026-11-10 A native block-scale FP8 GEMM for 17 to 32 rows Covers the row range where our FP8 dense kernel loses
#1325, #1537, issue #1341 squibloads merged; open; closed Row staging for large prefill batches; a pipeline speculation fix; the bug report on stale rows after a cache hit Issue #1341 is the bug our hot-ring refill fixes
issue #852 zambalee open Turning off asynchronous scheduling gives +55 % single-stream decode on a two-stage V100 build Why every build here schedules synchronously

Peuqui also maintains the Triton fork those prefill routes need.

The kernels: v100-skinny

Dennis Vertzyas (dnv2003), v100-skinny (MIT for kernels and documents, Apache-2.0 for its fork patches): hand-written NVFP4 and FP8 kernels for Volta’s mma.sync.m8n8k4, and the measurements this whole site leans on: the 879 GB/s read ceiling, 77 % of it at one row for NVFP4, the serving traps. His kernels reach these builds through 1Cat-vLLM #742. No v100-skinny source file is copied into this repository. ProprietaryLegal/v100-skinny is a mirror, not our work.

Ideas, tools and methods

Who What Where it shows up
The vLLM project vLLM (Apache-2.0): the serving engine underneath, and Model Runner V2 everywhere
Elevatormusic and the TensorFold authors TensorFold #25: coupled sampled drafts +14 % tok/s at temperature 1 on Volta sPLIt
0rand deepseek-v41-tensorfold-spark: the same idea in a DGX Spark recipe, and expert top-p pruning the pruning was tested here and lost
sudoingX The p4_kld.py fidelity method and its general panel; the two-Spark launcher Volta sPLIt’s fidelity figures
Leonccaa ReVolta, a V100 inference server reviewed for ideas
mw00, lxe project-maya, dsx read as research leads
0xSero the hill-climbing scripts at https://github.com/0xsero the builder skill’s optimization loop is borrowed from them
OpenMMLab TurboMind’s sm_70 GEMM code, carried by 1Cat-vLLM the operand decoders our prefill expert kernel keeps
NVIDIA The Volta, Ampere, Hopper and Blackwell tuning guides; compute-sanitizer the kernel rules

What PLI Labs sent upstream

Pull requests to 1Cat-vLLM from PLI Labs’ two accounts, read on 2026-11-18.

Pull request State What it does
#651 merged 2026-09-26 Accept a drafter embedding replica across split weight loads
#1021 merged 2026-11-16 Keep block-FP8 dense weights in FP8 on Volta
#1030 open Generalise the MXFP4 expert geometry; split sizes 4, 6, 9, 12, 18
#1031 open DeepSeek-V4.1-Flash on V100: three stages of four on twelve cards, two on eight
#1370 open Two stages of four for the #1277 package (the Volta sPLIt port)
#1371 open Preserve synchronous speculative step ownership in a pipeline
#1372 merged 2026-11-16 Tile the live decode index scoring
#1373 merged 2026-11-10 Retain persistent request rows across empty pipeline steps
#1374 open Shard the drafter’s main projection
#1375 merged 2026-11-16 Coupled sampled drafts with keyed verification
#1376 open Skip layers 21 to 39 in early prefill chunks
#1377 open Coupled drafts and keyed verification on vocabulary shards
#1379 open DeepSeek-V4.1 and its drafter on Model Runner V2
#1380 merged 2026-11-09 README: the eight-card result
#1381 merged 2026-11-10 Restore packed key-value bytes after a cache hit; reserve the speculative rewind span
#1382 merged 2026-11-10 Admit small aligned FP16 messages to the four-card push all-reduce
#1383 merged 2026-11-16 Build the routed-expert plan inside the quantize launch
#1384 merged 2026-11-16 Fast routed-expert decode kernels
#1385 merged 2026-11-16 Instruction-diet routed-expert kernels
#1386 merged 2026-11-16 Reduce decode index scores over the live prefix
#1387 merged 2026-11-16 Grouped prefill GEMM for the compressed routed experts
#1388 open Fused prefill index scores
#1389 open Fuse the attention prep and post kernels
#1390 merged 2026-11-16 Dense mat-vec on a double-buffered kernel
#1488 merged 2026-11-16 README: NVFP4 hot experts on eight V100s

Two earlier pull requests (#96, #100, July 2026) were closed without merging. The maintainers corrected some of the merged ones before merging (#1381’s description lists their corrections); the merged code is theirs as well as ours.

The llama.cpp fix for DeepSeek-V4 with a quantized key cache is reported upstream as llama.cpp #25382; the patches are in deepseek-v4-flash-v100.

Press

Tom’s Hardware, 2026-11-09, Shane Downing: Lawyer builds private AI server with eight used Nvidia V100 GPUs.