Skip to the content.

Origin. PLI Labs research archive, file v100-research/v100-skinny/FOUNDATION.md. Published 2026-11-18 in the Volta V100 research repository. Edited for publication. Machine names, private file paths, network addresses, session names and the operator’s name were replaced with generic terms (the V100 server, [private path], [address], the owner). Links to files that are not published were turned into plain text. No figure and no finding was changed. Dates inside the note are its own; later results are in the build pages of this repository.

Status: historical source/reproducibility review from initial discovery on 2026-09-16. The observations, gaps, recommendations, and commands below describe that inspection slice, not the current campaign state. No model weights were downloaded, no GPU serving was started, and no build or install was run by this review task.

Subsequent work established the isolated Qwen TP2 smoke baseline; its raw receipt records API completion, QPN/Marlin routes, and memory. It is not the published TP4 benchmark and is not a GLM result. GLM checkpoint acquisition also completed; see the storage verification. The current GLM implementation and acceptance ledger are maintained in the progress log and the adjacent component screening reports. Initial recommendations here to wait for those now-completed steps are retained as historical analysis, not active launch instructions.

TASK

Determine what the upstream dnv2003/v100-skinny release actually provides, which exact engine/checkpoint it reproduces, and whether that contract can be carried to GLM-5.3 NVFP4 on 10 and 12 Tesla V100s. Record the QPN2/QPN8/Marlin format contracts, KV and attention state behavior, memory duplication, model assumptions, and an honest pinned baseline for the fleet bootstrap.

The target used for the GLM analysis is NVIDIA’s current nvidia/GLM-5.3-Flash-NVFP4, not the much larger generic RadixArk/GLM-5.3-NVFP4 artifact. These are different architectures and different storage footprints; both distinctions are called out below.

WHAT I INSPECTED

Local external checkouts (kept outside the research repo):

Upstream v100-skinny source reviewed:

Current primary model metadata was read without downloading weights:

FILES / COMMITS

v100-skinny

1Cat-vLLM

Pinned published Qwen baseline

GLM-5.3-Flash metadata facts

From NVIDIA model revision 09b04e5e74bca08ca8549fc736d4cdd8624bfde3:

FINDINGS

1. What v100-skinny actually supports

The release is a Qwen3.8-specific serving stack, not a general GLM port. Its headline contract is mixed W4A16 NVFP4 plus W8A16 FP8, with FP16 activations and FP16 KV. It assumes Qwen3.8’s fused projection names, Qwen3Next/GDN recurrent-state metadata, chain-MTP layout, chat template, and MTP proposer behavior. The README’s model-specific count is 64 layers: 48 GDN/linear-attention layers and 16 full-attention layers; only the latter hold ordinary token KV.

The target NVIDIA Flash checkpoint instead exposes glm5_next, mHC, KDA, DeepSeek sparse attention, 288 routed experts, W4A4 expert/dense-MLP quantization, BF16 MTP, and FP8-KV metadata. None of that is an accidental rename. The 1Cat-vLLM v1.2.2 source has no glm5_next implementation, and the skinny GDN patch explicitly describes graph-visible metadata tensors for Qwen GDN operations. Reusing serve-qwen38-native.sh for GLM would therefore be a false baseline.

2. Quantization math and scale contracts

Qwen/v100-skinny NVFP4 W4A16 path

The checkpoint-native codes are two FP4 E2M1 nibbles per byte along K:

codes:       uint8 [N, K/2]
group scales: FP8 E4M3 [N, K/16]
global scale: FP32 scalar, ModelOpt amax / (6 * 448) = amax / 2688
activation:   FP16 (no activation quantization)

The intended numerical value is the E2M1 code multiplied by its FP8 group scale and the multiplicative global scale. kernels/skinny_kernels.cu folds the decoder’s 2^-14 representation correction by using gscale * 16384; there is no FP8-to-FP4 requantization. _qpn_prepack in marlin.py is a byte/nibble address permutation only: it expands nibbles, applies the Volta-fragment K order, and repacks. It requires N % 32 == 0 and K % 64 == 0; chunking limits temporary gather intermediates to roughly 300 MB. The one-shot lm_head gather would be about 2.4 GiB.

QPN2

QPN8

Target GLM NVFP4 stores activation-scale metadata; the POC may declare W4A16

NVIDIA’s GLM checkpoint serializes 4-bit E2M1 weights, FP8-E4M3 weight scales per 16-element group, an F32 weight_scale_2, and an F32 input_scale for each quantized linear. The native metadata is therefore W4A4-oriented, but a Volta POC may explicitly choose W4A16 semantics. Under that choice, the adapter must state how FP16 activations and the serialized input_scale are handled and must pass a real-tensor reference comparison; W4A4 equivalence is not the acceptance condition.

v100-skinny’s SM70 linear hook turns serialized NVFP4 into its W4A16 Marlin adapter: it keeps the codes and weight scales but discards input_scale. That behavior cannot be silently reused for GLM; the GLM adapter must make the choice explicit and preserve enough metadata to audit the numerical contract.

The regular ModelOptNvFp4Config still reports minimum capability 75, while the SM70 lowering is wired for the Qwen mixed/W4A16 route. The custom QPN2 linear op does not process routed expert batches. The GLM target’s MoE path is delegated to vLLM’s select_nvfp4_moe_backend / make_nvfp4_moe_kernel; no v100-skinny QPN2/Marlin expert implementation is present.

3. Resident memory and duplication

For a Qwen W4A16 linear that reaches skinny, load-time state is:

  1. checkpoint-native U8 codes and FP8 group scales;
  2. a second QPN fragment-order code/scale copy; and
  3. a Marlin-repacked weight/scale copy for prefill and M>=17.

VLLM_SKINNY_DROP_CT=1 frees (1) after the QPN prepack, but (2) and (3) remain. The release therefore has two packed low-bit copies per NVFP4 layer; it does not have a persistent FP16 model copy. QPN8 keeps one packed FP8 copy and frees its source only after the invertibility check. Large-M FP8 prefill uses transient reconstruction; QPN2/Marlin path selection is based on runtime M, not a static model-wide choice.

For GLM, the 204.44 GB checkpoint alone is not the runtime footprint. Any attempt to route all 36,297 W4A4 expert/dense linears through the W4A16 skinny adapter would require activation-contract changes and potentially QPN + Marlin duplicates over the 190.4 GiB binary weight payload. The BF16 MTP layer, unquantized attention/routers/shared experts/embeddings/lm_head, expert dispatch buffers, recurrent KDA state, sparse-attention state, NCCL buffers, CUDA graph pools, and transient prepack workspaces all consume additional memory. A 10-GPU 320 GiB or 12-GPU 384 GiB raw HBM total therefore cannot be called feasible from checkpoint size alone.

The target’s model card recommends Blackwell-only execution and a Blackwell vLLM image; this is a support statement, not a Volta result. A Volta POC needs a model-loader census and per-rank peak-memory trace before any full weight load is approved.

4. KV, prefill, decode, and state contracts

The skinny patch deliberately ignores a checkpoint FP8-KV directive below SM80, resolving Qwen’s FP8-KV metadata to FP16 KV. README results attribute about +4.82 ms/round to scalar paged attention when FP8 KV is honored on SM70; the published Qwen path therefore gates for FP16 KV, XQA/tensor-core decode, and zero scalar_paged calls.

That policy is not enough to make GLM attention work. Qwen’s fast metadata patch is for GDN chain-MTP: it builds mamba_state_idx, speculative state slot selectors, query starts, and state indices for Qwen’s recurrent state contract. GLM’s KDA layers and DSA indexer have different state and attention layouts; its 11 sparse layers use compressed MLA/DSA state, and its model config requests FP8 KV. The v100-skinny patch has no GLM5Next state adapter, no DSA Volta validation, and no proof that the 1Cat FlashAttention V100 backend accepts GLM’s sparse/KDA metadata.

Prefill and decode also differ materially. QPN2/QPN8 are decode/small-M weight-stream kernels. Qwen NVFP4 M>=17 uses Marlin, while FP8 M>96 uses a transient FP16 reconstruction. GLM’s W4A4 routed-expert kernels must process expert-token batches after routing, activation scales, and grouped GEMM; the existing skinny linear custom op cannot be substituted for that MoE path. The GLM MTP layer is BF16 and its MTP state/proposer path is not the Qwen GDN chain path.

5. Ten/twelve-GPU topology constraints

The published skinny result is TP4, not TP10 or TP12. The GLM target has hidden_size=4096, 288 routed experts, and 8 experts per token. A blind --tensor-parallel-size 10 or 12 is not a safe launch assumption: hidden and projection partition divisibility, expert parallel partitioning (288 is divisible by 12 but not 10), and pipeline/expert parallel interactions need to be checked against the exact runtime/model implementation. The existing NVIDIA recipe is TP4 + expert parallel on Blackwell, not a recipe for either V100 topology.

The fleet’s Qwen reference smoke test may use the approved idle pair (TP2), but it must be labeled non-published because the pinned launcher and published numbers are TP4. The GLM POC’s “exactly 10 GPUs, then 12” condition should be applied only after the architecture/parallelism adapter exists and the coordinator approves UUID visibility. Do not infer a successful 2-GPU Qwen smoke test as GLM feasibility.

CHANGES MADE

TESTS RUN

Read-only checks performed:

Not run in this source-analysis slice:

RESULT

The published v100-skinny result has an identified Qwen3.8-27B TP4 reference recipe (not locally reproduced in this source-analysis slice), subject to an isolated Python 3.12 environment, a torch build whose runtime reports sm_70, CUDA 12.8 nvcc, the digest-pinned 1Cat wheel, and the paired 0.1.10 TileLang/tvm-ffi override. It is not a GLM-5.3-Flash runtime. The exact upstream reproduction is:

cd [private path]
export ENV_PREFIX=[private path]
# Use a fresh isolated Python 3.12 environment. The upstream script installs
# the digest-pinned wheel and the required tilelang/tvm-ffi overrides.
bash scripts/bootstrap-sm70.sh

hf download RadixArk/Qwen3.8-27B-NVFP4 \
  --revision 554ebba9b5f1b79dc11246341960360e6ef05ef4 \
  --local-dir [...]/Qwen3.8-27B-NVFP4

# Published configuration is TP4. Override CUDA_VISIBLE_DEVICES only after
# fleet approval; TP2 is a smoke test and not a published result.
bash scripts/serve-qwen38-native.sh \
  [...]/Qwen3.8-27B-NVFP4

Before allowing the bootstrap to JIT-build, the isolated environment should be gated with:

ENV_PREFIX=[private path]
"$ENV_PREFIX/bin/python" - <<'PY'
import torch
assert torch.cuda.is_available()
print('torch', torch.__version__, 'cuda', torch.version.cuda)
print('arch', torch.cuda.get_arch_list())
assert 'sm_70' in torch.cuda.get_arch_list()
x = torch.ones((8, 8), device='cuda', dtype=torch.float16)
assert torch.isfinite(x @ x).all().item()
print('sm70 fp16 smoke: PASS')
PY

The exact GLM checkpoint has no known-good minimal V100 model recipe in the inspected sources. The smallest honest GLM prerequisite is metadata-only validation followed by a loader/architecture smoke test against a single representative dense NVFP4 linear and a routed-expert grouped GEMM under the declared W4A16 semantics; a full 204 GB download should wait for that adapter and a per-rank memory plan.

OPEN ISSUES

  1. Architecture gap (blocking): pinned 1Cat-vLLM has no glm5_next model class, while NVIDIA Flash is Glm5NextForConditionalGeneration.
  2. Quantization gap (blocking): the checkpoint carries W4A4-oriented activation-scale metadata, while skinny QPN2/Marlin is W4A16 and intentionally discards input_scale. A W4A16 POC is allowed, but only with an explicit scale treatment and a real-tensor accuracy reference; it must not claim W4A4 equivalence.
  3. MoE gap (blocking): 36,288 routed-expert linears are the dominant target weight set; skinny QPN2/QPN8 custom linear dispatch does not cover routed expert grouped GEMM. Existing ModelOpt MoE selection is not a validated SM70 path for this target.
  4. Attention/state gap (blocking): Qwen GDN metadata and FlashAttention V100 assumptions do not establish GLM KDA/DSA support. FP8-KV metadata must be deliberately converted to a validated Volta storage policy, not merely inherited from Qwen.
  5. MTP accounting/deferment: target layer 45 is BF16 and has a different model/state contract; Qwen chain-MTP fast metadata cannot be assumed. MTP is optional for the initial text POC and may be explicitly deferred, but its tensors and the deferment must be recorded in the loader census.
  6. Scale-contract gap: any GLM adapter must reject logical-width/scale mismatches. It must not use the existing reference scales.max() fallback or silently requantize expert weights.
  7. Parallelism gap: TP10/TP12 divisibility and EP/PP layout have not been proven; raw HBM arithmetic is insufficient. The 10-GPU requirement is a future POC constraint, not a current launch recipe.
  8. Baseline environment gate: the fleet bootstrap must verify the isolated torch build’s actual sm_70 arch list and tiny FP16 CUDA op before JIT or model work. Host Python 3.13 / torch 2.12.1+cu130 is not an accepted baseline.
  9. Storage/download: the NVIDIA Flash artifact needs about 204.44 GB of shard storage before cache/temp overhead. The generic RadixArk artifact is about 464.87 GB and should not be selected accidentally.

RECOMMENDATION

  1. Complete and gate the isolated Qwen TP4 reference environment. If only the idle TP2 pair is available, run a clearly labeled smoke test and preserve the published TP4 claim as untested on this host.
  2. Treat GLM-5.3-Flash on V100 as blocked pending a separate GLM5Next + validated W4A16-semantics MoE/attention adapter (or a separately chosen native W4A4 path), not as a checkpoint swap. Acceptance is a real-tensor reference under the declared semantics, not W4A4 equivalence. Keep the exact NVIDIA revision pinned before any 204 GB download.
  3. Do not enable lm_head requantization or borrow lm_head data from another checkpoint. The target lm_head is excluded from quantization and must stay on its own published storage/loader path.
  4. Require a loader census that proves, per module and per expert: tensor shape, activation-scale use, weight-scale ownership, route/backend, and peak resident bytes. Fail closed on missing scales, width mismatch, or fallback backend.
  5. Only after the single-module and attention-state gates pass should the coordinator evaluate 10-GPU and 12-GPU parallel layouts and full model storage. A successful Qwen baseline is evidence that the Volta kernel stack works, not evidence that GLM fits or computes correctly.