We started with one borrowed machine and a question: what could we serve well on eight H200s? The first useful result was not a model benchmark. It was a topology map. We had two islands of four NVLink-connected GPUs, with a much slower crossing between them.
That discovery set the direction for the next three days. We brought up smaller models, pushed prompts toward a million tokens, tried the giants, changed parallelism layouts, and followed the failures into kernels, memory pools and request harnesses. Some promising recipes improved the numbers. Some did not boot. Others produced attractive results that failed our own publication gate.
This is that journey in 15 parts, preserving the experiment stories, commands, charts and limitations. Parts 1-13 describe the original 9-11 October 2026 campaign. The unnumbered afterword records the loader rescues and 256K follow-up; Parts 14-15 continue through the extended, recovery and two-hour tests completed on 12 October, including the later GLM SGLang rescue now added to Part 14. Read in order to follow the decisions, or use the chapter list below to return to a particular problem.
Start with Part 1: read the machine
A second kind of long-context test
Reading a fresh million-token document and growing a cached conversation are different workloads. The supplementary conversation-ramp report retains the per-turn measurements referenced throughout the series. It sits outside the numbered sequence so the original experiment stays easy to find.
After the campaign: the next run changed the conclusion
The original chapters end where the campaign ended. The afterword follows the problems we returned to: a missing GLM replica baseline, a quality check that accepted too much, and DeepSeek loaders that interpreted tensor scales incorrectly.
Those fixes led to different deployment choices. Two GLM TP4 replicas improved throughput and first-token latency against the saved prefill/decode split, with an important CUDA-graph difference between the configurations. For DeepSeek-V4.1-Flash, the native checkpoint fit inside one four-GPU quad with Engram CPU offload. Adding DSpark raised single-request output throughput from 116 to 269 tokens/s on the small chat test; two replicas without speculation reached 2,788 aggregate tokens/s on a separate synthetic test at concurrency 64.
The afterword keeps the workload sizes, GPU counts, correctness checks and limits beside those results. The earlier 32K tests are followed by a matched two-replica DSpark comparison at a 256K configured ceiling, with retrieval checked up to 250K and three short sustained-load windows per configuration; neither stage replaces the original million-token measurements or establishes that every configuration has been tested.
Latest update: the GLM SGLang retry worked
We returned to the incomplete GLM-5.3-Flash engine result in Part 14. The original recipe reproduced its CUDA crash, but enabling prefix caching or switching to EP4 each completed three sweeps with 2,424 successful requests and all 96 JSON/tool probes passed. Disabling overlap or merely enlarging the state pool did not fix it.
A cache-enabled EP4 recipe also passed with five-step MTP speculation. Its median single-request output rate rose from 107.0 to 208.8 tokens/s, while C128 throughput was slightly lower and reported KV capacity fell from 1.60 million to 241,344 tokens. The chapter explains the hidden state-pool limit, the compilation stalls and why the best recipe depends on the workload. The new evidence JSON and per-repeat CSV preserve both the successful retries and the failed attempts.
DeepSeek verdict: the fastest load did not pass the final check
Part 14 extends the DeepSeek ceiling to 1M, tests concurrency through 256, compares draft lengths and engines, and explains why zero failures in a finite benchmark ledger did not make every recipe usable. Four recovery configurations passed the complete short-test sequence; others still returned incomplete JSON or failed in the engine.
Part 15 follows the corrected two-hour tests at concurrency 128. DSpark K3 completed 128,156 short-chat load requests without a recorded request failure at 4,237.158 output tokens/s, then failed one post-load JSON request with HTTP 500. The baseline completed 64,507 load requests at 2,129.887 tokens/s and passed all before/after JSON and tool checks. It is the more conservative handoff from this evidence.
The recipes used different sequence limits and memory-allocation targets, so the throughput gap is not an isolated speculation speed-up. A 1M configured ceiling did not make this a 128-user million-token test, and one two-hour run does not establish a production SLA. The extended evidence JSON and results CSV keep the successful measurements and failed states together.
Read the evidence with the story
The original collector produced 683 rows across 87 runs and 14 model variants. It marked 592 rows as passing our publication filters (the needle gate and a 60K tokens/s cache screen) and 22 invalid. The cache screen only catches implausibly fast prefills, so short-prompt first-token times may still include undetected prefix-cache reuse. These are separate flags; the remaining rows should not be silently promoted into claims. The chapters preserve exclusions, provisional MiniMax figures, small sample sizes and missing controlled baselines where they matter.
The curated chart-data JSON records selected rows, run IDs, units and comparison notes. The GLM rescue comparison CSV contains its four measured comparison points. The DeepSeek follow-up CSV adds the completed native-checkpoint and speculative-decoding tests. These extracts support the overview and later rescue discussion; the original chapter tables and source lists retain their own workload conditions. Neither download replaces the full campaign logs.
Leapswitch Networks sponsors this engineering series. Their support is acknowledged throughout; the measurements and their limitations remain our responsibility.