A short benchmark can tell us which configuration deserves more time. It cannot tell us what will happen after two hours, or whether the next structured request will still work. The recovery in Part 14 produced four configurations that passed its short acceptance sequence. We chose the adaptive K5 candidate for a longer run because it led C128 throughput among those candidates. That first attempt did not produce a valid reliability result.
By the end of the corrected round, we had two different findings. Nonadaptive K3 completed 128,156 load requests without a recorded request failure and produced 4,237.158 output tokens/s. It then failed one post-load JSON request with HTTP 500. The slower baseline completed 64,507 load requests and passed every recorded before/after structured check. Reporting only the larger number would have hidden the final outcome.
First, the client had to earn its concurrency claim
The initial adaptive soak encountered a client thread-creation failure, can't start new thread. That made the intended 128-worker load untrustworthy. A concurrency setting in a script is not proof that the requested number of workers actually started and sustained pressure on the server.
The server also later reported an illegal-memory failure during an asynchronous output copy on replica 1, about 17 minutes into that attempt. That location does not identify the kernel or operation that originally caused the memory fault. We retained both facts. The client defect invalidated the intended load measurement; it did not explain away the GPU failure. Conversely, the GPU failure did not repair the missing evidence about client concurrency. There was no defensible two-hour success number to recover from that attempt.
We corrected the generator with a startup barrier: all 128 worker threads had to start before the timed issuing phase could begin. The final runs verified that condition. This guard establishes that the client reached the requested worker count; it does not mean 128 requests were executing on GPUs at every instant, since admission, queueing and request completion still intervene.
Change the acceptance boundary before reading the result
The corrected comparison used a nonadaptive K3 configuration and the no-speculation baseline. Both served the native DeepSeek-V4.1-Flash checkpoint at revision 2cba9e42aa026125f3ed06c6d98c1db82f7ca027, with the same patched vLLM image, Engram CPU offload, FP8 KV, CUDA graphs and disabled prefix caching. Both allocated all eight GPUs as two quad-local TP4 replicas and configured a 1,048,576-token ceiling.
They were not otherwise identical. The baseline allowed 32 sequences per replica with a 95% GPU-memory allocation target. K3 allowed 64 per replica at 90%, as well as enabling three-token drafting. At client concurrency 128, the baseline had only 64 total sequence slots while K3 had 128. That difference is directly relevant to waiting time. This is a comparison of two serving recipes, not an isolated estimate of speculation’s contribution.
We issued requests for 7,200 seconds, then drained the outstanding work. The fixed 256-prompt ShareGPT pool contained short prompts; responses were capped at 256 output tokens with EOS honored. Generated lengths therefore varied. The throughput denominator includes the drain time, and the test cycles a fixed pool rather than recreating a production arrival distribution. A million-token engine ceiling does not turn these short requests into a million-token concurrency test.
| Measurement | Baseline · 32 slots/replica · 95% allocation | DSpark K3 · 64 slots/replica · 90% allocation |
|---|---|---|
| Completed requests | 64,507 | 128,156 |
| Load-phase request failures | 0 | 0 |
| Elapsed including drain | 7,211.57 s | 7,204.78 s |
| Aggregate output tokens/s | 2,129.887 | 4,237.158 |
| P99 first-token latency | 7.854 s | 0.396 s |
| P99 whole-request latency | 15.604 s | 10.871 s |
| JSON before / after | 24/24 / 24/24 | 24/24 / 23/24 |
| Tools before / after | 8/8 / 8/8 | 8/8 / 8/8 |
| Whole-run verdict | Passed recorded checks | Failed post-load JSON check |
The faster recipe made better use of this particular short-chat load, but the different sequence limits and memory allocation prevent a clean “DSpark doubled performance” claim. The first-token difference is especially sensitive to admission capacity: the baseline’s P99 wait was 7.854 s, versus 0.396 s for K3. Whole-request P99 was 15.604 s versus 10.871 s. Those are observations of the complete configurations; we did not profile an exact causal decomposition.
Zero load failures was not the last check.
- 01 · Before loadBoth passed
24/24 strict JSON tasks and 8/8 tool calls.
- 02 · Two-hour loadBoth completed
Baseline: 64,507 requests. K3: 128,156. Zero recorded load-request failures.
- 03 · After loadOnly baseline fully passed
K3: 23/24 JSON because one request returned HTTP 500; 8/8 tools still passed.
The final JSON check changed the verdict
K3 entered the load with 24/24 JSON tasks and 8/8 tool calls passing. It ended the issuing/drain phase with zero recorded request failures. The subsequent structured check returned 23/24 JSON and 8/8 tools, with one JSON request receiving an HTTP 500 response.
That is an availability failure in this probe, not evidence that the model generated an incorrect JSON value. We do not know from the score alone which server component produced the failure, or whether retrying would have succeeded. We also do not average it into a tiny error percentage over the much larger load request count: the post-check was a separate acceptance gate designed to exercise a different contract.
The queue therefore correctly marked the K3 run failed despite its successful load summary. This distinction is useful operationally. An engine may serve many short completions and still fail when the request format or code path changes. Treating the post-check as optional would make the measured throughput easier to advertise and the handoff harder to trust.
The baseline passed 24/24 JSON and 8/8 tool calls both before and after its own two-hour run. Its load completed without recorded failures, so it passed this complete test sequence. That makes it the more conservative handoff from the evidence we have. It does not make a claim about all tasks, all input lengths or every future request.
What two hours can and cannot buy us
We now have a successful extended check for one baseline recipe and a faster candidate with a specific unresolved post-load failure. The repeated short tests from Part 14 help explain why both were worth running; they cannot override the outcome of the final gate. The original adaptive attempt remains invalid as a load measurement and failed as a server attempt, rather than being merged with the corrected K3 result.
Neither final run establishes a production SLA. Each recipe had one two-hour measurement under a fixed short-prompt pool, not randomized alternating trials, multi-day load, realistic arrival bursts or fault injection. The earlier 510K/1M retrieval checks belonged to baseline32 and K5/32 in the extension round; they were not repeated across 128 concurrent million-token sessions here. Structured probes were narrow contract tests, not broad reasoning evaluations.
Our next action would be to reproduce and diagnose K3’s post-load HTTP 500 with the complete request and server trace, then repeat the full before/load/after sequence. A matched baseline with the same slots and memory target would help isolate the speculative path. Longer mixed traffic and repeated long runs would answer different questions and should keep their own gates.
Keep the failure next to the successful measurement
The extended findings JSON retains both load summaries, quality outcomes and queue states. The results CSV carries the concise measurements with their status labels. The corrected raw record is in h200-reliability-20261012; the invalid adaptive attempt remains in h200-recovery-20261012.
The final run IDs are dsv41_flash__ext_dspark_k3_s64__1024k__vllm__reliability-confirm__1012-064215 and dsv41_flash__ext_base_s32__1024k__vllm__reliability-confirm__1012-084845. Keep those identities when comparing a future fix. This journey started by finding two communication islands inside an eight-GPU server. It ends, for now, with an equally concrete boundary: the fastest measured load was not the configuration that passed the last check. Return to the series overview for the current reading order and verdict.