On one RTX PRO 6000 with SGLang, DFlash2 with 8 draft tokens had the highest output rate for stock Qwen3.8-27B NVFP4 in every scenario I ran, from short chat to a 120k-token prompt. It was ahead of the model’s own MTP head at 3 and 7 steps, DFlash2 with 4 draft tokens, and RadixArk’s DSpark. On a 205-case tool-calling subset (BFCL) BFCL’s automatic function-call graders passed 171–176 of 205 cases across the six configs, a spread this run can’t resolve. These are results for these checkpoints on this stack, not for the drafter formats in general.

The end-to-end gain drops from 2.5x on chat to 1.16x at 120k. The long wait before the first token explains most of that, not the drafter: at 1 user, median time per output token stays 2.5–2.7x lower at every length. From 40k up the lead over the nearest drafter is only 3–4% from one run per config, too small to separate from run-to-run variation (see Caveats).

Results

Aggregate output tok/s across all requests in the scenario, including prompt-prefill time (one run per config; 1 or 4 concurrent requests as labeled). Bold is the highest observed rate.

Mean accept length (tokens per verify step, 1 user):

Takeaways

• Acceptance held roughly flat with context, on a repeated ShareGPT prompt. Swings from chat to 120k are 0.15–0.43 per drafter. Only DFlash2 4 drifts down (2.81 to 2.66); DFlash2 8 doesn’t. The 20k–120k rows are one prompt repeated, and real prompts are only checked up to 32k (see the update below).
• At 1 user the drafter speeds up decode 2.5–2.7x at every length. Median time per output token for DFlash2 8 is 5.4–6.5 ms against 13.8–16.5 ms without a drafter. At 20k with 4 users it’s 15.3 vs 20.5 ms, only 1.34x.
• So why does the end-to-end gain shrink? These requests are 512 output tokens behind a long prompt. At 120k, median time to first token (TTFT) is 27.8 s with DFlash2 8 and 27.6 s without a drafter, while 512 × median time per output token is about 3.3 s with it and 8.5 s without. TTFT is a proxy, not a separately timed prefill phase, but the drafter only speeds up the part after it.
• More accepted tokens isn’t faster. MTP 7 accepts more per step than DFlash2 8 at 40k, 80k and 120k (3.65, 3.79 and 3.55 vs 3.28, 3.43 and 3.46) and is still slower everywhere, because its time per token is higher (6.9–8.3 ms at 1 user vs 5.4–6.5).
• DSpark is 1–5% ahead of DFlash2 4 in five of seven rows and level in the other two. Its accept length is about MTP 3’s. It also has the slowest BFCL calls of all six configs, slower than no drafter (3.24 s mean vs 2.53 s; DFlash2 8: 0.91 s). I haven’t looked into why.

The drafter costs KV cache

Same --mem-fraction-static 0.41 for every config. The draft model and verify buffers come out of the same budget:

DFlash2 8 gives up 26% of the KV pool and 76% of the mamba slots (max_total_num_tokens and mamba_cache_size from each server.log). The largest scenarios above (4 × 20k, 1 × 120k) fit in the 237k-token pool. I haven’t measured what it costs in multi-turn prefix cache hits.

Is the 3.4 accept length a thinking or sampling artifact?

The DFlash2 model card reports 4.10 on MT-Bench (H200, xhigh reasoning, block size 8 with 7 draft tokens per step, the setting I call DFlash2 8). I get 3.43 on ShareGPT at medium. So I reran DFlash2 8 with thinking off and with temperature 0:

Neither one closes the gap. The update below narrows where it could come from.

Update (2026-10-30): On the DFlash2 card’s own five sets and protocol, the same stock NVFP4 setup lands within 0.18 of its accept length on all five sets (MT-Bench 4.13 vs 4.10), which makes a large hardware-or-quant penalty less plausible. The ShareGPT gap could come from the different prompts, reasoning effort, sampling or output limit; I haven’t isolated which. SPEED-Bench’s real prompts from 1k to 32k tokens accept 3.9–4.3 (12 prompts per length, standard error 0.2–0.3), so acceptance staying roughly flat with context holds on real prompts up to 32k too. The 20k–120k rows above repeat one ShareGPT prompt to length, so their accept lengths aren’t representative. The 80k and 120k rows, including the ~2.5x decode speedup there, are still repeated-prompt measurements; I haven’t rerun them on real prompts. Details: Swift 1.5 vs stock.

Setup

Serving stack:

• Image: lmsysorg/sglang:nightly-dev-cu13-20261010-961bcf48 for every config (it has both DFLASH and DSPARK).
• Target: RadixArk/Qwen3.8-27B-NVFP4 @ 319f741c.
• Drafters: z-lab/Qwen3.8-27B-DFlash2 @ 50307d4c, RadixArk/Qwen3.8-27B-DSpark @ b9a5dbdf. MTP uses the head built into the target checkpoint.
• Serving-test requests use reasoning_effort=medium (BFCL does not, see below).

Shared flags, the same as the first post:

python3 -m sglang.launch_server --model-path <checkpoint>
  --mem-fraction-static 0.41 --max-running-requests 4 --context-length 153600
  --dtype auto --attention-backend flashinfer
  --reasoning-parser qwen3 --tool-call-parser qwen3_coder
  --mamba-full-memory-ratio 0.9 --mamba-ssm-dtype bfloat16
  --mamba-radix-cache-strategy extra_buffer_lazy
  --chunked-prefill-size 2048 --page-size 64
  --enable-hierarchical-cache --hicache-size 64 --hicache-write-policy write_through
  --kv-cache-dtype fp8_e4m3 --schedule-policy lpm --enable-session-radix-cache
  --sleep-on-idle --trust-remote-code

Then the speculative flags, appended after the shared ones. The number in a name is the verify width: --speculative-num-draft-tokens for DFlash2 (4, 8) and steps for MTP (3, 7), so MTP 3 and DFlash2 4 both verify 4 tokens per step, and MTP 7 and DFlash2 8 both verify 8.

Serving: python3 -m sglang.benchmark.serving --backend sglang-oai-chat --flush-cache --warmup-requests 2 --seed 1.

• Chat: ShareGPT prompts, 1024 output tokens; 12 prompts at 1 user, 32 at 4 users.
• 20k prompt: SGLang random dataset (one ShareGPT prompt repeated to length), 20,000-token input, 512 output tokens; 4 prompts at 1 user, 8 at 4 users.
• 40k, 80k, 120k prompt: same random dataset at that length, 512 output tokens, 4 prompts at 1 user.

BFCL (Berkeley Function Calling Leaderboard) scores are passes by BFCL’s own grader against its reference tool calls, not human judgments. It ran separately from the serving tests: bfcl-eval 2026.3.23, the Qwen/Qwen3-32B-FC handler at its default effort and temperature 0.001, not reasoning_effort=medium. It doesn’t test general answer quality or whether outputs are identical. Same fixed 205-case subset as the first post (case ids).

Caveats

• One run per config. The 40k–120k scenarios have 4 prompts each, so their rates carry a few percent of noise, and DFlash2 8’s measured leads over the nearest drafter were 3.2% at 40k, 3.8% at 80k and 3.4% at 120k. I can’t distinguish leads that small from run-to-run variation. The same config reran in the other two posts: 20k at 4 users came out 142.3 and 138.7 against 137.5 here, and BFCL 174 and 168 against 176. BFCL categories have 10–25 cases, so 171 vs 176 is not enough to establish a difference or equivalence.
• DSpark runs on the shared flags so only the drafter changes. Its model card serves it with float32 mamba state, --mamba-radix-cache-strategy extra_buffer, --mamba-full-memory-ratio 11.93 and 32k prefill chunks. With those it might do better than here.
• 400 W power limit, not the default 600 W.
• Output lengths are fixed (ignore_eos), so this measures speed only.
• Deep-AI-Evo found MTP slower than no MTP at long context on the same card: 18.2 vs 43.7 tok/s decode at ~177k tokens, NVFP4, MTP k=2 (repo). I don’t see that up to 120k: MTP 3 keeps decode at 8.5 ms per token vs 16.5 without a drafter. They ran vLLM (0.21 and 0.26) with unsloth/Qwen3.8-27B-NVFP4 and went to ~177k.

Raw data

• Scripts, configs and raw results: adcostanza/benchmarks. run.sh reruns every config, about two hours.
• The scripts on main keep changing. To rerun with them exactly as these runs had them, check out 4b5cf81.
• results/<label>/ per config: bench-serving.jsonl (one line per scenario, full server args included), server.log, and bfcl/ with every model response.

Test system

• GPU: NVIDIA RTX PRO 6000 Blackwell Workstation Edition, 96 GB, power limit 400 W (default 600 W). Nothing else on the GPU, one config at a time.
• PCIe: Gen 4 x16 (the card supports Gen 5; nvidia-smi reports Gen 4 as this system’s maximum).
• CPU: AMD Ryzen 9 7950X, 16 cores, 32 threads.
• RAM: 250 GiB DDR5.
• OS: Ubuntu 24.04.2, kernel 6.8.0-147.
• NVIDIA: driver 580.178.04, CUDA 13.0.
• Containers: rootless Docker 28.0.4, NVIDIA Container Toolkit 1.18.1, GPU through CDI.
• Full list: hardware.md at the commit these runs used.