On one RTX PRO 6000 with SGLang, DFlash2 with 8 draft tokens is the fastest drafter for stock Qwen3.8-27B NVFP4 in every scenario I ran, from short chat to a 120k-token prompt. It beats the model’s own MTP head at 3 and 7 steps, DFlash2 with 4 draft tokens, and RadixArk’s DSpark. Tool calling (BFCL) doesn’t move.

The end-to-end gain drops from 2.5x on chat to 1.16x at 120k. That’s prefill, not the drafter: time per output token stays about 2.5x faster at every length.

Results

Output tok/s, one run per config. Bold is the fastest.

Mean accept length (tokens per verify step, 1 user):

Takeaways

• Acceptance doesn’t fall with context. No drafter shows a trend from chat to 120k, only swings of 0.1–0.3. Median time per output token for DFlash2 8 is 5.4–6.5 ms against 13.9–16.5 ms without a drafter, i.e. 2.5–2.7x faster at every length.
• So why does the gain shrink? Because these requests are 512 output tokens behind a long prompt. At 120k, prefill takes 27.8 s and the 512 tokens take about 3.3 s. The drafter only speeds up the 3.3 s. Output tok/s is total output over wall time, so prefill eats the gain. With longer answers the end-to-end number would sit closer to the decode number.
• More accepted tokens isn’t faster. MTP 7 accepts more per step than DFlash2 8 at 40k and 80k (3.65 and 3.79 vs 3.28 and 3.43) and is still slower everywhere, because each MTP step costs more (6.9–8.3 ms per token at 1 user vs 5.4–6.5).
• DSpark doesn’t beat DFlash2 4 here. It accepts about as many tokens as MTP 3. It also has the slowest BFCL calls of all six configs, slower than no drafter (3.24 s mean vs 2.53 s; DFlash2 8: 0.91 s). I haven’t looked into why.

DFlash2 with 8 draft tokens is what I run day to day, and nothing here changes that.

The drafter costs KV cache

Same --mem-fraction-static 0.41 for every config. The draft model and verify buffers come out of the same budget:

DFlash2 8 gives up 26% of the KV pool and 76% of the mamba slots (max_total_num_tokens and mamba_cache_size from each server.log). With 4 running requests and a 153,600-token context that’s still enough for every scenario above. I haven’t measured what it costs in multi-turn prefix cache hits.

Is the low accept length a thinking or sampling artifact?

The DFlash2 card reports 4.10 on MT-Bench with 7 draft tokens (H200, xhigh reasoning). I get 3.43 on ShareGPT at medium. So I reran DFlash2 8 with thinking off and with temperature 0:

Neither one closes the gap. The prompts (ShareGPT vs MT-Bench) and the hardware differ, so the card’s number isn’t a target I should expect to hit.

Setup

Hardware: same machine as the NVFP4 vs FP8 post. RTX PRO 6000 Blackwell 96 GB at a 400 W power limit, Ryzen 9 7950X, 250 GiB DDR5, Ubuntu 24.04.2, driver 580.178.04, rootless Docker. Full list in hardware.md.

Serving stack:

• Image: lmsysorg/sglang:nightly-dev-cu13-20261010-961bcf48 for every config (it has both DFLASH and DSPARK).
• Target: RadixArk/Qwen3.8-27B-NVFP4 @ 319f741c.
• Drafters: z-lab/Qwen3.8-27B-DFlash2 @ 50307d4c, RadixArk/Qwen3.8-27B-DSpark @ b9a5dbdf (DSpark v2). MTP uses the head built into the target checkpoint.
• Requests use reasoning_effort=medium.

Shared flags, the same as the first post:

python3 -m sglang.launch_server --model-path <checkpoint>
  --mem-fraction-static 0.41 --max-running-requests 4 --context-length 153600
  --dtype auto --attention-backend flashinfer
  --reasoning-parser qwen3 --tool-call-parser qwen3_coder
  --mamba-full-memory-ratio 0.9 --mamba-ssm-dtype bfloat16
  --mamba-radix-cache-strategy extra_buffer_lazy
  --chunked-prefill-size 2048 --page-size 64
  --enable-hierarchical-cache --hicache-size 64 --hicache-write-policy write_through
  --kv-cache-dtype fp8_e4m3 --schedule-policy lpm --enable-session-radix-cache
  --sleep-on-idle --trust-remote-code

Then the speculative flags, appended after the shared ones:

Serving: python3 -m sglang.benchmark.serving --backend sglang-oai-chat --flush-cache --warmup-requests 2 --seed 1.

• Chat: ShareGPT prompts, 1024 output tokens; 12 prompts at 1 user, 32 at 4 users.
• 20k prompt: random 20,000-token input, 512 output tokens; 4 prompts at 1 user, 8 at 4 users.
• 40k, 80k, 120k prompt: random input, 512 output tokens, 4 prompts at 1 user.

BFCL: bfcl-eval 2026.3.23, the Qwen/Qwen3-32B-FC handler, the same fixed 205-case subset as the first post (case ids).

Caveats

• One run per config. The 40k–120k scenarios have 4 prompts each, so their rates carry a few percent of noise. BFCL categories have 10–25 cases, so 171 vs 176 is noise.
• DSpark runs on the shared flags so only the drafter changes. Its model card serves it with float32 mamba state, --mamba-radix-cache-strategy extra_buffer, --mamba-full-memory-ratio 11.93 and 32k prefill chunks. With those it might do better than here.
• 400 W power limit, not the default 600 W.
• Output lengths are fixed (ignore_eos), so this measures speed only.
• Deep-AI-Evo found MTP slower than no MTP at long context on the same card: 18.2 vs 43.7 tok/s decode at ~177k tokens, NVFP4, MTP k=2 (repo). I don’t see that up to 120k: MTP 3 keeps decode at 8.5 ms per token vs 16.5 without a drafter. Their stack is vLLM (0.21 and 0.26) with unsloth/Qwen3.8-27B-NVFP4, and I stopped at 120k, so this doesn’t contradict them.

Raw data

• Scripts, configs and raw results: adcostanza/benchmarks. run.sh reruns every config, about two hours.
• results/<label>/ per config: bench-serving.jsonl (one line per scenario, full server args included), server.log, and bfcl/ with every model response.