On one RTX PRO 6000 with SGLang, DFlash2 with 8 draft tokens is the fastest drafter for stock Qwen3.8-27B NVFP4 in every scenario I ran, from short chat to a 120k-token prompt. It beats the model’s own MTP head at 3 and 7 steps, DFlash2 with 4 draft tokens, and RadixArk’s DSpark. Tool calling (BFCL) doesn’t move.

The end-to-end gain drops from 2.5x on chat to 1.16x at 120k. That’s prefill, not the drafter: time per output token stays about 2.5x faster at every length.

Results

Output tok/s, one run per config. Bold is the fastest.

Mean accept length (tokens per verify step, 1 user):

Takeaways

• Acceptance doesn’t fall with context. No drafter shows a trend from chat to 120k, only swings of 0.1–0.3. Median time per output token for DFlash2 8 is 5.4–6.5 ms against 13.9–16.5 ms without a drafter, i.e. 2.5–2.7x faster at every length.
• So why does the gain shrink? Because these requests are 512 output tokens behind a long prompt. At 120k, prefill takes 27.8 s and the 512 tokens take about 3.3 s. The drafter only speeds up the 3.3 s. Output tok/s is total output over wall time, so prefill eats the gain. With longer answers the end-to-end number would sit closer to the decode number.
• More accepted tokens isn’t faster. MTP 7 accepts more per step than DFlash2 8 at 40k and 80k (3.65 and 3.79 vs 3.28 and 3.43) and is still slower everywhere, because each MTP step costs more (6.9–8.3 ms per token at 1 user vs 5.4–6.5).
• DSpark doesn’t beat DFlash2 4 here. It accepts about as many tokens as MTP 3. It also has the slowest BFCL calls of all six configs, slower than no drafter (3.24 s mean vs 2.53 s; DFlash2 8: 0.91 s). I haven’t looked into why.

DFlash2 with 8 draft tokens is what I run day to day, and nothing here changes that.

The drafter costs KV cache

Same --mem-fraction-static 0.41 for every config. The draft model and verify buffers come out of the same budget:

DFlash2 8 gives up 26% of the KV pool and 76% of the mamba slots (max_total_num_tokens and mamba_cache_size from each server.log). With 4 running requests and a 153,600-token context that’s still enough for every scenario above. I haven’t measured what it costs in multi-turn prefix cache hits.

Is the low accept length a thinking or sampling artifact?

The DFlash2 card reports 4.10 on MT-Bench with 7 draft tokens (H200, xhigh reasoning). I get 3.43 on ShareGPT at medium. So I reran DFlash2 8 with thinking off and with temperature 0:

Neither one closes the gap. The prompts (ShareGPT vs MT-Bench) and the hardware differ, so the card’s number isn’t a target I should expect to hit.

Setup

Hardware: same machine as the NVFP4 vs FP8 post. RTX PRO 6000 Blackwell 96 GB at a 400 W power limit, Ryzen 9 7950X, 250 GiB DDR5, Ubuntu 24.04.2, driver 580.178.04, rootless Docker. Full list in hardware.md.

Serving stack:

• Image: lmsysorg/sglang:nightly-dev-cu13-20261010-961bcf48 for every config (it has both DFLASH and DSPARK).
• Target: RadixArk/Qwen3.8-27B-NVFP4 @ 319f741c.
• Drafters: z-lab/Qwen3.8-27B-DFlash2 @ 50307d4c, RadixArk/Qwen3.8-27B-DSpark @ b9a5dbdf (DSpark v2). MTP uses the head built into the target checkpoint.
• Requests use reasoning_effort=medium.

Shared flags, the same as the first post:

python3 -m sglang.launch_server --model-path <checkpoint>
  --mem-fraction-static 0.41 --max-running-requests 4 --context-length 153600
  --dtype auto --attention-backend flashinfer
  --reasoning-parser qwen3 --tool-call-parser qwen3_coder
  --mamba-full-memory-ratio 0.9 --mamba-ssm-dtype bfloat16
  --mamba-radix-cache-strategy extra_buffer_lazy
  --chunked-prefill-size 2048 --page-size 64
  --enable-hierarchical-cache --hicache-size 64 --hicache-write-policy write_through
  --kv-cache-dtype fp8_e4m3 --schedule-policy lpm --enable-session-radix-cache
  --sleep-on-idle --trust-remote-code

Then the speculative flags, appended after the shared ones:

Serving: python3 -m sglang.benchmark.serving --backend sglang-oai-chat --flush-cache --warmup-requests 2 --seed 1.

• Chat: ShareGPT prompts, 1024 output tokens; 12 prompts at 1 user, 32 at 4 users.
• 20k prompt: random 20,000-token input, 512 output tokens; 4 prompts at 1 user, 8 at 4 users.
• 40k, 80k, 120k prompt: random input, 512 output tokens, 4 prompts at 1 user.

BFCL: bfcl-eval 2026.3.23, the Qwen/Qwen3-32B-FC handler, the same fixed 205-case subset as the first post (case ids).

Caveats

• One run per config. The 40k–120k scenarios have 4 prompts each, so their rates carry a few percent of noise. BFCL categories have 10–25 cases, so 171 vs 176 is noise.
• DSpark runs on the shared flags so only the drafter changes. Its model card serves it with float32 mamba state, --mamba-radix-cache-strategy extra_buffer, --mamba-full-memory-ratio 11.93 and 32k prefill chunks. With those it might do better than here.
• 400 W power limit, not the default 600 W.
• Output lengths are fixed (ignore_eos), so this measures speed only.
• Deep-AI-Evo found MTP slower than no MTP at long context on the same card: 18.2 vs 43.7 tok/s decode at ~177k tokens, NVFP4, MTP k=2 (repo). I don’t see that up to 120k: MTP 3 keeps decode at 8.5 ms per token vs 16.5 without a drafter. Their stack is vLLM (0.21 and 0.26) with unsloth/Qwen3.8-27B-NVFP4, and I stopped at 120k, so this doesn’t contradict them.

Raw data

• Scripts, configs and raw results: adcostanza/benchmarks. run.sh reruns every config, about two hours.
• results/<label>/ per config: bench-serving.jsonl (one line per scenario, full server args included), server.log, and bfcl/ with every model response.

On one RTX PRO 6000 with SGLang and DFlash2 speculative decoding, NVFP4 is 32–37% faster than FP8 for stock Qwen3.8-27B, on short chat and on 20k-token prompts. BFCL tool calling stays within noise (174 vs 167 of 205).

Results

Stock Qwen3.8-27B: Qwen/Qwen3.8-27B-FP8 vs RadixArk/Qwen3.8-27B-NVFP4.

Takeaways

• The gain holds with a 20k-token prompt. Time to first token drops 27–32%, so prefill gets faster as well as decode.
• DFlash2 acceptance barely moves (accept length 3.49 vs 3.41), so the gain comes from the target model, not the drafter. Median time per output token in chat at 1 user drops from 7.30 ms to 5.59 ms.
• BFCL shows no measurable change. Most of the +7 is irrelevance (12 → 18) and multi_turn_base (14 → 16), categories of 20 cases each.

BFCL by category

Setup

Hardware and system:

• GPU: NVIDIA RTX PRO 6000 Blackwell Workstation Edition, 96 GB. PCIe Gen 4 x16 (the card supports Gen 5; nvidia-smi reports Gen 4 as this system’s maximum). Power limit 400 W, set at boot (default 600 W). Idle during runs, one config at a time.
• CPU: AMD Ryzen 9 7950X (16 cores, 32 threads).
• RAM: DDR5, 250 GiB usable.
• Board: Gigabyte B650 AORUS ELITE AX.
• Models on a Crucial P3 Plus 4 TB NVMe (ext4 on LUKS).
• OS: Ubuntu 24.04.2, kernel 6.8.0-147, NVIDIA driver 580.178.04 (CUDA 13.0), rootless Docker 28.0.4, NVIDIA Container Toolkit 1.18.1.

Serving stack:

• Image: lmsysorg/sglang:nightly-dev-cu13-20261010-961bcf48.
• Checkpoints: Qwen/Qwen3.8-27B-FP8 @ 017b9c7a, RadixArk/Qwen3.8-27B-NVFP4 @ 319f741c.
• Speculative decoding: DFlash2 (z-lab/Qwen3.8-27B-DFlash2), 8 draft tokens, for every run.
• --mem-fraction-static 0.50 for FP8, 0.41 for NVFP4, so both get about the same KV pool (240,512 vs 236,864 tokens, 28 mamba slots each).
• Requests use reasoning_effort=medium.

Launch flags:

python3 -m sglang.launch_server --model-path <checkpoint>
  --mem-fraction-static <0.50|0.41> --max-running-requests 4 --context-length 153600
  --dtype auto --attention-backend flashinfer
  --reasoning-parser qwen3 --tool-call-parser qwen3_coder
  --speculative-algorithm DFLASH --speculative-draft-model-path <DFlash2>
  --speculative-num-draft-tokens 8
  --mamba-full-memory-ratio 0.9 --mamba-ssm-dtype bfloat16
  --mamba-radix-cache-strategy extra_buffer_lazy
  --chunked-prefill-size 2048 --page-size 64
  --enable-hierarchical-cache --hicache-size 64 --hicache-write-policy write_through
  --kv-cache-dtype fp8_e4m3 --schedule-policy lpm --enable-session-radix-cache
  --sleep-on-idle --trust-remote-code

Serving: python3 -m sglang.benchmark.serving --backend sglang-oai-chat --flush-cache --warmup-requests 2 --seed 1.

• Chat: ShareGPT prompts, 1024 output tokens; 12 prompts at 1 user, 32 at 4 users.
• 20k prompt: random 20,000-token input, 512 output tokens; 4 prompts at 1 user, 8 at 4 users.

BFCL: bfcl-eval 2026.3.23, the Qwen/Qwen3-32B-FC handler, a fixed 205-case subset across 10 categories (case ids).

Caveats

• One run per config. Categories have 10–25 cases, so a few points either way is noise.
• The GPU ran at a 400 W power limit, not the default 600 W. I haven’t measured either format at 600 W.
• Output lengths are fixed (ignore_eos), so tok/s compares speed only, not how much each model thinks.
• BFCL uses its own Qwen3 FC prompt, not the model’s chat template. Compare these scores with each other, not with the public leaderboard.
• Deep-AI-Evo measured a smaller gap on the same card with vLLM 0.21 and unsloth/Qwen3.8-27B-NVFP4: 50.9 → 56.4 tok/s without MTP, 115.8 → 132.3 with MTP k=2, single stream (repo). The engine, checkpoint and speculative decoding all differ from mine, and I haven’t tested which of them explains the difference.

Raw data

• Scripts, configs and raw results: adcostanza/benchmarks. run.sh there reruns both configs.
• Serving results (one JSON line per scenario, including full server args): FP8, NVFP4.
• BFCL scores and every model response: results/<label>/bfcl/ in the same folder.