On one RTX PRO 6000 with SGLang and DFlash2 (8 draft tokens), ukisai’s Swift 1.5 NVFP4 scores the same as stock Qwen3.8-27B NVFP4 on GSM8K and a 205-case BFCL subset (97.04% vs 96.82%, 168 vs 168) and decodes at about the same speed per token. At reasoning_effort=medium it spends 14–19% fewer thinking tokens on GSM8K and SPEED-Bench, so a response is 8–10% shorter and finishes about 10% sooner.

Results

Stock RadixArk/Qwen3.8-27B-NVFP4 vs Swift ukisai/Swift-1.5-Qwen3.8-27b-NVFP4, both with DFlash2 8, reasoning_effort=medium.

¹ The 125 of 142 turns where both models stopped on their own; the output-token row counts all 142.

Accept length on the five sets of the DFlash2 model card, with its protocol (xhigh, temperature 1.0, top-p 0.95, top-k 20, max 4096 new tokens), 32 prompts per set, 1 user. “8 draft tokens” is SGLang’s --speculative-num-draft-tokens 8, DFlash2’s block size 8 (7 drafted per step):

Mean thinking tokens per answer on the same sets, on the prompts where both models stopped on their own. At medium that’s 30–32 of 32 prompts; at xhigh 31 (MATH-500), 26 (MBPP), 24 (MT-Bench) and 21 (HumanEval), with GSM8K at 32.

Mean tokens per answer, thinking plus answer, all 32 prompts (answers cut off at 4096 count as 4096):

Takeaways

• Same speed per token; the time saved is the shorter answers. Median time per output token on SPEED-Bench is 4.49 vs 4.47 ms. Swift finishes the 110 prompts in 913 s instead of 1,016 s because it writes 10% fewer tokens. Both runs hit the 4,096-token cap on some turns (11% of stock’s, 9% of Swift’s). Cut-off answers count as 4,096, so the real gap is probably a bit larger.
• Chat at 1 user is the exception: 4% slower, accept length 3.27 vs 3.44. One run on 12 prompts, so I can’t say whether it’s real. The other serving rows in the repo (4 users, 20k, 80k, 120k) are within 1%; 40k at 1 user is +4% for Swift (4 prompts).
• The saving is mostly thinking. At medium Swift thinks 5–38% less on the DFlash2 card’s sets and 14–19% less on GSM8K and SPEED-Bench; at xhigh, 17–47% less. Total minus thinking (an estimate: completion_tokens against re-tokenized reasoning) is about the same on full GSM8K (134 vs 136) and at xhigh on GSM8K and MATH-500, but it moves elsewhere: MT-Bench at xhigh −29%, HumanEval at xhigh +26%, and MBPP at medium saves 82 tokens of which only 36 are thinking. Swift’s model card claims 24.8% fewer thinking tokens at medium and 41.9% at xhigh (card @ b4c84d42, checked 2026-10-30). That is BF16 on vLLM; the xhigh figure is its GPQA-Diamond row and the table doesn’t name a benchmark for medium.
• On three of five sets xhigh costs Swift about what medium does. Whole responses at xhigh vs medium: GSM8K 296 vs 368 tokens, MATH-500 864 vs 833, MBPP 899 vs 895. MT-Bench (1,163 vs 1,002) and HumanEval (1,454 vs 1,131) cost 16–29% more. Stock pays more for xhigh on four of five (HumanEval 2,042 vs 1,196). I didn’t measure accuracy at xhigh, and the 4,096 cap hides the longest answers.
• Stock accept length holds up on Swift. It is within 2 standard errors of stock on every set and SPEED-Bench domain, and slightly higher on all five DFlash2 card sets at xhigh. So the fine-tune didn’t hurt the stock drafter. I didn’t test a drafter tuned on Swift.
• The DFlash2 model card’s numbers reproduce here. On its prompts and protocol, stock NVFP4 lands within 0.18 of the DFlash2 card on every set, and output tok/s at 1 user is within 7% of the H200 (180–240 vs 184–236). The 3.4 in the spec-decode post is ShareGPT chat at medium. Prompts, sampling, effort and max length all differ from the DFlash2 card’s runs, so the gap is most likely the workload.
• BFCL ties at 168. Stock NVFP4 scored 174 and 176 on this config in the earlier posts, so the same config already moves by 8 cases between runs. By category, Swift gains multi_turn_base (14 → 16 of 20) and loses irrelevance (15 → 11); with 20 cases each, neither is outside noise.
• Quality here is two benchmarks. GSM8K is near ceiling and BFCL is a 205-case subset, so “same” means no difference I could see. Swift’s model card shows 82.22% vs 84.14% for the base at medium, on a benchmark it doesn’t name. I ran nothing harder.

Setup

Serving stack:

• Image: lmsysorg/sglang:nightly-dev-cu13-20261010-961bcf48.
• Checkpoints: RadixArk/Qwen3.8-27B-NVFP4 @ 319f741c, ukisai/Swift-1.5-Qwen3.8-27b-NVFP4 @ 25482027 (ukisai’s own NVFP4 build).
• Drafter: z-lab/Qwen3.8-27B-DFlash2 @ 50307d4c, 8 draft tokens, on every run.
• Flags: the launch command from the NVFP4 vs FP8 post (scripts/args/qwen38-27b.sh) with --mem-fraction-static 0.41. Both get a 236,864-token KV pool and 43.8 GB of GPU memory after load.

Benchmarks, all at reasoning_effort=medium unless noted:

• GSM8K: the full test split, sgl-eval’s prompt and grader (last \boxed{}), the model’s own sampling, max 32,768 tokens, 32 requests in flight. No answer was cut off.
• BFCL: bfcl-eval 2026.3.23, the Qwen/Qwen3-32B-FC handler, the same fixed 205-case subset as the earlier posts (case ids).
• DFlash2 card sets: GSM8K, MATH-500, HumanEval, MBPP, MT-Bench (first turn), prompt formats and shuffle from z-lab/dflash at 07ebd93, 32 prompts per set, 1 user. Once with the DFlash2 card’s protocol (xhigh), once at medium with the same sampling.
• SPEED-Bench: the qualitative split, 10 prompts per domain (110 prompts, 142 turns), max 4096 tokens, 1 user, server-default sampling.
• Serving: python3 -m sglang.benchmark.serving --backend sglang-oai-chat --flush-cache --warmup-requests 2 --seed 1, the same chat and 20k scenarios as the first post.

Caveats

• One run per config, 32 prompts per DFlash2 card set and 10–26 turns per SPEED-Bench domain. Gaps under about 2 standard errors can’t be told from noise, and every Swift-vs-stock accept-length gap is under that. Chat at 1 user is 12 prompts; the 20k–120k rows are 4 prompts each (8 at 4 users).
• “Output tokens” and “tokens per answer” are completion_tokens, i.e. thinking plus answer; answers cut off at 4096 count as 4096. “Thinking tokens” is the saved reasoning text of each response, tokenized with the model’s tokenizer, counted only on prompts where both models stopped on their own (thinking_tokens.py in the repo).
• The DFlash2 model card doesn’t say how many prompts it ran or at what precision; its launch example serves BF16 Qwen/Qwen3.8-27B. Mine is NVFP4 on an RTX PRO 6000 at 400 W.
• The serving rows use fixed output lengths (ignore_eos), so they measure speed only and can’t show the token savings. The 20k prompt is SGLang’s random dataset (one ShareGPT prompt repeated to length).
• BFCL uses its own Qwen3 FC prompt, not the model’s chat template. Compare these scores with each other, not with the public leaderboard.

Raw data

• Scripts, configs and raw results: adcostanza/benchmarks. run.sh there reruns all four configs, about two and a half hours.
• The scripts on main keep changing. To rerun with them exactly as these runs had them, check out 50fa2ce.
• results/stock-nvfp4-quality/ and results/swift-nvfp4-quality/: GSM8K and BFCL, every response included.
• results/dflash8/ (stock) and results/swift-dflash8/ (Swift): accept-sets/, speed-bench/ (one line per request, generated text included), bench-serving.jsonl and server.log.
• thinking_tokens.py: the thinking-token tables, from the saved responses.

Test system

• GPU: NVIDIA RTX PRO 6000 Blackwell Workstation Edition, 96 GB, power limit 400 W (default 600 W). Nothing else on the GPU, one config at a time.
• PCIe: Gen 4 x16 (the card supports Gen 5; nvidia-smi reports Gen 4 as this system’s maximum).
• CPU: AMD Ryzen 9 7950X, 16 cores, 32 threads.
• RAM: 250 GiB DDR5.
• OS: Ubuntu 24.04.2, kernel 6.8.0-147.
• NVIDIA: driver 580.178.04, CUDA 13.0.
• Containers: rootless Docker 28.0.4, NVIDIA Container Toolkit 1.18.1, GPU through CDI.
• Full list: hardware.md at the commit these runs used.