On one RTX PRO 6000 with SGLang and DFlash2 (8 draft tokens), ukisai’s Swift 1.5 NVFP4 and stock Qwen3.8-27B NVFP4 are close on the two quality checks I ran: GSM8K (human-authored answers, graded by sgl-eval against the final answer) 1,280 vs 1,277 of 1,319 correct (97.04% vs 96.82%) and a tie at 168 of 205 on a subset of BFCL, a function-calling benchmark graded by its own checkers, not human judges. That is a three-case gap and a tie on two limited checks, not a test of equivalence. Median time per output token is about the same. At reasoning_effort=medium Swift spends 14–19% fewer thinking tokens on GSM8K and SPEED-Bench, so mean output per response is 8–10% lower and the SPEED-Bench run finished about 10% sooner. These are these two checkpoints on this stack, not a statement about the fine-tune or the NVFP4 format in general.

Results

Stock RadixArk/Qwen3.8-27B-NVFP4 vs Swift ukisai/Swift-1.5-Qwen3.8-27b-NVFP4, both with DFlash2 8. GSM8K and the generation benchmarks use reasoning_effort=medium; BFCL uses its handler’s default effort and temperature 0.001.

¹ The 125 of 142 turns where both models stopped on their own; the output-token row counts all 142.

Accept length on the five sets of the DFlash2 model card, with its protocol (xhigh, temperature 1.0, top-p 0.95, top-k 20, max 4096 new tokens), 32 prompts per set, 1 user. “8 draft tokens” is SGLang’s --speculative-num-draft-tokens 8, DFlash2’s block size 8 (7 drafted per step):

Mean thinking tokens per answer on the same sets, on the prompts where both models stopped on their own. At medium that’s 30–32 of 32 prompts; at xhigh 31 (MATH-500), 26 (MBPP), 24 (MT-Bench) and 21 (HumanEval), with GSM8K at 32.

Mean tokens per answer, thinking plus answer, all 32 prompts (answers cut off at 4096 count as 4096):

Takeaways

• About the same speed per token; the time saved goes with the shorter answers. Median time per output token on SPEED-Bench is 4.49 vs 4.47 ms, and aggregate output is about 203 tok/s for both. Swift produced 10.1% fewer recorded tokens (185,736 vs 206,551) and finished the 110 conversations 10.2% sooner (913 s vs 1,016 s). The 4,096-token cap cut off 15 stock turns and 13 Swift turns and counts them as 4,096, so the uncapped difference in length is unknown. The 32 follow-up turns also continue from each model’s own earlier answers, so the two runs don’t see identical input.
• Chat at 1 user is the exception: 4% slower, accept length 3.27 vs 3.44. One run on 12 prompts, so I can’t say whether it’s real. The other serving rows in the repo (4 users, 20k, 80k, 120k) are within 1%; 40k at 1 user is +4% for Swift (4 prompts).
• In these two exports, the saving is mostly thinking. At medium Swift thinks 5–38% less on the DFlash2 card’s sets and 14–19% less on GSM8K and SPEED-Bench; at xhigh, 17–47% less. Total minus thinking (an estimate: completion_tokens against re-tokenized reasoning) is about the same on full GSM8K (134 vs 136) and at xhigh on GSM8K and MATH-500, but it moves elsewhere: MT-Bench at xhigh −29%, HumanEval at xhigh +26%, and MBPP at medium saves 82 tokens of which only 36 are thinking. The two NVFP4 exports were converted separately (different ModelOpt versions, and only stock’s checkpoint carries FP8 KV scales), so this measures the deployed checkpoints, not the fine-tune in isolation. Swift’s model card claims 24.8% fewer thinking tokens at medium and 41.9% at xhigh (card @ b4c84d42, checked 2026-10-30). That is BF16 on vLLM; the xhigh figure is its GPQA-Diamond row and the table doesn’t name a benchmark for medium.
• On three of five sets xhigh costs Swift no more than a little over medium. Whole responses at xhigh vs medium: GSM8K 296 vs 368 tokens, MATH-500 864 vs 833, MBPP 899 vs 895. MT-Bench (1,163 vs 1,002) and HumanEval (1,454 vs 1,131) cost 16–29% more. Stock pays more for xhigh on four of five (HumanEval 2,042 vs 1,196). I didn’t measure accuracy at xhigh, and the 4,096 cap hides the longest answers.
• The stock drafter works with the Swift export. At xhigh, mean accept length is higher for Swift on all five DFlash2 card sets (gaps of 0.04–0.28 per set). Pairing the 32 prompts, the per-prompt gaps are about 2–3.4 standard errors on GSM8K, HumanEval, MBPP and MT-Bench and 0.6 on MATH-500; that is an exploratory check on one run, not proof of a gain, and it doesn’t isolate the fine-tune from the separate exports. I didn’t test a drafter tuned on Swift.
• On the DFlash2 card’s sets and protocol, stock NVFP4 lands within 0.18 of the card’s accept length on every set, and output tok/s at 1 user is up to about 7% below the H200’s (HumanEval 199.5 vs 214.6). The 3.4 in the spec-decode post is ShareGPT chat at medium. Prompts, sampling, effort, max length, hardware and precision all differ from the card’s runs, and I didn’t isolate which one explains the ShareGPT gap.
• BFCL ties at 168 of 205. Stock NVFP4 scored 174 and 176 on this config in the earlier posts, so the same config already moves by 8 cases between runs. By category, Swift gains multi_turn_base (14 → 16 of 20) and loses irrelevance (15 → 11); with 20 cases each, I can’t read anything into either.
• Quality here is two benchmarks. GSM8K is near ceiling (12 questions only Swift got right, 9 only stock) and BFCL is a 205-case subset, so “no clear difference” is all this shows, not equivalence. I did not audit Swift’s training data or RL environments for overlap with these prompts. Swift’s model card shows 82.22% vs 84.14% for the base at medium, on a benchmark it doesn’t name. I ran nothing harder.

Setup

Serving stack:

• Image: lmsysorg/sglang:nightly-dev-cu13-20261010-961bcf48.
• Checkpoints: RadixArk/Qwen3.8-27B-NVFP4 @ 319f741c, ukisai/Swift-1.5-Qwen3.8-27b-NVFP4 @ 25482027 (ukisai’s own NVFP4 build).
• Drafter: z-lab/Qwen3.8-27B-DFlash2 @ 50307d4c, 8 draft tokens, on every run.
• Flags: the launch command from the NVFP4 vs FP8 post (scripts/args/qwen38-27b.sh) with --mem-fraction-static 0.41. Both get a 236,864-token KV pool and use about 42.8 GiB of GPU memory after load (43,849 and 43,837 MiB).

Benchmarks, at reasoning_effort=medium unless noted (BFCL uses its handler’s default effort and temperature 0.001):

• GSM8K: the full test split, sgl-eval’s prompt and grader (last \boxed{}), the model’s own sampling, max 32,768 tokens, 32 requests in flight. No answer was cut off.
• BFCL: bfcl-eval 2026.3.23, the Qwen/Qwen3-32B-FC handler, the same fixed 205-case subset as the earlier posts (case ids).
• DFlash2 card sets: GSM8K, MATH-500, HumanEval, MBPP, MT-Bench (first turn), prompt formats and shuffle from z-lab/dflash at 07ebd93, 32 prompts per set, 1 user. Once with the DFlash2 card’s protocol (xhigh), once at medium with the same sampling.
• SPEED-Bench: the qualitative split, 10 prompts per domain (110 prompts, 142 turns; 32 are follow-up turns that include the model’s own earlier answer), max 4096 tokens, 1 user, server-default sampling.
• Serving: python3 -m sglang.benchmark.serving --backend sglang-oai-chat --flush-cache --warmup-requests 2 --seed 1, the same chat and 20k scenarios as the first post.

Caveats

• One run per config, 32 prompts per DFlash2 card set and 10–26 turns per SPEED-Bench domain. Gaps under about 2 standard errors of the gap can’t be told from noise; the standard errors in the accept-length tables are per model, not for the paired gap. Chat at 1 user is 12 prompts; the 20k–120k rows are 4 prompts each (8 at 4 users).
• “Output tokens” and “tokens per answer” are completion_tokens, i.e. thinking plus answer; answers cut off at 4096 count as 4096. “Thinking tokens” is the saved reasoning text of each response, tokenized with the model’s tokenizer, counted only on prompts where both models stopped on their own (thinking_tokens.py in the repo).
• The DFlash2 model card doesn’t say how many prompts it ran or at what precision; its launch example serves BF16 Qwen/Qwen3.8-27B. Mine is NVFP4 on an RTX PRO 6000 at 400 W.
• The serving rows use fixed output lengths (ignore_eos), so they measure speed only and can’t show the token savings. The 20k prompt is SGLang’s random dataset (one ShareGPT prompt repeated to length).
• BFCL uses its own Qwen3 FC prompt, not the model’s chat template. Compare these scores with each other, not with the public leaderboard.

Raw data

• Scripts, configs and raw results: adcostanza/benchmarks. run.sh there reruns all four configs, about two and a half hours.
• The scripts on main keep changing. For a pinned public rerun, check out 50fa2ce. The original runs’ suite.json files mark the harness dirty and name earlier internal commits, so this is not a byte-for-byte snapshot of them.
• results/stock-nvfp4-quality/ and results/swift-nvfp4-quality/: GSM8K and BFCL, every response included.
• results/dflash8/ (stock) and results/swift-dflash8/ (Swift): accept-sets/, speed-bench/ (one line per request, generated text included), bench-serving.jsonl and server.log.
• thinking_tokens.py: the thinking-token tables, from the saved responses.

Test system

• GPU: NVIDIA RTX PRO 6000 Blackwell Workstation Edition, 96 GB, power limit 400 W (default 600 W). Nothing else on the GPU, one config at a time.
• PCIe: Gen 4 x16 (the card supports Gen 5; nvidia-smi reports Gen 4 as this system’s maximum).
• CPU: AMD Ryzen 9 7950X, 16 cores, 32 threads.
• RAM: 250 GiB DDR5.
• OS: Ubuntu 24.04.2, kernel 6.8.0-147.
• NVIDIA: driver 580.178.04, CUDA 13.0.
• Containers: rootless Docker 28.0.4, NVIDIA Container Toolkit 1.18.1, GPU through CDI.
• Full list: hardware.md at the pinned public rerun commit.