Qwen3.8-27B is the first local model I’ve actually been productive with. I’ve been using it for local agentic work on personal projects for roughly the last 2 months and I’ve been incredibly impressed with it. It does the tedious throwaway stuff for me.

The overthinking on stock Qwen was a bit frustrating at first. I’ve landed on a setup I’m very happy with: the Swift 1.5 variant as an NVFP4 quant, loaded into SGLang pretty much all the time. It cut the overthinking down a lot. It runs on an RTX PRO 6000 but I’m not trying to fill its 96 GB of VRAM. Qwen keeps roughly 43 GiB occupied and the rest stays free for other GPU workloads like other ML models and GenAI with ComfyUI.

What I use it for

Agentic work in the terminal: renaming a bunch of files in some weird way, moving files across drives, writing random Python scripts, dealing with HomeAssistant YAML config hell. Along the way it writes scripts and even skills that I keep and reuse. It does a great job for probably about 90% of the types of tasks I do on my homelab and it’s all local!

I can pass it images and it extracts content and understands them decently well. I primarily use it from my phone by SSHing into my computer with persistent tmux sessions that I nuke once they get unwieldy (similar to when you have 100 tabs open). It runs in a Docker sandbox. I can run many instances in different folders and decide per folder whether it gets the internet (typically not) or read-only access to other folders.

The harness is oh-my-pi (omp), the batteries-included version of the minimalist pi. I prefer it over Claude Code (my previous daily driver) and Codex because it’s so freaking customizable. It’s definitely not perfect. It works great for both local and frontier models so I don’t context switch between Qwen for most tasks and Opus 5.5 for very complex or nuanced work. It’s terminal-driven, which suits me since I mostly write code and run agentic workflows. For computer use it doesn’t compare to Codex but I rarely need that.

I’ve tried so many things in isolation (llama.cpp and vLLM, then different drafters) that I’ve forgotten most of them. Benchmarking in public makes me remember and keeps me honest about trade-offs.

The setup

Each piece has its own benchmark post with the tables and caveats. The short version:

• Model: Swift 1.5, ukisai’s fine-tune of Qwen3.8-27B that thinks less. No clear quality gap vs stock and 14–19% fewer thinking tokens.
• Quant: NVFP4. 32–37% more tokens/s than Qwen’s FP8.
• Drafter: DFlash2 with 8 draft tokens. Fastest of five drafter setups from chat to 120k.
• Not Flash-Next: the obvious alternative on a 96 GB card. Twice the memory and no faster until prompts pass 40k.
• Effort: medium. It’s what I use day to day and what most of the benchmarks ran at.

Swift 1.5 is the change I noticed most. Day to day it feels like a much bigger difference than the benchmarks show. It doesn’t get stuck in weird reasoning loops nearly as often or overthink everything with “but wait…”.

Decisions on the same model

The part I’m most curious about is using the chat model for decisions too: picking one of a set of labeled options with thinking off. SGLang does that from the weights already loaded so there’s no second model in VRAM. I haven’t used it much beyond the decisions benchmarks yet.

My launch command

For anyone who wants to copy it (swap in your own model paths; exact image and revisions are in each benchmark post):

python3 -m sglang.launch_server --model-path "$MODEL" \
  --mem-fraction-static 0.41 --max-running-requests 4 --context-length 153600 \
  --dtype auto --attention-backend flashinfer \
  --reasoning-parser qwen3 --tool-call-parser qwen3_coder \
  --speculative-algorithm DFLASH --speculative-draft-model-path "$DRAFTER" \
  --speculative-num-draft-tokens 8 \
  --mamba-full-memory-ratio 0.9 --mamba-ssm-dtype bfloat16 \
  --mamba-radix-cache-strategy extra_buffer_lazy \
  --chunked-prefill-size 2048 --page-size 64 \
  --enable-hierarchical-cache --hicache-size 64 --hicache-write-policy write_through \
  --kv-cache-dtype fp8_e4m3 --schedule-policy lpm --enable-session-radix-cache \
  --sleep-on-idle --trust-remote-code

The hierarchical cache takes about 76 GB of system RAM (I have 250 GiB). It’s only there so stale sessions load back into the KV cache super fast. Drop the --enable-hierarchical-cache and --hicache-* flags if you’re short on RAM.

Smaller cards

This isn’t a guide for running Qwen on a potato or on a 4090 or 5090. NVFP4 needs a Blackwell GPU to run natively and DFlash2’s memory cost matters a lot more on a 24 GB card. I haven’t tried any of it on a smaller card.

I’m waiting on Qwen 4 and I’m excited to see how it matches up to Qwen3.8-27B. Small changes might get an update here but big ones will get their own benchmarks and a new post.