Qwen3.8-27B-FP8 Beats Opus 4.6 Max on Several Coding Benchmarks

My notes

Summary

Qwen released Qwen3.8-27B-FP8, an FP8-quantized, deployment-friendly dense vision-language model with near-identical performance to the full-precision original. It is positioned as Qwen’s strongest model yet for coding, agentic, and long-horizon tasks, with benchmark scores that beat Anthropic’s Opus 4.6 Max on several coding and agentic tests despite being a much smaller 27B model.

Key Insight

• On SWE-bench Pro, Qwen3.8-27B scores 61.7 against Opus 4.6 Max’s 53.4; on QwenSWEBench 79.0 against 63.8; on Terminal-Bench 2.1 73.0 against 78.2, a close second. A 27B open model is now competitive with or beating a frontier closed model on several agentic-coding benchmarks, a meaningful shift for self-hosted coding agents.
• Vision-agent benchmarks show large jumps: computer use (OSWorld-Verified) 84.3 against Opus at 72.7; browser use (WebArena-Verified) 64.8 against Qwen3.6’s 48.8; application recreation 47.1 against Qwen3.6’s 29.8. The gains are strongest specifically in agentic multimodal tasks such as screen and browser control, not just static benchmarks.
• Native context is 262,144 tokens, extensible to 1,000,000 via YaRN RoPE scaling. Static YaRN degrades short-text performance, so Qwen recommends enabling it only when long context is actually needed, and tuning the factor parameter to the target context length (factor 2.0 for roughly 524k tokens) rather than always maxing it out.
• The reasoning_effort parameter (xhigh, medium, low) trades reasoning depth against cost, but Qwen explicitly warns that lower effort can increase total latency and token use in multi-turn agentic tasks because it produces more failures and retries. Do not default to “low” for agent workloads assuming it saves cost.
• preserve_thinking (on by default) keeps the model’s reasoning trace across a whole multi-turn conversation rather than just the latest turn, improving decision consistency in agent scenarios and KV-cache efficiency at the cost of context usage.
• FP8 quantization here is fine-grained with a block size of 128, reported as near-identical to the unquantized model. A useful reference point for anyone deciding whether to run FP8 or full precision for cost and throughput reasons.