Qwen3-Coder-Next Achieves Record Speeds with Latest llama.cpp Update

Qwen3-Coder-Next Achieves Record Speeds with Latest llama.cpp Update

A significant breakthrough in local AI inference has emerged as developers report unprecedented speed gains for the Qwen3-Coder-Next model, powered by the latest updates to the llama.cpp framework. According to user benchmarks shared on Reddit’s r/LocalLLaMA community, the model now achieves over 130 tokens per second on a single NVIDIA RTX PRO 6000 Blackwell workstation GPU — a more than 50% improvement over previous performance levels. This leap, enabled by a recently merged pull request in the llama.cpp repository, signals a major advancement in the accessibility and efficiency of large language models for on-device coding assistants.

The Qwen3-Coder-Next model, developed by Alibaba’s Tongyi Lab and introduced as an open-weight architecture optimized for agentic coding tasks, is built atop the Qwen3-Next-80B-A3B-Base foundation with hybrid attention and Mixture-of-Experts (MoE) capabilities. As noted on the official Qwen research page, the model was specifically trained at scale using executable task synthesis, environment interaction, and reinforcement learning to enhance its utility in autonomous coding agents and local development environments. Its 80-billion-parameter size, quantized to Q8_0 precision, makes it a formidable candidate for high-performance local deployment — but until now, its full potential was bottlenecked by inference efficiency.

The key to this breakthrough lies in PR #19375 to llama.cpp, an open-source C/C++ library for running GGUF-quantized LLMs on consumer hardware. The update introduced critical optimizations in CUDA kernel scheduling, memory mapping, and multi-GPU tensor partitioning. Benchmarks conducted by user StardockEngineer reveal that before the update, the Qwen3-Coder-Next model delivered approximately 87 tokens per second on dual-GPU setups and around 80 on single GPUs. After applying the new build (commit 079feab9e), single-GPU performance on the RTX PRO 6000 Blackwell surged to 132 tokens per second — a 61% increase — while dual-GPU throughput climbed to 119 tokens per second, up from 87.

Notably, the performance gains were most pronounced in prompt processing (pp500), where throughput jumped from 2,470 to 3,563 tokens per second on the RTX PRO 6000 — nearly a 44% improvement. This suggests that the update particularly benefits the prefill phase, critical for reducing latency in interactive coding tools. The improvements were consistent across different context lengths (500 and 1,000 tokens), indicating robustness in memory management under load.

These enhancements are particularly timely as the AI developer community increasingly shifts toward local, privacy-preserving LLM deployments. With models like Qwen3-Coder-Next now capable of near-real-time code generation on high-end workstations, developers can bypass cloud API costs and latency while maintaining enterprise-grade security. The model’s compatibility with Ollama — which lists Qwen3 as one of its most downloaded models, with over 19 million runs — further underscores its growing adoption in developer toolchains.

According to Alibaba’s Qwen research team, future iterations of Qwen3-Coder-Next will integrate tighter tooling for code execution environments and agent-based task automation, making this performance leap not just a technical milestone but a strategic enabler for next-generation AI-assisted development workflows. The llama.cpp team has not yet issued an official statement on the optimization, but the merge of PR #19375 into the main branch indicates broad community validation.

For developers, the message is clear: updating llama.cpp to the latest commit is no longer optional — it’s essential for unlocking the full potential of high-parameter models like Qwen3-Coder-Next. As local AI continues to outpace cloud-based alternatives in speed, cost, and control, this update may well become a defining moment in the democratization of enterprise-grade code generation.