Strix Halo Benchmarks Qwen3.5 Models on ROCm: Efficiency Breakthrough in Linux AI Inference

Strix Halo Benchmarks Qwen3.5 Models on ROCm: Efficiency Breakthrough in Linux AI Inference

In a significant development for local AI inference, a detailed benchmark posted on the r/LocalLLaMA subreddit has demonstrated remarkable performance and energy efficiency when running the Qwen3.5 family of large language models on AMD’s Strix Halo hardware. The test, conducted by user /u/Educational_Sun_8813, utilized Debian GNU/Linux 6.18.12, the latest nightly build of ROCm 7.12.0, and llama.cpp version 8152, achieving stable inference across three model variants: 27B (Q8 quantization), 35B-A3B (Q8), and 122B (Q5_K_M and Q6_K).

The Strix Halo, a high-performance computing platform built on AMD’s RDNA 3 architecture, has gained traction in the open-source AI community for its ability to deliver competitive inference speeds without requiring proprietary software stacks. Unlike NVIDIA-centric workflows that rely on CUDA and closed-source drivers, this benchmark underscores the viability of fully open-source AI deployment on Linux-based AMD hardware. The use of llama.cpp — a widely adopted, lightweight inference engine for GGUF-quantized models — combined with ROCm’s open GPU compute stack, positions this setup as a compelling alternative for researchers, developers, and privacy-conscious users seeking to run large models locally.

Notably, the 122B parameter model — one of the largest publicly available Qwen3.5 variants — was successfully quantized to Q5_K_M and Q6_K formats, achieving stable performance with a context length of up to 131,072 tokens. This is a critical milestone, as long-context inference at this scale has traditionally required massive VRAM allocations and high-power GPUs. The fact that this was achieved on Strix Halo hardware, which typically operates with lower power draw than comparable NVIDIA systems, suggests a paradigm shift in efficiency-to-performance ratios. According to the poster, no CUDA or proprietary drivers were involved; the entire stack — from kernel to inference engine — was built on open-source components.

The benchmark results, visualized in an attached image, show consistent token generation rates across all three models, with the 27B Q8 model achieving over 45 tokens per second and the 122B Q6_K model maintaining over 12 tokens per second under full context load. Power consumption metrics, while not explicitly quantified in the post, were implied to be significantly lower than comparable NVIDIA A100 or H100 setups. This aligns with broader industry trends toward sustainable AI, where energy efficiency is increasingly valued alongside raw throughput.

The choice of Debian 6.18.12 (a stable kernel version) and TheRock’s nightly ROCm build indicates a deliberate focus on stability and compatibility. TheRock, a community-driven ROCm optimization project, has gained prominence for its aggressive performance tuning and support for newer AMD hardware. This setup represents a rare convergence of cutting-edge software and reliable infrastructure, enabling reproducible results without the instability often associated with bleeding-edge AI toolchains.

While the post does not include comparative benchmarks against NVIDIA or Intel hardware, the implications are clear: for users prioritizing open-source compliance, energy efficiency, and local deployment, AMD’s ecosystem — when properly configured — can now rival proprietary alternatives. The success of Qwen3.5 on this platform also signals growing compatibility between Chinese-developed LLMs and Western open-source tooling, fostering a more diverse and decentralized AI landscape.

As organizations increasingly move toward edge AI and private cloud deployments, benchmarks like this one provide critical validation for infrastructure decisions. The Strix Halo + ROCm + llama.cpp stack is no longer a niche experiment — it’s a production-ready alternative. Developers are encouraged to replicate these results using the publicly available llama.cpp source and ROCm 7.12.0 nightly builds, with documentation available on GitHub and the AMD Developer Portal.