Qwen3.5-397B-A17B Benchmarked Locally: Performance Insights from Community Testing

Qwen3.5-397B-A17B Benchmarked Locally: Performance Insights from Community Testing

In a rare glimpse into the real-world performance of one of the most ambitious open-weight language models to date, a community member has shared detailed benchmark results for a model referred to as Qwen3.5-397B-A17B. Running on a dual NVIDIA RTX 3090 Ti setup with an AMD EPYC 7402P processor and 256GB of DDR4 memory, the test—conducted using the llama-bench framework—revealed that while the model successfully loaded and generated responses, inference speeds were severely constrained, with a single run taking approximately one hour to complete. The tester, who identified themselves only as /u/ubrtnk on Reddit’s r/LocalLLaMA, ran the Q4_K_M quantized version of the model with NGC at 10 and CPU-MoE at 51, activating 61 total layers across the architecture.

While the model’s name suggests a direct lineage to Alibaba’s officially released Qwen3 series, there is no public documentation from Qwen.ai confirming the existence of a Qwen3.5-397B-A17B variant. According to Alibaba’s official Qwen3 announcement, the flagship model is Qwen3-235B-A22B, a Mixture-of-Experts (MoE) architecture with 235 billion total parameters and 22 billion activated parameters per inference. The Qwen3-30B-A3B, a smaller MoE variant, was highlighted for its efficiency, outperforming larger dense models like QwQ-32B despite using only 3 billion activated parameters. The absence of official references to a 397B parameter model raises questions about whether this is an experimental build, a community amalgamation, or a mislabeled variant of an internal research prototype.

Unsloth.ai, a leading provider of optimized inference tools for large language models, offers a comprehensive guide for running Qwen3.5 models locally, emphasizing quantization techniques and memory-efficient attention mechanisms. Their documentation recommends using 4-bit quantization (such as Q4_K_M) and leveraging GPU offloading for MoE layers to reduce VRAM pressure. The tester’s configuration aligns with these best practices, yet the performance bottleneck suggests that even optimized quantization cannot fully compensate for the computational overhead of scaling MoE layers beyond the limits of current consumer hardware.

The disparity between official benchmarks and this community test underscores a critical tension in the open-weight AI ecosystem: while companies like Alibaba release models with impressive theoretical performance, real-world deployment on non-cloud infrastructure remains a formidable challenge. The Qwen3.5-397B-A17B test, though unofficial, provides valuable empirical data for developers seeking to understand the practical trade-offs between model scale and inference latency. The tester’s willingness to run the model during off-peak electricity hours (after 7 PM CST) also reflects a growing trend among hobbyists and researchers who optimize for cost-efficiency over speed, leveraging cheap power to push hardware to its limits.

Experts in AI infrastructure note that models exceeding 200 billion total parameters, particularly MoE variants, require specialized hardware such as NVIDIA H100 clusters with NVLink interconnects and high-bandwidth memory to achieve acceptable latency. The dual 3090 Ti setup, while formidable for its generation, lacks the memory bandwidth and unified memory architecture of newer data center GPUs. Recommendations from the Reddit thread suggest increasing NGC to 12–14 and adjusting CPU-MoE to 64, though this may exacerbate memory fragmentation without additional optimizations like tensor parallelism or kernel fusion.

As the open-source LLM community continues to push boundaries, unofficial benchmarks like this one serve as crucial feedback loops for both developers and vendors. While Alibaba’s Qwen3 series is designed for scalability and efficiency, this community-driven test reveals the untapped potential—and the current limitations—of deploying next-generation MoE architectures on accessible hardware. Whether Qwen3.5-397B-A17B is a real model or a speculative construct, its performance profile offers a sobering reminder: bigger is not always better when it comes to local AI inference.