Minimax 2.5 Delivers Exceptional AI Quality on AMD Hardware—But Performance Bottlenecks Persist

Minimax 2.5 Delivers Exceptional AI Quality on AMD Hardware—But Performance Bottlenecks Persist

On the cutting edge of open-source large language model deployment, an anonymous Reddit user has documented an in-depth benchmark of Minimax 2.5—a newly released 230-billion-parameter model—running on AMD’s Strix Halo graphics hardware. The experiment, conducted on a headless Fedora 43 system with ROCm nightlies and a 6.18.9 kernel, reveals that while the model produces remarkably coherent, nuanced, and context-aware responses, its inference speed remains prohibitively slow for real-time applications.

According to the user’s detailed logs, the MiniMax-M2.5-Q3_K_M GGUF quantized model (101.76 GiB) achieves prompt processing speeds of 43.67 tokens per second (t/s) and text generation rates as low as 3.34 t/s after extended context usage—figures that fall far short of commercial cloud-based alternatives. Despite extensive environment tuning—including HIP_VISIBLE_DEVICES, GGML_HIP_UMA, ROCBLAS_USE_HIPBLASLT, and HSA_OVERRIDE_GFX_VERSION flags—the performance plateaued, suggesting hardware or software-level bottlenecks in the ROCm stack.

Notably, the model’s quality far exceeds expectations for a quantized, locally run LLM. The user reports that even with 40,000+ token contexts and multiple tool invocations, Minimax 2.5 maintains logical consistency, precise reasoning, and stylistic fluency—hallmarks typically associated with proprietary models like GPT-4 or Claude 3. This has sparked renewed interest among local AI enthusiasts who prioritize privacy and model control over speed.

Performance comparisons between ROCm and Vulkan backends revealed subtle but telling differences. While ROCm delivered faster prompt processing (200+ t/s), Vulkan improved text generation throughput to 33 t/s, a 15% gain over ROCm’s 27 t/s. However, Vulkan’s prompt evaluation lagged by 30%, indicating a fundamental imbalance in how AMD’s drivers handle attention computation versus token sampling. The user noted that switching to the larger Q3_K_XL quantization did not improve speed, confirming that memory bandwidth—not precision—is the limiting factor.

Technical analysis suggests that the root cause lies in the combination of AMD’s current ROCm support for large GGUF models and the Strix Halo’s memory architecture. Despite the GPU’s theoretical 1.5 TB/s memory bandwidth, the lack of unified memory optimization in llama.cpp’s HIP backend results in frequent host-device transfers, especially under long-context workloads. The use of GGML_CUDA_ENABLE_UNIFIED_MEMORY—intended for NVIDIA systems—is a misconfiguration that may even introduce overhead, as noted by multiple developers in the LocalLLaMA community.

While Minimax’s official website (minimax.si) appears unrelated—focused on Slovenian accounting software—the model’s open release on Hugging Face under the Unsloth project signals a growing trend: community-driven development of high-performance, commercially viable models outside traditional corporate labs. The Minimax 2.5 model, though not officially endorsed by the company of the same name, is being treated as a landmark achievement in quantization efficiency.

For now, users are advised to consider trade-offs: Minimax 2.5 is ideal for batch processing, research, or offline analysis where latency is acceptable. For interactive use, smaller models like Phi-3 or Mistral 7B remain more practical. However, the fact that a 230B-class model can even run locally on consumer-grade AMD hardware—albeit slowly—is a testament to the rapid progress in model compression and open-source tooling. Developers are already exploring FlashAttention-3 optimizations and dynamic KV caching for ROCm, with early benchmarks promising 2x speedups by Q3 2024.

As the open LLM ecosystem matures, Minimax 2.5’s performance challenges may soon be resolved—but for now, its greatest contribution may be proving that world-class AI doesn’t require cloud subscriptions. It just requires patience, a powerful GPU, and a community willing to tinker.