UG Student Builds Dhi-5B AI Model for $1,200, Challenging Tech Giants

UG Student Builds Dhi-5B AI Model for $1,200, Challenging Tech Giants

In a landmark achievement that could redefine the economics of artificial intelligence, a University of Ghana undergraduate student, known online as gradNorm, has successfully trained Dhi-5B — a 5-billion-parameter multimodal language model — with a total training cost of just ₹1.1 lakh ($1,200). The model, developed entirely from scratch using a custom-built codebase, outperforms commercially developed models that required orders of magnitude more computational resources and funding.

According to the student’s detailed post on Reddit’s r/LocalLLaMA community, Dhi-5B was trained in five distinct phases: Pre-Training, Context-Length Extension, Mid-Training, Supervised Fine-Tuning, and Vision Extension. The final model, dubbed "The Dhi-5B," integrates both text and image understanding capabilities, making it a true multimodal AI. The base variant, Dhi-5B-Base, with 4 billion parameters, was trained on 40 billion natural language tokens primarily drawn from the FineWeb-Edu dataset — a high-quality, education-focused corpus.

Technical specifications reveal an advanced architecture: 32 transformer layers, 3072-dimensional hidden states, SwiGLU MLPs, FlashAttention-3 optimized attention mechanisms, and a 64k vocabulary size. The model was trained with a batch size of 2 million tokens, leveraging the novel Muon optimizer for matrix layers and AdamW for the remainder — a hybrid approach that enhances convergence efficiency. Notably, the context length was extended from 4k to 16k tokens through a specialized fine-tuning phase, enabling the model to process long-form documents and complex reasoning tasks.

The most astonishing aspect of this project is its cost-efficiency. While leading AI labs such as OpenAI, Google, and Meta have spent hundreds of millions of dollars training models of similar scale, gradNorm achieved comparable or superior performance on benchmarks using consumer-grade hardware and strategic resource optimization. The student reportedly utilized cloud credits, donated GPU time from academic institutions, and optimized data pipeline efficiency to minimize expenses. This stands in stark contrast to the industry norm, where training a single large model can consume over $10 million in compute costs.

While the student has not disclosed their full identity or institutional affiliation beyond being an undergraduate, their work has drawn immediate attention from the global AI research community. Experts in open-source AI have praised the project as a "proof of concept for democratized AI development." The model’s performance on standardized benchmarks such as MMLU, GSM8K, and Vision-Language Alignment tasks rivals models like Llama 3-8B and Mistral-7B, which were trained on significantly larger budgets.

Importantly, the student emphasized that the project was not funded by any corporate entity or government grant. Instead, it was financed through personal savings, part-time work, and community support — a testament to individual ingenuity in the face of institutional barriers. The student’s journey echoes broader conversations about equity in AI, where access to capital often determines who can innovate. As global education systems grapple with rising tuition and limited research funding, Dhi-5B offers a compelling case study in resourcefulness.

The next phases — Dhi-5B-Instruct and the full multimodal Dhi-5B — are slated for release in the coming weeks. All models will be open-sourced under a permissive license, enabling researchers, educators, and developers worldwide to use, modify, and build upon the architecture. This move could accelerate innovation in low-resource regions and inspire a new generation of AI developers outside Silicon Valley.

As the AI landscape becomes increasingly dominated by proprietary models and centralized control, gradNorm’s achievement serves as a powerful reminder that groundbreaking innovation does not require vast capital — only vision, discipline, and technical mastery. The world may soon see more "Dhi-5B" projects emerge — not from corporate labs, but from university dorm rooms and remote research hubs, proving that the next AI revolution may be quietly unfolding in the most unexpected places.