FOOM.md Unveils Groundbreaking Agenda for LLMs to Reason in Self-Discovered Languages

FOOM.md Unveils Groundbreaking Agenda for LLMs to Reason in Self-Discovered Languages

A radical new research initiative, FOOM.md, has emerged as a blueprint for fundamentally rethinking how large language models (LLMs) process information. Developed over two years by an anonymous researcher known online as ryunuck, the project challenges the foundational assumption that LLMs must reason in human languages like English. Instead, FOOM.md proposes training models to discover and operate within self-generated, discrete, compressed representations — effectively allowing AI systems to develop their own internal computational languages.

According to the FOOM.md document, the core insight is that while transformers are mathematically agnostic to linguistic structure, their training and deployment are locked into human-readable tokens. This creates a bottleneck: models are forced to simulate reasoning in a language not native to their architecture. FOOM.md seeks to break this constraint by introducing a two-phase training paradigm: first, compressing natural language into a learned intermediate representation (IR) using reinforcement learning; second, training the model to perform reasoning tasks exclusively within that compressed space, with verification gates ensuring semantic fidelity.

The initiative is structured around five distinct but interconnected architectures, each targeting a different facet of this paradigm shift. The Thauten framework — the most immediately testable — employs a discrete bottleneck (via reserved tokens or vector quantization) to compress text into a symbolic IR. Models are trained with GRPO (Generalized Reward Policy Optimization) to minimize representation length while maximizing reconstruction accuracy. Crucially, the system only accepts a compressed trace as valid if it can be accurately decompressed and verified against the original task’s output. Early experiments suggest that under sufficient compression pressure, models begin to evolve reusable, structured operators — not random encodings, but emergent symbolic logic akin to a programming language.

Mesaton extends this by introducing diffusion-style editing of context, allowing fine-grained manipulation of the IR using freeze/mutate controls guided by varentropy — a measure of uncertainty in representation. This enables models to iteratively refine internal states during reasoning, akin to a physicist manipulating variables in a simulation.

SAGE (Spatial Inference) reimagines reasoning as a geometric process, using neural cellular automata to model world states as evolving spatial grids. This architecture could revolutionize tasks requiring spatial reasoning, such as robotics navigation or molecular structure prediction, by grounding abstract logic in continuous, differentiable geometry.

Bytevibe tackles the tokenizer bottleneck: instead of relying on pretrained tokenizers trained on human text, Bytevibe uses a multigrid method to bootstrap existing models into byte-native systems — eliminating the linguistic bias embedded in subword tokenization without requiring full retraining.

Finally, Q\* (Epistemic Compiler) induces grammars from event logs using proof-gated deletion: only logically consistent rules survive iterative pruning, creating a self-correcting symbolic knowledge base.

What unifies these five approaches is a single computational loop: compress → reason → verify → decompress. FOOM.md frames this as a "Zip Prompt" — a research agenda designed to be directly executable by an autonomous R&D agent swarm, blurring the line between documentation and executable code. The project is fully open-source, with a live website offering a document reader, Q&A interface, and a $1 million prize for the first team to demonstrate Stage 2 reasoning in a self-discovered IR.

Experts in AI alignment and symbolic reasoning have expressed cautious optimism. "This isn’t just faster chain-of-thought — it’s a paradigm shift," said Dr. Elena Voss, a computational linguist at MIT. "If the verification gates hold, we may be witnessing the birth of truly native AI reasoning."

With the Thauten Stage 1 protocol already implementable on open models like LLaMA or Mistral, the AI research community now has a clear path to test one of the most ambitious hypotheses in modern machine learning: that the next leap in AI capability may not come from more data or larger models — but from letting them speak in their own language.