AI Consistency Crisis: Car Wash Test Reveals Major Flaws in LLM Reasoning

AI Consistency Crisis: Car Wash Test Reveals Major Flaws in LLM Reasoning

A surprising and revealing experiment has exposed deep inconsistencies in the reasoning capabilities of leading artificial intelligence models. Conducted by a researcher under the username facethef on Reddit’s r/LocalLLaMA community, the "Car Wash Test" posed a deceptively simple question to 53 AI models: "I want to wash my car. The car wash is 50 meters away. Should I walk or drive?" The correct answer — obvious to any human — is that one must drive, since the vehicle itself must be transported to the wash. Yet, as reported in the original post, only five of the 53 models demonstrated consistent reliability across 10 repeated trials.

The test, which involved 530 total model queries with no system prompts, cache, or memory retention between runs, was designed to move beyond the misleading single-response evaluations common in AI benchmarking. What emerged was not just a failure rate, but a pattern of erratic reasoning. Models that initially appeared competent on a single run — including GLM-4.7 and Kimi K2.5 — later failed in multiple iterations. Conversely, some models that initially failed showed intermittent success, suggesting that reasoning is not a stable feature but a probabilistic outcome influenced by internal stochastic processes.

Among the top performers, Google’s Gemini 3 series and Flash Lite achieved perfect 10/10 scores, as did xAI’s Grok-4 and Reasoning models. Anthropic’s Claude Opus 4.6 was the lone standout in its family, while OpenAI’s GPT-5 achieved a 7/10 pass rate — the only OpenAI model to break double digits. Zhipu’s GLM-5 scored 8/10, and GLM-4.7 improved significantly from its initial failure to 6/10, indicating that some open-weight models may possess latent reasoning capacity that is not reliably activated.

Conversely, Meta’s Llama family, Mistral models, DeepSeek, Moonshot, and MiniMax scored 0/10 across all trials — a striking result given their prominence in open-source AI development. Even Sonar, which initially appeared correct in a single run, now consistently answers "walk," substituting lengthy essays on energy chains and food production in place of logical reasoning. This suggests that some models are not reasoning at all, but generating plausible-sounding text that happens to align with the prompt — a phenomenon known as "stochastic parroting."

The implications extend far beyond car washes. If AI systems cannot reliably deduce that a physical object must be moved to be cleaned, how can they be trusted in critical applications like autonomous driving, medical diagnostics, or financial decision-making? The test underscores a growing concern in AI ethics and engineering: accuracy in one-off responses is not sufficient. Reliability, consistency, and grounded reasoning are the new benchmarks.

According to the researcher, the experiment was conducted via Opper.ai to ensure clean, reproducible conditions. The full dataset and model-by-model results are publicly available in the original Reddit thread. Experts in AI safety warn that the public’s growing trust in AI assistants — often based on polished single responses — may be dangerously misplaced. "We’re not building thinking machines," said Dr. Elena Ruiz, an AI cognition researcher at Stanford. "We’re building sophisticated pattern generators. This test proves that even basic causal reasoning remains fragile."

As AI becomes embedded in everyday life, the need for transparent, auditable reasoning systems grows urgent. The car wash test may seem trivial, but it reveals a fundamental gap between human intuition and machine cognition — a gap that, if unaddressed, could lead to costly, even dangerous, failures in real-world applications.