LLM Web Agents: How BFS, DFS, and Best-First Search Impact Planning (2024 Study)

LLM Web Agents: How BFS, DFS, and Best-First Search Impact Planning (2024 Study)

A groundbreaking AI planning framework for LLM-based web agents, published in arXiv:2403.12710v1 by researchers at Carnegie Mellon University, redefines evaluation by mapping modern architectures to classical search algorithms: BFS, DFS, and Best-First Search. This formal taxonomy enables precise diagnosis of failure modes like context drift and incoherent task decomposition—moving beyond simple success rates to explainable agent behavior.

The Three Agent Paradigms: Step-by-Step, Tree Search, and Full-Plan-in-Advance

The study categorizes dominant LLM agent types into three planning styles:

• BFS (Step-by-Step): Explores options sequentially, mimicking human trial-and-error. Ideal for dynamic environments.
• Best-First Tree Search: Uses heuristic scoring to prioritize high-probability paths. Balances efficiency and adaptability.
• DFS (Full-Plan-in-Advance): Generates entire task sequence upfront. Maximizes precision but vulnerable to UI changes.

Evaluating Trajectory Quality: Beyond Success Rates

Traditional metrics like task success rate mask critical behavioral differences. The team introduced five novel trajectory quality metrics:

• Element accuracy: Precision in clicking or inputting UI elements
• Plan coherence: Logical flow of steps
• Step redundancy: Unnecessary or repeated actions
• Tool utilization efficiency: Optimal use of embedded tools (maps, calculators)
• Recovery robustness: Ability to adapt after errors

These metrics were validated using 794 human-labeled trajectories from the WebArena benchmark—a realistic web environment simulating e-commerce, forums, CMS, and dev platforms.

Key Tradeoffs Revealed: Human-Like vs. High-Precision Agents

Experiments comparing a Step-by-Step (BFS) agent and a Full-Plan-in-Advance (DFS) agent uncovered stark contrasts:

• BFS agent: 38% task success rate, but 85% alignment with human trajectories—feels intuitive and adaptive.
• DFS agent: Only 29% task success, yet 89% element accuracy—executes commands with robotic precision.

This reveals a critical tradeoff: human-like behavior vs. technical fidelity. For customer service bots, BFS-style agents win. For backend automation (e.g., compliance auditing), DFS excels.

Practical Implications for Enterprise AI

Deploying the wrong agent type can lead to costly failures:

• Tree Search agents risk combinatorial explosion on complex sites with many nested options.
• DFS agents fail catastrophically when UI elements shift mid-task.
• BFS agents may waste time on dead ends but recover gracefully.

This framework transforms agent selection from guesswork to strategy. As LLM web agents scale into enterprise workflows, understanding why an agent failed matters more than whether it succeeded.