Plan once, ground locally: 83.6% fewer agent tokens

If you have built or priced a web agent, an AI system that operates a browser to complete a task, you will know where the money goes. Most agents call a large language model after every single action: look at the page, decide, click, look again, decide again. A ten-step task becomes ten or more model calls, each carrying a large chunk of page text.

Our research preprint, Plan Once, Ground Locally by Divyansh Shukla, tests a different design. The model writes the whole plan in one call, and a small model running on the device carries out each step. On a benchmark of 200 tasks, that design matched a fairly-budgeted conventional agent on success while using 83.6% fewer input tokens.

This post explains the idea, how it was tested, what the numbers say and what they do not, without assuming you read machine learning papers.

The usual design: ReAct

The common pattern for agents is called ReAct, short for "reason and act". At every step the language model sees the current state of the page, writes down its reasoning and chooses one action. The browser performs it, the new page state goes back to the model, and the loop repeats until the task is done.

It is flexible, because the model reacts to whatever the page shows. It is also expensive and slow, because every step is a round trip to a large model with a large prompt.

The alternative: plan once, ground locally

The plan-and-execute agent splits the work in two.

1. Plan. The language model reads the task and writes the complete plan in a single call: go to this page, search for this, open the third result, read the price.
2. Ground. A small local model takes each step and works out which element on the actual page it refers to. In the paper this model is Laya, a ModernBERT-large decision model used without task-specific training, running on Apple's MLX framework on a laptop. It also decides when content that loads late has arrived and checks actions that left the page unchanged.

The language model is called again only to replan after a failure and to write the final answer. In the benchmark, that came to about two LLM calls per task.

Why this can work: most of what an agent does on a page is not hard reasoning. "Click the button that says Add to cart" needs matching words to elements, not a frontier model. Moving that matching to a small local model, which answered in 31 ms median per grounding call, removes most of the expensive calls.

How it was tested

The study built a new benchmark rather than reusing developer test tasks the agent had been tuned on.

• 200 tasks on seven public websites, including practice sites for scraping and shopping flows, Wikipedia, and a TodoMVC to-do app.
• Ground truth scraped from the sites or produced by scripted browser runs, and graded with fixed regular expressions, so no model judged another model's answer.
• Four configurations, each run three times on every task with the same model (DeepSeek v4.1 flash, temperature 0, reasoning off): 2,400 runs.
• A second model (Gemini 2.5 Flash Lite) on the two central configurations: another 400 runs, for 2,800 recorded runs in all.
• Every trace published: each prompt, completion, tool call and local-model decision, with the code on GitHub.

The stack was Playwright driving a real Chromium browser, ModernBERT on MLX for grounding, and OpenRouter for the language model.

The fair comparison

The most important design choice in the study is the baseline. A plain ReAct agent was given a 4,000-character page-reading limit and 15 turns. The plan agent could read more of the page. That is not a fair fight, so the study also ran a budget-matched ReAct agent with a 12,000-character limit and 30 turns. The headline comparison is against that stronger baseline.

The results

Configuration	Success rate	Mean time	LLM calls per task	Input tokens per task	Cost per task
ReAct	80.7%	32.8 s	6.03	20,523	$0.00186
ReAct, budget-matched	92.0%	42.2 s	6.22	24,697	$0.00202
Plan + local grounding	88.8%	10.2 s	2.03	4,060	$0.00040
Plan + word matching only	86.5%	10.8 s	2.12	4,305	$0.00042

Read plainly:

• On success, it is a tie. Against the budget-matched baseline the plan agent was 3.2 points lower, with a 95% confidence interval from 7.7 points lower to 1.2 points higher. That difference is not statistically significant in any of the three repeats.
• On efficiency, it is not close. Against the same baseline the plan agent used 83.6% fewer input tokens, 67.3% fewer LLM calls and 80.4% less money, and ran 4.1 times faster.
• The apparent win over plain ReAct is mostly budget. The plan agent beats the unmatched baseline by 8.2 points, but giving ReAct the same reading budget gains it 11.3 points. The study reports this rather than leading with the flattering number.

The limits, stated honestly

The paper is direct about what it did not show, and so should we be.

• The local model is not clearly better than simple word matching. Replacing it with word-overlap matching cost 2.3 points, not a significant difference. It did help where identical buttons needed context, for example on a shopping practice site, 60 of 60 runs against 49 of 60.
• Planning ahead has a blind spot. On the TodoMVC tasks the plan agents passed 0 and 2 of 30 runs against 26 of 30 for the budget-matched baseline. A plan written before the page exists cannot name list items that appear only after you type them.
• Cost savings depend on the model. With the second model, input-token savings held (81.8% fewer), and the plan agent was clearly more accurate, 66.0% against 47.5%, because the weaker model used up its turns as a ReAct agent. But it cost 11% more per task. In 15 of 200 runs that model fell into repetitive output while writing a plan and hit the provider's generation cap. Planning once puts the whole output budget into one long generation, so a model that loops turns the best case into the worst. Excluding those runs, the plan agent cost 67% less.
• Scope. One main model, seven sites that overlap with the development suite, the same author building both agents, and regex grading. These are real constraints on how far the result generalises.

What it means if you are building agents

If you are commissioning or building an agent for your business, a few practical lessons follow.

1. Count model calls, not just model price. Choosing a cheaper model saves less than calling a model fewer times.
2. Push simple decisions to small models. Matching a step to a button, checking whether a page has loaded and confirming that an action did something rarely need a large model.
3. Keep a fallback for pages that change as you act. Forms that grow, lists that appear after typing and multi-step flows with branching may still need a step-by-step loop. A hybrid that plans once and replans when the page surprises it is a sensible default.
4. Compare against a fair baseline. If a vendor shows a large improvement, ask whether the baseline was given the same budget.
5. Report cost per model. Input-token savings come from the architecture. Total cost depends on how the chosen model behaves, including how it fails.
6. Keep traces. Every finding above, including the failures, came from reading recorded traces. If your agent does not log what it saw and did, you cannot improve it.

Read the paper

The preprint, data and traces are archived at doi.org/10.5281/zenodo.22904595, and the code is at github.com/TheDivyanshShukla/plan-once-ground-locally. The code is MIT licensed and the benchmark tasks, results and traces are CC BY 4.0.

If you are planning a browser agent or another automation where model costs matter, we are happy to talk through the design with you. Read more about our AI automation work, or get in touch.