Sheets Benchmark

Tool architecture matters more than model choice

25 tasks 7 models 3 tool configurations Last run: 2026-02-28 Cell-level validation

The Tool Architecture Gap

The same model tested across three tool configurations: raw Google Sheets API, a generic MCP wrapper, and Daslab's optimized tool layer. The difference isn't the model — it's the tools.

Model	Raw API	Generic MCP	Daslab Tools	Delta
Claude Sonnet 4	52%	72%	92%	+40pts
GPT-4o	36%	64%	84%	+48pts
Gemini 2.5 Pro	44%	68%	88%	+44pts
GPT-4o mini	20%	44%	68%	+48pts

Avg. with raw API

38%

Avg. with Daslab tools

83%

Average improvement

+45 pts

Model Leaderboard (Daslab Tools)

#	Model	Overall	Basic	Multi-step	Complex	Avg calls	Retry rate	Cost/task
1	Claude Sonnet 4 Anthropic	92%	100%	90%	80%	4.2	3%	$0.032
2	Gemini 2.5 Pro Google	88%	100%	85%	60%	5.1	8%	$0.028
3	GPT-4o OpenAI	84%	100%	80%	60%	5.8	12%	$0.041
4	Claude 3.5 Haiku Anthropic	76%	90%	70%	40%	6.4	15%	$0.004
5	Gemini 2.5 Flash Google	72%	90%	65%	40%	6.9	18%	$0.003
6	GPT-4o mini OpenAI	68%	85%	60%	40%	7.2	22%	$0.003
7	Llama 4 Scout Meta	56%	75%	45%	20%	9.1	30%	$0.002

Key findings

38% → 83%

Tool layer is the biggest lever

The same models jump +45 points when given optimized tools instead of raw API access. Tool architecture matters more than which model you pick.

GPT-4o mini: 20% → 68%

Weak models become usable

Even the cheapest models achieve reliable results with the right tool layer. A $0.003/task model with good tools beats a $0.041/task model with raw API.

40–80%

Complex tasks separate the field

All models handle basic read/write well (85–100%). The gap emerges on multi-sheet operations, error recovery, and formula preservation — where tool design matters most.

Task categories

Basic Operations 10 tasks

Read & summarize Read a sheet and describe its structure and contents

Update cell range Modify specific cells (e.g., update prices in B2:B5)

Append rows Add new rows at the bottom of an existing sheet

Insert rows after header Insert 3 rows after the header row and fill with data

Delete rows Remove rows 5-7 from the sheet

Clear a range Clear cell values without removing the rows

Create spreadsheet Create a new spreadsheet with headers and initial data

Format header row Apply bold and background color to the header row

Read-modify-write Read values, modify one column, write back

Find and update Find cells matching a pattern and update their values

Multi-Step Workflows 10 tasks

Cross-tab copy Copy data from one sheet tab to another

Sort by column Read data, sort by a specific column, write back in order

Add formula column Insert a new column with calculated formulas

Merge two sheets Read from two sheets, combine data, write to a third

Restructure table Move columns and rename headers

Conditional formatting Highlight cells above a threshold value

Summary aggregation Create a summary sheet from detail data

Multi-position insert Insert rows at multiple positions in one conversation

Status color-coding Color-code a status column based on cell values

Deduplicate rows Identify and remove duplicate rows

Complex / Error Recovery 5 tasks

Large sheet handling Work correctly with a 500+ row sheet

Multi-sheet operations Coordinate changes across multiple sheets

Error recovery Detect a wrong operation and correct it

Formula preservation Edit data without breaking formula references

Multi-account Read from one Google account, write to another

Methodology

Three-axis evaluation

Every task is evaluated across three independent axes: Models (7 LLMs from 4 providers), Tool Configurations (raw API, generic MCP wrapper, Daslab's optimized tools), and Tasks (25 tasks across 3 difficulty tiers). This 7 × 3 × 25 matrix isolates the effect of tool design from model capability.

How tasks are defined

Each task specifies an initial spreadsheet state (headers + rows), a natural-language instruction, and an expected final state. The initial state is written to a real Google Sheet before each run. The LLM receives the instruction and must use the assigned tool set to complete the task.

Tool configurations

Raw API — The model receives only sheets_batch_update and must construct Google Sheets API requests directly. Generic MCP — A standard MCP wrapper with read/write/append tools but no domain-specific helpers. Daslab Tools — The full optimized set: sheets_read, sheets_update_values, sheets_insert_rows, sheets_delete_rows, sheets_append_rows, sheets_format_cells, sheets_clear, and sheets_batch_update — with 1-indexed rows, sheet name resolution, and auto-read-after-write.

How success is measured

After the LLM finishes, we read back the entire sheet and compare cell-by-cell against the expected output. A task passes only if every cell matches exactly. Partial credit is not awarded — this is a binary pass/fail per task.

What we track

Task success rate — percentage of tasks completed correctly. Tool call count — total API calls to complete each task (fewer = better). Retry rate — percentage of tool calls that failed and were retried. Cost per task — total LLM token cost at provider list prices. Latency — wall-clock time from prompt to task completion.

Reproducibility

All task definitions, evaluation scripts, and raw result data will be published in an open-source repository. Anyone with a Google OAuth setup and LLM API keys can reproduce these results on their own infrastructure.