CAMP-BENCH / SEPTEMBER 2026

What do our benchmarking tools cost?

Trident, Firecrawl Agent and Crawl4AI: the cost of one attempt, how repeated runs add up, and where each number comes from.

Trident produced draft data in 4 of 12 attempts.

It did extract candidate camp records. None of the attempts delivered a final result Trident accepted. The pilot used $3.69 in estimated model usage, averaging $0.31 per attempt.

What does “failed” mean in this pilot?

We checked whether Trident finished and saved an extraction that passed its own checks. We have not separately checked each draft’s camp names, dates, prices and ages against the captured website pages.

  • 6 attempts reached the 20-minute deadline during page capture. 5 of those made no model calls.
  • 4 attempts were rejected by Trident’s own source-validation checks, which test whether its generated extractor relies on page evidence.
  • 2 attempts stopped on other Trident execution errors.

The exact cause of the page-capture stalls has not been isolated. These results do not establish how much is due to Trident versus this local execution setup. The earlier Trident benchmark completed with a different Trident version.

1. What Trident extracted from each website

Three fresh attempts per website
Website
Draft data Trident produced
How the attempts ended
All 3 attempts
Great Work Montessori
Attempt 1: 1 activity rows, 5 session rowsAttempt 3: 3 activity rows, 3 session rows
Attempt 1: Trident rejected the draft in its source checksAttempt 2: Page capture reached the 20-minute deadlineAttempt 3: Trident rejected the draft in its source checks
$0.49
Sundial Classical School & Farmstead
No nonempty draft output was captured
Attempt 1: Page capture reached the 20-minute deadlineAttempt 2: Page capture reached the 20-minute deadlineAttempt 3: Page capture reached the 20-minute deadline
$0.00
Little Yellow House Art
Attempt 1: 2 activity rows, 6 session rows
Attempt 1: Trident rejected the draft in its source checksAttempt 2: Trident stopped on an extraction execution errorAttempt 3: Page capture reached the 20-minute deadline
$1.15
Northwest Pottery Collective
Attempt 1: 3 activity rows, 7 session rows
Attempt 1: Trident rejected the draft in its source checksAttempt 2: Page capture reached the 20-minute deadlineAttempt 3: An internal Trident step timed out
$2.05

These counts describe the last captured nonempty draft from each attempt, before final acceptance. They are unverified rows, not a count of unique or correct camps. Repeated attempts can describe the same sessions.

A rejected draft is different from no extraction. Page-capture timeouts and execution errors can end an attempt before any draft is captured. No refreshes ran.

2. What each tool costs

Per website attempt · different estimation bases

We cannot rank these tools by cost yet. Trident’s number is measured unsuccessful usage, Firecrawl’s is a credit allowance, and Crawl4AI’s is an assumed workload. Provider rates were checked 2026-09-16.

Tool
Cost for one website attempt
Where the number comes from
Sources
TridentWebsite extraction
$0.31 observed averageUnsuccessful attempts · $3.69 across 12
Measured usage estimateActual tokens × model prices. All 12 attempts failed or timed out; successful-run cost is still unknown.
Firecrawl AgentSpark 2 · high effort
$1.00 credit allowance200 credits per attempt · actual usage varies
Configured budget at published ratesHobby top-up: $5 per 1,000 credits, or $0.005 per credit. A credit is a billing unit, not an entire website extraction.
Crawl4AI + AI modelLocal crawler + GPT-5.6 Terra
$0.26 assumed model cost100,000 input + 5,000 output tokens
Published rates + assumed usageCrawl4AI is free. Terra costs $2 per million input tokens and $12 per million output tokens. Running the browser adds compute cost.

Trident comes from recorded usage. Firecrawl and Crawl4AI prices come from provider documentation; their example costs use the assumptions shown here.

The same unit makes the arithmetic readable; different workloads and outcomes prevent a measured price ranking. Subscription fees, local compute and team time are additional.

3. How we calculated Trident’s cost

Actual token usage × published model rates

Run the same test repeatedly

We tried 4 websites 3 times each, starting fresh each time. Two had been tested locally before; two were new to the local run history.

Count every model call

We recorded 23 calls, including repairs: 1,684,219 input tokens and 52,171 output tokens. Tokens are the units used to bill model text.

Price the tokens and divide

We applied Terra and Luna’s input, cached-input and output rates to each call. Total: $3.69. Divide by 12 attempts: $0.31 per attempt.

This is what the recorded model usage would cost at API prices. The pilot used the existing subscription, so $3.69 is not an extra charge on a bill. See the token counts and exact rates ↗

4. What a repeated benchmark would cost

A worked planning example

One attempt means one tool trying one website. Websites × attempts per website × benchmark cycles gives the number of attempts for each tool.

Whole numbers from 1 to 1,000. This calculator does not start any runs.

4 × 3 × 1 = 12 attempts per tool$18.81Illustrative total across all three tools
Tool
One attempt
Your benchmark
Calculation basis
Trident
$0.31
$3.69
Observed failed-attempt average; assumes the same workload and failure mix.
Firecrawl Agent
$1.00
$12.00
200 credits per attempt × $0.005/credit. Values the configured credit budget.
Crawl4AI + Terra
$0.26
$3.12
Assume 100,000 uncached input + 5,000 output tokens per attempt. Model only.

The total combines a Trident failed-attempt average, a Firecrawl credit budget, and assumed Crawl4AI model usage. It is an illustration, not a forecast or spending cap. Refreshes are excluded.

What these examples assume

Firecrawl: 200 credits per attempt at the Hobby top-up rate. Top-ups require a paid plan; its subscription fee is excluded. Included credits, promotions and other plans can change the actual bill.

Crawl4AI: 100,000 uncached input tokens and 5,000 output tokens for one Terra request. That is $0.20 input + $0.06 output = $0.26. This workload is an example we chose, not a provider promise or a measured Crawl4AI run.

Trident: The same usage and failure mix as the pilot. A useful successful-extraction budget still needs successful runs.