CrawlSpecBot/1.0

See what an AI coding agent would fix on your site

We crawl 25 pages, find the template-level problems, and hand your agent a prompt it can act on. Free, no account, no card.

Pages crawled
25

Depth 2, robots.txt honoured, one polite pass

Fixes in full
3

Each with the prompt to hand your agent, and the rest counted honestly

Accounts and cards
0

An address to send the report to, and nothing else

Scan a site

The report opens in your browser as soon as the crawl settles, and we email you the link.

25 pages, depth 2, robots.txt honoured. Most scans finish in under a minute, and the report link stays live for 14 days.

What we do not do

  • We never edit or publish anything on your site. Every brief is an editable artifact you arrange and finish, then hand to your team or your own AI tools.
  • We respect robots.txt on every run, with no setting that overrides it, for any customer.
  • We read at most 25 pages at depth 2, a handful of requests at a time, and we identify ourselves as CrawlSpecBot on every one of them.
  • We do not send your report to anyone but the address you typed, and we never sell or share it.
  • We do not run AI over your site. The free scan is crawl output and arithmetic, start to finish.

The crawler has its own page, including how to block it in one line of robots.txt. About CrawlSpecBot

What a free audit does not include

The free scan is a real audit of a small sample, not a trial with the answers removed. Here is what it cannot do.

Pages 2,000 at a time
A free scan reads 25 pages, which finds template problems but cannot see a section it never reached.
Every fix card, not three
The rest are counted and titled on your report. They open when the audit moves into a workspace.
Re-crawls that close cards
A card closes itself when the next crawl re-observes the template. A free scan runs once, so nothing can close.
Search Console and content briefs
Clicks and positions joined onto every page, keyword maps, cannibalization pairs, and briefs written from the audit.

Every one of those ships on the paid plan. See what a plan costs.

A free SEO audit for Claude Code, Cursor and Codex

Your report carries a prompt per fix and one for the whole audit, plus spec.md and spec.json downloads. None of them carries a credential, so they are safe to paste into a shared chat. A workspace adds the other half of the loop: 11 read-only MCP tools an agent queries directly, documented on the API page.

What the report looks like

An illustrative audit of a fictional store. Every number below is written by hand and internally consistent with a 25-page scan; no customer data appears anywhere on this page.

Sample report - a fictional store, not a real site

SEO audit of example-shop.test

Pages scanned
25
Issues found
312
Fix cards
8
Scanned
2026-09-08
Expires in
14d

14 days from the scan

Issues by severity

  • CRITICAL2
  • HIGH24
  • MEDIUM61
  • LOW187
  • INFO38

The crawl was capped, so these counts describe the pages we reached, not the whole site.

Top 3 fixes

1. Product pages ship without a meta description

Fix the template
P 8.6reach: 19 pagesMISSING_META_DESCRIPTION

Nineteen of the twenty-five pages we read are product pages built from one template, and that template emits no meta description. Google writes its own snippet from whatever text it finds, which on these pages is the shipping notice.

Evidence and affected URLs

How to fix

  • Next.js Return a description from the route segment generateMetadata, composed from the same product record the page renders. A missing description is a data gap, not a copy task.

Evidence

All nineteen pages return an empty <meta name="description">; the tag is present, so the template renders it unconditionally from a field that is never set.

Examples

  • /p/brass-kettle-1l
  • /p/enamel-mug-navy
  • /p/cast-iron-skillet-26cm

Affected URLs

/p/*

  • https://example-shop.test/p/brass-kettle-1l
  • https://example-shop.test/p/enamel-mug-navy
  • https://example-shop.test/p/cast-iron-skillet-26cm
  • https://example-shop.test/p/linen-apron-sand
  • https://example-shop.test/p/copper-measuring-set

2. Paginated category pages all share one title

Fix the template
P 7.4reach: 6 pagesDUPLICATE_TITLE

Six category pages carry the identical title "Kitchen - Example Shop", because the template ignores the page number. Google picks one of them to show and treats the rest as near-duplicates of it.

Evidence and affected URLs

How to fix

  • Next.js Read the page parameter in generateMetadata and suffix the title with " - page N" when it is greater than one. The same value should drive the self-canonical.

Evidence

Six URLs, one title string, one meta description. The only difference between the documents is the product grid.

Examples

  • /c/kitchen
  • /c/kitchen?page=2
  • /c/kitchen?page=3

Affected URLs

/c/kitchen*

  • https://example-shop.test/c/kitchen
  • https://example-shop.test/c/kitchen?page=2
  • https://example-shop.test/c/kitchen?page=3
  • https://example-shop.test/c/kitchen?page=4

3. Product photography carries no alt text

Fix the content
P 6.9reach: 148 resourcesIMAGE_MISSING_ALT

One hundred and forty-eight images across the pages we read have an empty alt attribute. That costs you image search, and it is the single most common accessibility complaint a shop gets. The gallery component sets alt="" on every slide.

Evidence and affected URLs

How to fix

  • Next.js The alt prop on next/image is required but accepts an empty string, which is why this passed review. Pass the product title from the same record that supplies the src.

Evidence

Of 161 images seen, 148 have alt="". The 13 that do not are the site logo and the payment icons in the footer.

Affected URLs

/p/*

  • https://example-shop.test/p/brass-kettle-1l
  • https://example-shop.test/p/enamel-mug-navy

5 more fixes this scan found

  • 4. Filtered category URLs canonicalise to themselves9 pages
  • 5. Journal posts have no internal links pointing at them4 pages
  • 6. The sitemap lists URLs that redirect7 pages
  • 7. No Product structured data on product pages19 pages
  • 8. Category pages carry under 40 words of their own copy5 pages

Ranked by the same score as the cards above. They open when this audit moves into a workspace.

Hand the whole audit to your agent

One prompt with the site facts, the 3 fixes above, the order to do them in and how to verify. It carries no credential, so it is safe to paste anywhere.

Point your agent at this audit

The prompts above are self-contained: an agent can do the work with no account and no credential. The MCP server adds the other half of the loop, so the agent re-reads the audit and re-checks a page itself after it edits one. 11 tools over the same read-only API, documented on the API page.

{
  "mcpServers": {
    "crawlspec": {
      "command": "npx",
      "args": [
        "-y",
        "@crawlspec/mcp"
      ],
      "env": {
        "CRAWLSPEC_API_KEY": "<your CrawlSpec token>",
        "CRAWLSPEC_API_URL": "https://crawlspec.dev"
      }
    }
  }
}

The token goes in that placeholder. Tokens come by email, never on this page, and a workspace mints its own from project settings once this audit is claimed.

Not in a free scan

Fixes still locked
5

Ranked by the same score, hidden on the free scan

Pages per crawl on Pro
2,000

25 in a free scan

Re-crawls
0

A free scan runs once and never again

Content briefs
0

Written from the audit plus real search demand

Search Console days
0

Clicks and positions joined onto every page

Agent tools on a workspace
11

Briefs, topics, keywords and the consolidated audit

Frequently asked

Do I have to own the site?
Yes, and the checkbox says so in the sentence we store: "I own this site or have permission to audit it, and I understand CrawlSpec will crawl up to 25 public pages while respecting robots.txt." That sentence is frozen on the record of your request, with the time and the address that made it. A domain that already belongs to a CrawlSpec workspace is refused.
What does the crawler do to my server?
It reads robots.txt first and obeys it, fetches at most 25 HTML pages plus the resources they reference, a few requests at a time, and stops. It identifies itself as CrawlSpecBot and links to a page explaining itself on every request.
How long do you keep the report and my address?
The report stays readable for 14 days and is then deleted. Your address is kept only while one of your reports is live, after which the record keeps a one-way hash and nothing readable. An unclaimed scan project is deleted 90 days after its last scan.
How do I stop this, or delete what you have?
Every report email carries a link with two buttons: stop these emails, and delete my data. The second one removes the scans, the record and the project. If the site is not yours to begin with, the crawler page has a removal request and a self-serve block that needs no account.

Ready for the whole site instead of a sample? Founding 50 is 50% off the yearly price, for the first 50 workspaces.