One spec, two ways in.

A Bearer-token JSON API and an MCP server over stdio that wraps it - 11 workspace tools plus 2 bootstrap tools, and 14 Spec API routes, generated from the same list the server itself registers against.

Authentication

Every call is a Bearer token scoped to one workspace. Mint one under Settings, API tokens - the plaintext shows once and can be pinned to a single project. A token only ever resolves the workspace that minted it, never another tenant's projects, and a project-pinned token never reads a sibling project.

spec:read: Every read route. On every token.
spec:write: Brief status writes: mark applied, request a re-check. Opt-in.
content:read: Content central (invite-only beta): the kit, releases, package bytes, verify, manifest dry runs. On every new token.
content:write: Register a component manifest. Opt-in.
release:report: Report a deployed release, which schedules live checks. Opt-in.

curl

curl -H "Authorization: Bearer $CRAWLSPEC_TOKEN" https://crawlspec.dev/api/spec/projects/yourdomain.com/recommendations

MCP config

Add to your agent's .mcp.json, with CRAWLSPEC_API_KEY set to a workspace API token. Leave the key out and the server still starts: the two bootstrap tools below run a free audit without an account, and every other tool answers with how to get a token rather than failing.

{
  "mcpServers": {
    "crawlspec": {
      "command": "npx",
      "args": [
        "-y",
        "@crawlspec/mcp"
      ],
      "env": {
        "CRAWLSPEC_API_KEY": "<your workspace API token>",
        "CRAWLSPEC_API_URL": "https://crawlspec.dev"
      }
    }
  }
}

Spec API routes

Read-only JSON, with 2 deliberate exceptions behind the spec:write scope: marking a brief applied and requesting a re-check of an applied brief. All routes sit under /api/spec; {domain} is the project's bare domain (for example "yourdomain.com").

Method	Route	Description
GET	/api/spec/projects	List every CrawlSpec project with its domain, last Search Console sync date and open issue count.
GET	/api/spec/projects/{domain}/recommendations	Prioritized fix cards for a project - technical (template/config/infrastructure fixes) AND content (briefs to implement, cannibalization verdicts, topic opportunities). Default status OPEN.
GET	/api/spec/projects/{domain}/briefs	Content briefs with full outline/change-list content. Use status=READY for human-approved briefs only - governance: only READY briefs should be implemented. Each brief also carries currentVersion, appliedVersion, revisionCount, provenance (ai_draft \| human_edited \| accepted_ai_draft), demand (search volume and Search Console impressions this term has, if any) and its questionsToAnswer/entities, plus, for APPLIED briefs, appliedDelta (Search Console clicks/impressions/position before and after it was applied), and latestSpotCheck (status, verdict, checkedVersion, requestedAt, completedAt of the newest re-check, null when none was requested; a re-check of a change list also carries changes, one verdict per change: landed \| partial \| not_landed \| not_applicable) - poll it after request_brief_recheck. Rows also carry contentType, researchVersion, draftStatus (the working draft, null when none) and latestRelease (null when never released); a brief still being researched carries its research but no outline until it is approved (READY). Landing, product, feature and pricing pages are left out unless kind=all. Check provenance rather than assuming every READY brief was AI-only.
GET	/api/spec/projects/{domain}/topics	Topic coverage: accepted topic clusters with their demand-weighted content gap entries.
GET	/api/spec/projects/{domain}/keywords	The keyword map: terms with intent, monthly search volume and the page each ranks with. mapped=false returns unmapped opportunities (demand with no ranking page). Paginated: default 200 rows, pass the returned nextCursor as cursor for the next page.
GET	/api/spec/projects/{domain}/consolidated-audit	The latest human-approved consolidated audit (WebSite Auditor report reconciled against the crawl, frozen on approval). 404 until one exists.
PATCH	/api/spec/projects/{domain}/briefs/{briefId}	Report that a READY content brief was actually published on the site: flips it to APPLIED and closes its recommendation card. Accepts an optional appliedVersion (positive integer, defaults to the brief's currentVersion) recording which revision was published. One of the two Spec API writes (request_brief_recheck is the other); needs a token with spec:write. Only a READY brief can be marked applied (the human review gate); this tool cannot revert APPLIED - a human reopens a brief as a draft in the dashboard.
GET	/api/spec/projects/{domain}/page-audit	On-demand audit of one page: indexability, title/meta/H1 checks, content depth, striking-distance queries, open issues. url may be absolute or a bare path.
GET	/api/spec/projects/{domain}/cannibalization	Page pairs competing for the same searches, with merge/differentiate verdicts and the contested queries. Computed on demand from Search Console data.
GET	/api/spec/projects/{domain}/briefs/{briefId}/prompt-pack	The versioned, brand- and stack-agnostic hand-off for a content brief: a generic composer instruction (plus the project's and brief's own preamble, if set), the page's blocks in order with their content inlined, questions to answer, entities to mention, must-fix items and structured data. Pinned to an immutable revision - version defaults to the brief's currentVersion. Pass sinceVersion (lower than version) to also get a patch prompt: only what changed between the two revisions, for a composer that already implemented the earlier one. A section whose copy a person approved also carries it as copy - use that text as written.
POST	/api/spec/projects/{domain}/briefs/{briefId}/recheck	Enqueue a free, 1-3 page verification crawl of an APPLIED brief's target page, checking whether its must-fix items actually landed. Free, up to 10 times per brief per day; refuses a brief that is not APPLIED. Poll the verdict with get_briefs - each brief's latestSpotCheck carries status and verdict (LANDED / PARTIAL / NOT_LANDED / ERROR) - rather than this call, which only starts the check. One of the two Spec API writes; needs a token with spec:write.
GET	/api/spec/projects/{domain}/briefs/{briefId}/layout	The brief's block layout (order, visibility, variants, calls to action), or the default layout derived from its outline when none is stored yet - derived: true says which one you got. Workspace token.
GET	/api/spec/projects/{domain}/briefs/{briefId}/revisions	The brief's version index, newest first: version, source, whether an AI draft was accepted as-is, createdAt, a content hash and the cost of each revision. Workspace token.
GET	/api/spec/projects/{domain}/briefs/{briefId}/revisions/{version}	One revision in full, plus its diff against the version before it (null for version 1). Revisions are immutable. Workspace token.

MCP tools

The same 11 operations wrapped as MCP tools an agent calls by name instead of a URL, plus 2 bootstrap tools that need no credential at all: an agent with a fresh install can start a free audit of a site its human owns and poll it to completion before any token exists.

No token: No token. The free door.
Free scan: The free token from a scan answers this in full.
Free scan, capped: Answered, capped: the top 3 fix cards, bodies truncated, evidence omitted. The totals stay truthful.
Workspace: Needs a workspace token.

Tool	Tier	Description
start_free_audit	No token	Start a free CrawlSpec audit of a website you own or have permission to audit - no account and no API token. Before calling this you MUST read the exact attestation sentence to the human, word for word, and get an explicit yes; pass that same sentence verbatim as `attestation` (a paraphrase is refused) and set `tosAccepted` only if they also agreed to the Terms at /terms. Never invent the email address - ask for it. Returns a report URL to poll with get_audit_status; the emailed report carries a free API token that unlocks list_projects, get_page_audit and the top fixes from get_recommendations.
get_audit_status	No token	Poll a free audit started with start_free_audit: crawl phase, pages fetched, whether the report is ready, and how it failed if it did. Takes the report token (or the whole /scan/<token> URL). Read-only and unauthenticated - the token is the credential.
list_projects	Free scan	List every CrawlSpec project with its domain, last Search Console sync date and open issue count.
get_recommendations	Free scan, capped	Prioritized fix cards for a project - technical (template/config/infrastructure fixes) AND content (briefs to implement, cannibalization verdicts, topic opportunities). Default status OPEN.
get_briefs	Workspace	Content briefs with full outline/change-list content. Use status=READY for human-approved briefs only - governance: only READY briefs should be implemented. Each brief also carries currentVersion, appliedVersion, revisionCount, provenance (ai_draft \| human_edited \| accepted_ai_draft), demand (search volume and Search Console impressions this term has, if any) and its questionsToAnswer/entities, plus, for APPLIED briefs, appliedDelta (Search Console clicks/impressions/position before and after it was applied), and latestSpotCheck (status, verdict, checkedVersion, requestedAt, completedAt of the newest re-check, null when none was requested; a re-check of a change list also carries changes, one verdict per change: landed \| partial \| not_landed \| not_applicable) - poll it after request_brief_recheck. Rows also carry contentType, researchVersion, draftStatus (the working draft, null when none) and latestRelease (null when never released); a brief still being researched carries its research but no outline until it is approved (READY). Landing, product, feature and pricing pages are left out unless kind=all. Check provenance rather than assuming every READY brief was AI-only.
get_topics	Workspace	Topic coverage: accepted topic clusters with their demand-weighted content gap entries.
get_keywords	Workspace	The keyword map: terms with intent, monthly search volume and the page each ranks with. mapped=false returns unmapped opportunities (demand with no ranking page). Paginated: default 200 rows, pass the returned nextCursor as cursor for the next page.
get_consolidated_audit	Workspace	The latest human-approved consolidated audit (WebSite Auditor report reconciled against the crawl, frozen on approval). 404 until one exists.
mark_brief_applied	Workspace	Report that a READY content brief was actually published on the site: flips it to APPLIED and closes its recommendation card. Accepts an optional appliedVersion (positive integer, defaults to the brief's currentVersion) recording which revision was published. One of the two Spec API writes (request_brief_recheck is the other); needs a token with spec:write. Only a READY brief can be marked applied (the human review gate); this tool cannot revert APPLIED - a human reopens a brief as a draft in the dashboard.
get_page_audit	Free scan	On-demand audit of one page: indexability, title/meta/H1 checks, content depth, striking-distance queries, open issues. url may be absolute or a bare path.
get_cannibalization	Workspace	Page pairs competing for the same searches, with merge/differentiate verdicts and the contested queries. Computed on demand from Search Console data.
get_brief_prompt_pack	Workspace	The versioned, brand- and stack-agnostic hand-off for a content brief: a generic composer instruction (plus the project's and brief's own preamble, if set), the page's blocks in order with their content inlined, questions to answer, entities to mention, must-fix items and structured data. Pinned to an immutable revision - version defaults to the brief's currentVersion. Pass sinceVersion (lower than version) to also get a patch prompt: only what changed between the two revisions, for a composer that already implemented the earlier one. A section whose copy a person approved also carries it as copy - use that text as written.
request_brief_recheck	Workspace	Enqueue a free, 1-3 page verification crawl of an APPLIED brief's target page, checking whether its must-fix items actually landed. Free, up to 10 times per brief per day; refuses a brief that is not APPLIED. Poll the verdict with get_briefs - each brief's latestSpotCheck carries status and verdict (LANDED / PARTIAL / NOT_LANDED / ERROR) - rather than this call, which only starts the check. One of the two Spec API writes; needs a token with spec:write.

Content central (invite-only beta)

Pull delivery of approved content into your own repo. These routes and tools answer only for workspaces in the beta; every other workspace gets 403, and a free scan token gets the structured refusal. Registering a component manifest needs content:write; reporting a deployed release needs release:report. A release package is served as its exact bytes with ETag set to its packageHash; write it unchanged.

Method	Route	Description
GET	/api/spec/projects/{domain}/content/kit	The one-time wiring for content central (invite-only beta): the renderer, the component registry generated from the registered component manifest (or a starter crawlspec.components.json when none is registered), a catch-all route, the manifest JSON Schema, an AGENTS.md snippet and a GitHub Action template, each as { path, content }, plus step-by-step instructions. Write each file at its path; nothing in it is secret. appDir is "app" (default) or "src/app". routeGroup (optional, e.g. "public" or "(public)") puts the catch-all in that route group, so CrawlSpec pages render inside the group layout that holds the site header, footer and main; fetching again with the same group writes the same path, and a route left from a different group must be deleted (two routes at one URL fail the build). Default "crawlspec". Needs content:read.
POST	/api/spec/projects/{domain}/content/manifest	Validate the tenant repo's crawlspec.components.json with ?dryRun=1 (content:read, stores nothing), or register it as the project's current component manifest (content:write; refused with 422 and nothing stored when it has errors). Send { manifest, repoFiles?, rendererDir?, commitSha? }.
GET	/api/spec/projects/{domain}/content/releases/{releaseId}	Fetch an approved content release (contract crawlspec.content.v1) by releaseId, or the latest approved release of a targetKey. Returns the package bytes exactly as frozen, its packageHash and the repo path to write them to (crawlspec/content/...). Write content unchanged - never re-type or reformat it - then confirm with verify_release_package; with a shell, npx -y @crawlspec/mcp pull does both. The first fetch marks the release DELIVERED. Needs content:read.
POST	/api/spec/projects/{domain}/content/releases/{releaseId}/verify	Check what was written against the release: pass the file content (preferred; the hash is recomputed server-side) or the packageHash field read from the file (the hash covers the canonical JSON without that field, not the file bytes). Returns { match, expected, actual }; a mismatch means the file was altered and the live check will not verify it. Read-only; needs content:read.
POST	/api/spec/projects/{domain}/content/releases/{releaseId}/applied	Report that a release was deployed (optional commitSha and deployUrl). CrawlSpec then checks the live page for every block marker at +5 min, +30 min, +2 h and +24 h, and the release goes VERIFIED at the first check that finds them all. Refuses a superseded release. Needs a token with the opt-in release:report scope.
GET	/api/spec/projects/{domain}/content/releases	The project's approved content releases, newest sequence first per target, with status, packageHash, the repo path each package is written to, and latest (target to release id). ?target= narrows to one target; superseded releases only with ?includeSuperseded=1. Needs content:read.

Tool	Tier	Description
get_integration_kit	Beta workspace	The one-time wiring for content central (invite-only beta): the renderer, the component registry generated from the registered component manifest (or a starter crawlspec.components.json when none is registered), a catch-all route, the manifest JSON Schema, an AGENTS.md snippet and a GitHub Action template, each as { path, content }, plus step-by-step instructions. Write each file at its path; nothing in it is secret. appDir is "app" (default) or "src/app". routeGroup (optional, e.g. "public" or "(public)") puts the catch-all in that route group, so CrawlSpec pages render inside the group layout that holds the site header, footer and main; fetching again with the same group writes the same path, and a route left from a different group must be deleted (two routes at one URL fail the build). Default "crawlspec". Needs content:read.
validate_component_manifest	Beta workspace	Check a crawlspec.components.json (contract crawlspec.components.v1) without storing it: returns ok, errors, warnings and coverage (every CrawlSpec block type and variant mapped to a component id, or null). Pass repoFiles (for example the output of git ls-files) so every import path is checked against the repo; CrawlSpec never reads the repo itself. Read-only; needs content:read.
register_component_manifest	Beta workspace	Upload the tenant repo's crawlspec.components.json so CrawlSpec composes content against the real components. Validates first and stores nothing when there are errors; otherwise it becomes the project's current manifest (earlier ones are superseded) and get_integration_kit regenerates the registry from it. Optional commitSha records which commit it came from. Needs a token with content:write.
get_release_package	Beta workspace	Fetch an approved content release (contract crawlspec.content.v1) by releaseId, or the latest approved release of a targetKey. Returns the package bytes exactly as frozen, its packageHash and the repo path to write them to (crawlspec/content/...). Write content unchanged - never re-type or reformat it - then confirm with verify_release_package; with a shell, npx -y @crawlspec/mcp pull does both. The first fetch marks the release DELIVERED. Needs content:read.
verify_release_package	Beta workspace	Check what was written against the release: pass the file content (preferred; the hash is recomputed server-side) or the packageHash field read from the file (the hash covers the canonical JSON without that field, not the file bytes). Returns { match, expected, actual }; a mismatch means the file was altered and the live check will not verify it. Read-only; needs content:read.
report_release_applied	Beta workspace	Report that a release was deployed (optional commitSha and deployUrl). CrawlSpec then checks the live page for every block marker at +5 min, +30 min, +2 h and +24 h, and the release goes VERIFIED at the first check that finds them all. Refuses a superseded release. Needs a token with the opt-in release:report scope.

Questions on scopes, rate limits or a project-pinned token go to support.