Skip to main content
Use case

SEO analysis and audits

Map site structure, extract on-page signals, and monitor competitor pages at scale. Structured output via API, no browser extension, no CSV export, pipe results directly into your data stack.

Why SEO teams need an API

Desktop crawlers work for one-off audits but don't fit into programmatic workflows: you can't schedule them, pipe their output into a database, or trigger them from a CI job. Every audit is manual, and the data lives in a CSV file rather than your data warehouse.

Competitor monitoring is even harder. Checking fifty competitor pages weekly for content or markup changes doesn't fit into a desktop tool workflow. You end up with stale snapshots or over-triggered HTML-diff alerts that fire on nav changes and cookie banners.

How SuperScraper fits

Map, crawl, and extract on-page SEO signals programmatically. Schedule competitor monitoring feeds that alert on real field changes.

Map any site structure in seconds

/v1/map discovers every URL on a domain in one call by parsing sitemaps and following internal links. Use it to audit site architecture, find orphaned pages, or build a URL inventory before a migration.

Structured on-page signals at scale

/v1/extract with a schema pulls title tags, meta descriptions, H1s, canonical URLs, structured data, and internal link counts from any page. /v1/batch runs it across 100 URLs at once.

Confidence score on extracted fields

Every field returns _confidence. A title tag pulled from the HTML <title> element scores near 1.0. A title inferred from the first heading scores lower. Branch your audit logic on certainty rather than treating every field as equally reliable.

Monitor competitors on a schedule

/v1/feeds schedules recurring extractions. Set it on competitor pages to track title/description changes, new structured-data markup, and content refreshes, alert only on field deltas, not HTML noise.

/v1/map

Discover all URLs on a domain via sitemap + link crawl

/v1/extract

Pull structured SEO fields against a schema

/v1/batch

Extract SEO signals from up to 100 URLs in one request

/v1/crawl

Async full-site crawl for large domains

/v1/feeds

Scheduled monitoring, alert on field deltas, not HTML noise

Full-site SEO audit in two calls

Map the domain, then batch-extract SEO signals across all discovered URLs.

# Step 1, discover all URLs
curl -X POST https://superscraper-production-1381.up.railway.app/v1/map \
  -H "Authorization: Bearer $SS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "url": "https://example.com" }'

# Response
{
  "urls": [
    "https://example.com/",
    "https://example.com/about",
    "https://example.com/pricing",
    ...
  ]
}

# Step 2, extract SEO signals from each page
curl -X POST https://superscraper-production-1381.up.railway.app/v1/batch \
  -H "Authorization: Bearer $SS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "urls": ["https://example.com/", "https://example.com/about"],
    "schema": {
      "title": "string",
      "metaDescription": "string",
      "h1": "string",
      "canonical": "string",
      "hasStructuredData": "boolean",
      "internalLinkCount": "number"
    }
  }'

# Each result includes:
# { ..., "_confidence": 0.94, "_extraction_method": "json-ld+html" }

Frequently asked questions

How do I crawl a full site and extract SEO signals?

First call /v1/map with the domain to get all URLs. Then call /v1/batch with the URL list and a schema that describes the SEO fields you want (title, description, h1, canonical, structuredData). For sites with more than 100 pages, use /v1/crawl which processes the full domain asynchronously.

Can I extract JSON-LD structured data from pages?

Yes. The extraction cascade prioritizes JSON-LD as the highest-quality source. If a page has schema.org markup, /v1/extract returns it as a structured field with a confidence score near 1.0 and _extraction_method set to "json-ld". You can also request the raw JSON-LD via /v1/scrape.

How does the site mapping compare to Screaming Frog?

/v1/map is an API call that returns a URL list, not a desktop crawler. It is faster for programmatic workflows, pipe the output into a batch extraction or store it in a database without a CSV export step. Screaming Frog provides a richer GUI audit tool. The two serve different workflows.

Can I audit a JavaScript-rendered site?

Yes. The fetch cascade automatically escalates to Playwright stealth when a page requires JavaScript rendering. You can also force Playwright with "forcePlaywright": true. This handles single-page apps, lazy-loaded content, and client-rendered meta tags.

Start your first programmatic SEO audit

Run any endpoint live in the playground, or grab a free key. 1,000 credits a month, no card.