ENTITY

Screaming Frog

Properties1
tagstool, crawler, technical-seo, site-audit

1 min read

Screaming Frog

Type: product (desktop SEO crawler) | Domain relevance: Site-crawling tool used to audit structure and feed gap analysis; a shared web-dev/SEO utility.

Summary

Screaming Frog is a desktop crawler (the SEO Spider) Caleb uses to export a site’s pages and internal links, which he then feeds to Claude for competitor and Core 30 gap analysis. He notes it is free for up to 500 URLs, and if a site blocks the default spider you can switch to list mode and supply URLs from the site’s sitemap.xml. It sits on both pillars — a technical crawling/QA tool for developers and an SEO auditing instrument.

Key Facts

  • Desktop crawler; free ≤ 500 URLs.
  • Exports internal_all and links reports for analysis.
  • List mode + sitemap.xml workaround when spidering is blocked.
  • Output feeds AI-driven gap analysis for the Core 30.

In the Sources

SourceContext
OWN Your City in a MONTH with This SEO GuideCrawls a competitor site; exports fed to Claude for gap analysis.

Entities: GMB Everywhere, Claude, Core30 AI Tool Concepts: Core 30, GBP–Website Alignment (Consistency Signals)