Robots.txt
RFC 9309 matching, blocked paths and AI crawler access for GPTBot, ClaudeBot and Google-Extended.
Crawl your whole site, score every page on five pillars and see exactly which fixes move the score most. The same crawler runs on every plan.
Meta & Head, Content, Social, Schema, Accessibility and Images, each from 0 to 100, plus an overall site score you can track scan after scan.
Issues get a P1 to P4 priority from their severity and how many pages they hit, so the first ticket is always the one worth doing first.
How priority worksThe files and settings that can break the whole domain at once.
RFC 9309 matching, blocked paths and AI crawler access for GPTBot, ClaudeBot and Google-Extended.
Finds your XML sitemaps and compares them with the pages the crawler reached.
www and http variants, redirect chains and temporary 302 or 307 redirects.
Missing return links and language codes that do not match the page.
The icon sizes Google shows in results, plus the web app manifest.
Every JSON-LD type on the site, checked against schema.org and your website type.
Schema markup checker →Every link to another website is checked once per scan, so dead outbound links show up.
HSTS, Content-Security-Policy and X-Frame-Options, plus new-tab links without noopener.
Every URL as indexable, non-indexable or blocked, with the reason and where redirects lead, plus /page and /page/ answering as two pages.
Finds your accessibility statement and checks it names a standard, a conformance status, known limitations and a contact.
One website, 100 pages per scan, Search Console included. Upgrade when you need more.
Every JSON-LD type, checked against schema.org.
Schema markup checker →Clicks, queries and URL Inspection, free.
Search Console dashboard →Google's safety lists and spam searches in your data.
Spam and hacked site check →