Free Robots.txt
Checker
Catch the misconfigurations that block Googlebot from your site. Check Disallow rules, sitemap directives, and crawl directives on any domain in seconds.
Frequently asked questions
What does robots.txt do?
robots.txt is a file at the root of your domain (https://example.com/robots.txt) that tells search engine crawlers which URLs they're allowed to fetch. A misconfigured robots.txt can accidentally block Google from indexing your entire site, killing your SEO traffic. This checker shows you exactly what's in your robots.txt and flags common mistakes.
Is this robots.txt checker free?
Yes — 100% free, no signup, no credit card. Check unlimited URLs.
What does the checker flag?
Critical issues like Disallow: / (which blocks an
entire site from a user-agent), missing or empty robots.txt files,
missing Sitemap: directives, and broken sitemap URLs.
It also shows you parsed rules per user-agent so you can see exactly
what Googlebot, Bingbot, and other crawlers will see.
Did Google remove their official robots.txt tester?
Yes — Google retired the classic robots.txt Tester in Search Console in 2023. Newer GSC tools require domain verification. This checker works without any setup — paste any URL and get an instant report.
What's the difference between this and the full MetricVault platform?
This page is one tool. The full MetricVault AI dashboard has 30+ tools including AI Visibility tracking, Domain Overview, Backlink Profile, Keyword Magic, and more. You can sign up free for 9 more free SEO tools plus 5 premium reports a month — no credit card.