100% Free · Live Preview · Nothing Ever Sent to a Server
Robots.txt Generator
Control which crawlers — search engines and AI bots — can access your site, and which paths they should skip.
Robots.txt Generator
Frequently Asked Questions
Where do I put the generated file?
Upload it as robots.txt at the root of your domain (e.g. yoursite.com/robots.txt) — it won't work from a subfolder.
Does robots.txt guarantee a bot won't crawl a page?
No — it's a request that well-behaved crawlers (Google, Bing, and most reputable AI crawlers) respect, not an access control mechanism. Anything genuinely private needs real authentication, not just a Disallow rule.