Robots.txt Generator
Generate a robots.txt file to control search engine crawling. — free, fast and private.
How to use Robots.txt Generator
- Enter the user-agent (use * to apply the rules to all search engines).
- Add the paths to allow and disallow, one per line.
- Optionally add a crawl-delay and your sitemap URL, then click Generate robots.txt.
- Download the file as robots.txt and upload it to your website's root folder.
Note: this tool runs in your browser. Nothing is uploaded.
Frequently Asked Questions
What is a robots.txt file?
It is a small text file at the root of a website that tells search engine crawlers which pages they may and may not visit.
Does robots.txt hide pages from Google?
Not completely — it only asks well-behaved crawlers not to visit. For truly private pages, use password protection or a noindex tag as well.
Is my data uploaded to a server?
No. This tool runs entirely in your browser and your input is never uploaded to a server.
Can I use this tool on my phone?
Yes, the tool is fully mobile-friendly and works in any modern browser.