Custom Robots.txt Generator
Custom Robots.txt Generator
Create a perfectly optimized robots.txt file for your website
Website URL (without https://):
Sitemap URL (optional):
Search Engine Crawlers
Allow Googlebot
Allow Bingbot
Allow Slurp (Yahoo)
Allow Baidu
Allow Yandex
Crawl Settings
Add Crawl-Delay
Delay (seconds):
No Image Indexing
No Archive (Cached Pages)
Special Directives
Include Host Directive
Include Clean-Param
Allow URL Parameters
Disallowed Paths:
Add Path
Generate Robots.txt
# Your custom robots.txt file will appear here # Generated by Custom Robots.txt Generator
Copy to Clipboard
Robots.txt copied to clipboard!