Quick Presets
Rules
User-agent
Type
Path
Additional Options
Live Preview
0 lines
robots.txt
Place robots.txt in your site's root directory
Quick Reference
- User-agent: *
- All crawlers
- Disallow: /
- Block everything
- Allow: /
- Allow everything
- Crawl-delay: 10
- Wait 10s between requests
- Sitemap: URL
- Point to your sitemap