Robots.txt Generator for WordPress
Generate a WordPress-ready robots.txt that blocks admin and includes and allows admin-ajax.php and your sitemap.
A sensible WordPress robots.txt that keeps /wp-admin/ out of the index while allowing admin-ajax.php and pointing at your sitemap.
Paths to block (one per line):
Run this tool
Free daily runs · Pro is unlimited.
Robots.txt for WordPress tips
- Disallow /wp-admin/ but Allow /wp-admin/admin-ajax.php
- Disallow /wp-includes/
- Always keep Sitemap: https://yourdomain.com/sitemap.xml
- Yoast/RankMath can already serve robots.txt — use this if you manage it manually.
How to use Robots.txt for WordPress
- Choose allow-all or block specific paths.
- Add your sitemap URL.
- Copy the robots.txt to your site root.
Frequently asked questions
Where does robots.txt go?
At the root of your domain, e.g. https://example.com/robots.txt
Does it prevent indexing?
robots.txt controls crawling, not indexing. Use a noindex meta tag to keep a page out of results.
Is it free?
Yes, free with no signup.
Automate this at scale
MarsGeo Pro adds bulk audit, programmatic SEO page generation and API access.
See plansRelated tools
Robots.txt Generator
Generate a valid robots.txt with allow/disallow rules, sitemap and crawl-delay. Copy and p…
Open →XML Sitemap Generator
Paste your URLs and generate a valid XML sitemap ready to submit to Google Search Console.…
Open →Canonical Tag Checker
Paste a page's HTML and instantly see its canonical tag, title, meta description, H1 and O…
Open →Meta Description Generator
Turn a paragraph into a clean, length-checked meta description with a live SERP preview.…
Open →SEO Title Generator
Generate title tag variants for a keyword and see how they look in Google with a pixel-wid…
Open →Open Graph Generator
Generate Open Graph and Twitter Card meta tags so your link previews look right on social …
Open →