Robots.txt Generator for Drupal
Generate a Drupal robots.txt that blocks admin and search URLs.
Block Drupal admin, user and search paths while leaving content crawlable.
Paths to block (one per line):
Run this tool
Free daily runs · Pro is unlimited.
Robots.txt for Drupal tips
- Disallow /admin/ and /user/
- Disallow /search/
- Block /core/ and /modules/
- Reference /sitemap.xml
How to use Robots.txt for Drupal
- Choose allow-all or block specific paths.
- Add your sitemap URL.
- Copy the robots.txt to your site root.
Frequently asked questions
Where does robots.txt go?
At the root of your domain, e.g. https://example.com/robots.txt
Does it prevent indexing?
robots.txt controls crawling, not indexing. Use a noindex meta tag to keep a page out of results.
Is it free?
Yes, free with no signup.
Automate this at scale
MarsGeo Pro adds bulk audit, programmatic SEO page generation and API access.
See plansRelated tools
Robots.txt Generator
Generate a valid robots.txt with allow/disallow rules, sitemap and crawl-delay. Copy and p…
Open →XML Sitemap Generator
Paste your URLs and generate a valid XML sitemap ready to submit to Google Search Console.…
Open →Canonical Tag Checker
Paste a page's HTML and instantly see its canonical tag, title, meta description, H1 and O…
Open →Meta Description Generator
Turn a paragraph into a clean, length-checked meta description with a live SERP preview.…
Open →SEO Title Generator
Generate title tag variants for a keyword and see how they look in Google with a pixel-wid…
Open →Open Graph Generator
Generate Open Graph and Twitter Card meta tags so your link previews look right on social …
Open →