Free browser tool
Robots.txt Tester
Use this robots.txt tester to check crawler access and URL paths locally in your browser.
Test a crawler path
Change the file, crawler, or path to recalculate the verdict.
Crawler decision
The result appears here as you type.
Frequently asked questions
Use this robots.txt tester to check crawler access and URL paths locally in your browser.
Frequently asked questions
How does robots.txt decide which rule applies?
The most specific matching user-agent group is selected, then the longest matching path wins. An Allow wins a tie with Disallow. This tool shows the deciding line.
Does Disallow remove a page from Google?
No. Disallow blocks crawling, not necessarily indexing. Use an appropriate noindex directive when a page must stay out of search results.
What do * and $ mean?
An asterisk matches any sequence of characters. A dollar sign anchors the pattern to the end of the URL, which is useful for file extensions.
Is my robots.txt uploaded anywhere?
No. The parser and rule evaluator run entirely in your browser.