Robots.txt Tester

Test and validate robots.txt rules for a specific user-agent and URL path. See whether crawlers are allowed or blocked.

Test Robots Rules

Reset

How This Robots.txt Tester Works

The tester checks matching user-agent groups, compares Allow and Disallow rules against your URL path, then applies the longest matching rule. If Allow and Disallow have equal match length, Allow wins.

  • Supports wildcard * patterns.
  • Supports end marker $.
  • Tests full URLs or relative paths.
  • Shows the best matching rule and all matched rules.