How it works
- 1Enter the domain, selector or public URL requested by the tool.
- 2Run the check. Toolardo validates the input before any network request.
- 3Review the returned DNS, HTTP or HTML details.
← All tools
SERVER · php
robots.txt Checker
Fetch and inspect a site’s robots.txt directives, user agents and sitemap declarations. Runs directly from Toolardo without a commercial lookup API.
✓ No commercial API key✓ Direct DNS / HTTP inspection✓ Private-network targets blocked where applicable
About this tool
Fetch and inspect a site’s robots.txt directives, user agents and sitemap declarations.
Frequently asked questions
Does this tool use a commercial API?
No. It uses server DNS functions or direct HTTP/HTTPS requests to the public target needed for the check.
Why can a valid check fail?
DNS can be temporarily unavailable, the target may block automated requests, shared hosting can restrict outbound networking, or the site may be offline.
Can this tool access localhost or private networks?
No. Tools that fetch websites block private, loopback, link-local and reserved IP ranges and restrict connections to standard web ports.