Six free tools, and all of them actually run

Type a domain and see whether ChatGPT, Claude, Gemini, Perplexity and Copilot are allowed to read it. It checks all 20 tokens against your robots.txt and shows the exact directive that decided each one.

• Separates the crawlers that decide citations from the ones that only affect training
• Names the line to edit, not just the verdict
• Reads robots.txt once, honestly, and says what it cannot see

Check whether a URL is allowed for any crawler, and see the exact line that decided. Then edit the file and watch the verdicts change. That is the half Google removed when it retired the Search Console tester.

• Longest-match-wins, wildcards and query strings, the way crawlers implement them
• Says which User-agent group decided, not just the verdict
• The same parser this product's own crawler obeys

Pick a position on AI crawlers and get the exact robots.txt groups. The one that matters is opting out of model training without also removing yourself from the answers, which is what most copy-paste snippets do by accident.

• Three presets, or set every documented token by hand
• Tells you what each choice costs before you paste it
• Refuses to put Googlebot in a block-AI preset, because that is a different decision

Point it at your site. It crawls up to 40 pages, runs 42 technical checks, and gives you a score with every finding that pulled it down. The report stays at a private link that does not expire.

• Reads your pages, never writes to them, and obeys robots.txt
• Ranks findings by what they are costing rather than by severity label
• Results in about a minute

Builds an llms.txt from the sitemap you already publish: your pages, grouped into sections, in the conventional structure. The one-line descriptions are left blank on purpose.

• Reads your sitemap, never crawls your pages
• Says plainly that llms.txt is a proposal and no assistant has committed to it
• Leaves you the two lines nothing else can supply

Type a title and a meta description and see where Google would cut them off. It measures pixel width rather than counting characters, which is what Google actually does and why a title in capitals runs out sooner.

• Nothing is sent anywhere: the whole thing runs on your machine
• Uses the same limits the audit's title and description checks use
• Shows the cut, not just a number