AeolyBot

AeolyBot fetches pages when somebody runs a site audit with Aeoly, an SEO tool. It reads the HTML, checks it against a set of technical rules, and reports what it found to the person who asked.

It sends this user agent on every request:

AeolyBot/0.2 (+https://aeoly.ai/bot)

It only ever reads. It does not submit forms, follow logout links, write, comment, or attempt to sign in anywhere.

Put this in the robots.txt at the root of your domain. It obeys it.

User-agent: AeolyBot
Disallow: /

If it is the rate rather than the visit that is the problem, slow it down instead. It honours Crawl-delay, which most crawlers ignore.

User-agent: AeolyBot
Crawl-delay: 5

Either takes effect on the next audit, because robots.txt is re-read at the start of every crawl rather than cached between them. If something is still arriving after that, it is not us, and section 5 is how to tell.

Somebody entered your domain into an audit. That is usually you or someone working on your site. It can also be somebody else, because the free scan does not verify ownership before it runs: asking for proof before showing anything would make a free tool useless, so the check sits in our terms rather than in the code.

That is a real trade and we would rather state it than imply a verification step we do not perform. The crawl is bounded, obeys your robots.txt, and reads only what a search engine reads. If you would rather not be crawled by us at all, section 2 settles it permanently and we will not work around it.

• Up to a few dozen pages per audit, not your whole site
• Public HTML only. It does not execute your JavaScript or load your images
• It ignores anything your robots.txt disallows, including for other crawlers
• One audit is one visit. It does not run on a schedule against a site it has been sent to once

What is stored is the page content it read and what the rules made of it, kept in the report for the person who ran the audit, and deleted when they delete the project. The privacy policy covers it.

A user agent is a string anyone can send, so seeing our name proves nothing. We do not publish an IP range list, which means we cannot ask you to take our word either.

If traffic calling itself AeolyBot is behaving in a way this page does not describe, taking pages faster than your Crawl-delay, ignoring a Disallow, submitting forms, it is very likely not us. Send the log lines to support@aeoly.ai and we will tell you truthfully whether it was, and fix it if it was.

Blocking a spoofed agent at your network edge is the right response to that traffic, and it does not depend on us agreeing.

We publish a directory of AI crawlers that judges vendors on exactly this: whether the token is documented, whether there is a way to verify it, whether blocking it is honoured.

For a while our own user agent pointed at this address and nothing was here, which made the criticism cheap. A site owner who looks up an unfamiliar bot and finds a 404 has learned something accurate about whoever sent it.