How to get cited by Copilot
Copilot answers from the Bing index, which makes an old crawler the gatekeeper and two meta tags the controls. Almost nobody is blocking it.
Read itShort pieces about the things this product has an opinion about, written the way it works: a claim, the evidence for it, and what to do on Monday.
Copilot answers from the Bing index, which makes an old crawler the gatekeeper and two meta tags the controls. Almost nobody is blocking it.
Read itEvery guide names a DeepSeek user agent. We went looking for the documentation behind it and could not find any. Here is what you can actually verify.
Read itPerplexity runs one crawler that respects robots.txt and one fetcher that says it generally ignores it. Knowing which is which changes what you can control.
Read itGemini and AI Overviews are different surfaces with different controls, and Google-Extended only governs one of them. Google also says neither needs special optimisation.
Read itAnthropic runs three crawlers and honours robots.txt for all three. It also routes Claude's web search through somebody else's index, which changes where the work is.
Read itTwo OpenAI crawlers, two completely different jobs. Block the wrong one and you vanish from ChatGPT search while still feeding the training set.
Read itThe free scan reads your real pages and tells you which of these problems you actually have. About a minute, no account.