TangraBot

If you found this page from a user-agent string in your server logs, this is what that request was and why it was made.

The short version

TangraBot is not a crawler. It does not index, it does not follow links, and it does not collect content. It fetches one page — the homepage of a site whose owner asked us to check it — and reports back whether that page was served.

The user-agent

Requests from us look like this:

The last field names the AI crawler whose access is being checked, and it changes per request. We never claim to be that crawler. The agent string is our own identifier with the crawler's name inside it, so a substring rule in your bot protection — which is what those rules almost always are — behaves as it would for the real crawler, while the string itself never asserts anything untrue about who is asking.

We do this because the alternative is spoofing a header, and a tool that lies to your server to tell its own customer the truth is not one worth building. It costs us accuracy, and we say so in the results: a rule matching a user-agent exactly will not fire on a compound string, so a request we got through does not prove a crawler would.

What it actually does

• Fetches the site's homepage once as ourselves, as a control.
• Fetches the same page once more for each of the crawlers below.
• Sends a From header carrying the address below, so you can reply without following a link.
• Records the HTTP status and the response size. Nothing is stored from the page body itself.

Requests are sequential rather than parallel, with an eight-second timeout each — a burst of simultaneous requests from one address is itself the behaviour bot protection exists to stop, and we are not going to trip yours to test it. A single account can start at most four of these checks a minute.

The crawlers it names

OAI-SearchBot · ChatGPT-User · GPTBot · Claude-SearchBot · Claude-User · ClaudeBot · PerplexityBot · Perplexity-User · Google-Extended

Who triggers it

A person, on their own site, in our app. There is no scheduled crawl and no background job pointed at anybody. A request to your site means the operator of a Tangra account holding your domain pressed a button. If that is not you, and you would like it to stop, the section below is the reliable way to make it stop.

Blocking us

Block TangraBot in your bot rules and the requests stop. There is nothing to opt out of beyond that and nothing we hold that a block would leave behind.

It is worth knowing what a block does, though: the only thing it breaks is the check the site's own operator asked for. It has no effect on whether ChatGPT, Claude, Perplexity or Google can reach the site, and it does not remove anything from any assistant. It only means we can no longer tell your customer whether their site answers those crawlers — they will see “we could not read this” rather than an answer.

Contact

team@tangra.link. If a request from us caused a problem on your side, we would rather hear about it than not.