PlainReckonerBot

If you found PlainReckonerBot in your access log, this page is what that is. It is run by this site. It fetches three public files, once a day, and nothing else.

What it asks for

• /robots.txt — the file that says which automated visitors may read which parts of your site. We read it to find out whether we may look at your homepage at all.
• /ads.txt — the file naming the companies allowed to sell ad space on your site
• Your homepage, only if your robots.txt permits it

That is the whole list. It does not follow links, request images, scripts or stylesheets, look for anything behind a login, or visit any page other than your root.

How often, and how gently

• Once a day at most, and usually less.
• One request at a time, with a pause between each. Never in parallel.
• If we fail to reach you four times in a row, we stop asking.
• If your robots.txt sets a Crawl-delay, we wait that long. If it sets one longer than thirty seconds, we skip your site for the day rather than pretend the number was for somebody else.
• If your server answers with the code that means "too many requests" — 429 — we back off. That is the answer Google tells site owners to use when they want a crawler to slow down, so we treat it as the instruction it is rather than as an error to push past.
• If your robots.txt cannot be read at all — your server errors, times out, or refuses the connection — we fetch nothing else from you that day. The standard that governs these files, RFC 9309, says at §2.3.1.4 that a crawler in that position MUST assume it is not allowed to fetch anything, and a server having a bad minute is not permission.

That last one is the rule most crawlers quietly skip, because skipping it is invisible and costs them nothing. We publish findings about how other people's files are configured; being casual about a requirement in the document we cite would not survive the first person who checked.

Why

These files change without telling their owner. A plugin update rewrites a robots.txt and blocks the crawler that was sending you readers. An ads.txt line breaks and bids quietly stop. A network restructures and your seller entry disappears. None of that shows up in Search Console, in your analytics, or in your ad network's dashboard.

Watching the public files day to day is how we tell publishers what changed, and how we measure what is happening to this market as a whole. Everything we find is published free, with its method and its date.

What we keep

Your robots.txt and ads.txt are kept as they were served, because in those two cases the file is the finding — a claim about what your ads.txt said is only checkable if we still have what it said.

Your homepage is not kept. It is fetched, read for one thing — which ad network is running — and discarded. What is stored is that answer and the handful of hostnames that produced it, which is about a hundred bytes instead of two hundred kilobytes. We have one question for your homepage; keeping the whole page after answering it would be collecting something we have no use for, and this site does not do that.

Nothing about your readers is stored, because nothing about your readers is ever fetched. There is no login, no form, no page below the root.

How to stop it

Two ways, and either is enough.

Block it in your robots.txt. We resolve your file before asking for your homepage, and we honour it:

User-agent: PlainReckonerBot
Disallow: /

That stops us reading your pages. We do still read robots.txt and ads.txt, because both exist specifically to be read by automated clients — an ads.txt that no crawler may fetch authorises nobody. If you would rather we did not read those either, use the second way.

Email us and we will remove you entirely. contact@plainreckoner.com — say which domain, and that is all we need. It is a manual removal that nothing automated can undo, and we do not ask you to justify it.

Where the list came from

Ad networks each publish a file, sellers.json, naming every publisher they are allowed to sell for. Those files are public, and the networks are required to publish them. Our list of domains is built from those files, which means every site we look at has already been published as part of this market by its own ad network.

About this site