For website owners
PressProveBot visited your site. Here is why, and how to stop it.
PressProve checks whether a press release it was asked about is on a web page, whether the page links to the customer’s site, and whether it carries a paid label.
What it does
A customer gives us the address of a page. We send one request for that page and one for your robots.txt, slowly. We read what the page shows and write down what we saw, with the time.
- One page
- A check through our API fetches each page it was given once, plus your robots.txt, and follows at most five redirects. It does not crawl the rest of your site. Our free check on this site can read a few more pages of the same site, such as a section page, and obeys robots.txt the same way.
- No cookies, no login
- We keep no cookies and send none. We never log in. We do not run your page’s scripts.
- Slow, and shared
- We wait between requests to one site (two seconds by default, longer if your robots.txt asks). There is one limit per site, shared by all our customers, so many customers asking about the same site do not add up.
- Public addresses only
- We never connect to private or internal addresses, and we do not follow a redirect to one.
- What we keep
- Checks made through the API keep no copy of your page. We keep the result, the phrases of the customer’s text we used to recognize it, a short piece of the page around a paid label or the customer’s link, and the link attributes. Our free check also keeps a short excerpt of the page, up to 12,000 characters, for comparison.
How we introduce ourselves
Every request carries this name and a link to this page:
PressProveBot/1.0 (+https://pressprove.com/bot; page checks for PressProve customers; respects robots.txt)
The name to use in robots.txt is PressProveBot. An older name, VerifiedDeliveryBot, is still obeyed, so a rule you wrote for it keeps working. If both are in your file, we obey both.
To stop us
Add this to your robots.txt:
User-agent: PressProveBot Disallow: /
We read your robots.txt before we visit a site. If it is missing (the server says 404 or 410) there are no rules. If we cannot read it, because the server errors, refuses us or does not answer, we do not visit, and the answer to our customer is “could not check”, never “not there”.
Or write to us and we will add your site to our do-not-visit list: contact@pressprove.com
To slow us
Add a Crawl-delay line. We follow it up to 30 seconds:
User-agent: PressProveBot Crawl-delay: 10
Something wrong?
Tell us the address that was visited and roughly when. We answer within one business day, and we will stop visits to your site on request.
Our reports describe what a page showed on a date. If a report about your site is wrong, tell us and we will correct it.
See also: Privacy policy, Security, Platform terms, Legal notice.