The bot behind the free check

How to recognise it

It identifies itself as FindwickBot/1.0 (+https://findwick.com/bot).

What it does

• Reads robots.txt first, and follows it.
• Reads public pages only: never logins, accounts, baskets or checkouts.
• Stops after at most 50 pages, fetches no more than 4 at a time, and takes no longer than about a minute.
• Slows down, one page at a time, when a site asks it to.
• Runs only when someone asks for a check, never on a schedule.

When it reads another firm’s page

A check can show who comes up above a site on Google, and why. To say why, FindwickBot may read the one page Google shows for each of the top three results: that page only, never the rest of the site. It follows robots.txt and our opt-out list here too, asks each site for one thing at a time, and keeps what it read for 7 days so it doesn’t ask again.

If your site’s security blocks it

Some hosting firewalls turn automated checkers away, so the check fails even though your site works for visitors. To let FindwickBot check your site, ask whoever looks after your website, or your host, to look at the firewall’s log for FindwickBot and allow it. Some firewalls judge a visitor by how it connects rather than by its name, so allowing the name alone may not be enough. FindwickBot only reads public pages.

How to stop it

Add these lines to your robots.txt, and FindwickBot will stop at the door:

User-agent: FindwickBot
Disallow: /

Or ask us to stop

Tell us your website and an email address at that website, and we’ll email it a link to confirm. Once you press the button on the page it opens, we’ll refuse every free check of that website. We ask for an address at the website so we know the request comes from the people who run it.

This covers the one website address you give, with or without “www.”. For a part of your site with an address of its own, or if your email is somewhere else, such as Gmail, send us a message from the contact page and we’ll do it by hand.