About SiteHealthBot,and how to allow or block it.
SiteHealthBot is the crawler that runs health checks on websites people monitor with SiteHealth.
How to identify it
- User agent
SiteHealthBot/1.0 (+https://health.sitestats.dev/bot) - IPv4
31.97.43.170 - IPv6
2a02:4780:2d:d1d::1
How it behaves
- At most 1 request per second per website.
- Honors
robots.txtfor crawl-type checks. - Gives up after 15 seconds.
- Only submits forms or probes for exposed files on sites whose owner has verified ownership.
How to block or allow it
To block: add User-agent: SiteHealthBot followed by Disallow: / to your robots.txt. To allow only us on a firewall, allow the two IP addresses above.