Why does Steve Bot appear in my server logs?
Shareaholic powers content distribution, discovery and sharing of web content for over 450 million people each month across over 300,000+ websites. Part of how this works involves the temporary display of images or details related to the web content, such as the title, thumbnail and excerpt for the web page. Steve Bot is a web bot that fetches the HTML of a web page to gather, cache and display this information.
Steve Bot's process begins with a list of URLs, augmented with Sitemap data, both of which are provided by the site. New pages, changes to existing pages, and dead links are noted and used to update the data cached in the Shareaholic Content Index for a given site.
How does Steve Bot identify itself?
Mozilla/5.0 (compatible; Shareaholic/1.0; +http://www.shareaholic.com/steve)
What is the IP address used by Steve Bot?
How to verify Steve Bot?
You can verify if a web bot accessing your server really is Steve Bot. This is useful if you are concerned that troublemakers are accessing your site while claiming to be Steve Bot.
To verify Steve Bot as the caller:
- Use the host command to run a reverse DNS lookup on the IP address from your logs
- Verify that the domain name in the response ends with shareaholic.com
- Use the host command to run a forward DNS lookup on the response retrieved from step 1
- Verify that it is the same as the IP address from step 1
$ host 184.108.40.206 220.127.116.11.in-addr.arpa domain name pointer steve-197-16-208-54.shareaholic.com. $ host steve-197-16-208-54.shareaholic.com steve-197-16-208-54.shareaholic.com has address 18.104.22.168
If you receive a consistent volume of traffic from a client sending a valid Shareaholic Bot User-Agent but it does not pass the above DNS test, please open a support ticket.
How does Steve Bot access sites?
We do very aggressive caching so you shouldn't see Steve Bot in your logs more than once every few weeks for any one URL. Our goal is to index relevant information from as many web pages from your site as we can on each visit without overwhelming your server's bandwidth. We do not like waste either, and are clever at not wasting bandwidth – yours or ours. However, if you experience technical issues, please contact us. We recommend attaching any relevant server logs that may help with troubleshooting an issue.
Can I change Steve Bot's index rate?
Shareaholic has sophisticated algorithms that determine how much to visit each site. If Steve Bot is indexing content too often and slowing down your server, you can control the index rate for your domain in your Shareaholic site settings. The use of this setting is not always needed nor is it generally recommended, but it is available for use should the need arise.