37 different robots | Hits |
Unknown robot (identified by 'crawl') | 45386+545 |
legs | 43251+516 |
Inktomi Slurp | 30156+1067 |
Voila | 16897+1047 |
Unknown robot (identified by hit on 'robots.txt') | 0+15929 |
arks | 6673 |
Googlebot | 6275+9 |
LinkScan | 3693 |
Turn It In | 2825+14 |
Unknown robot (identified by 'robot') | 1431+537 |
Unknown robot (identified by 'spider') | 854+124 |
Motor | 542 |
BBot | 499+22 |
Phantom | 395 |
The Python Robot | 154+100 |
SurveyBot | 216+31 |
MSIECrawler | 197+6 |
Alexa (IA Archiver) | 31+110 |
Harvest | 71 |
larbin | 44+17 |
AskJeeves | 17+17 |
Yandex bot | 16+2 |
JBot Java Web Robot | 11 |
EchO! | 11 |
Raven Search | 11 |
ExactSeek Crawler | 3+3 |
Robozilla | 6 |
WallPaper (alias crawlpaper) | 5 |
Web Core / Roots | 5 |
mnoGoSearch search engine software | 4 |
churl | 4 |
InfoSeek Robot 1.0 | 3 |
MSNBot | 0+2 |
RoadHouse Crawling System | 1 |
Walhello appie | 1 |
Internet Shinchakubin | 1 |
Netcraft | 1 |