56 different robots | Hits |
Inktomi Slurp | 362031+43020 |
Voila | 22404+28970 |
Unknown robot (identified by 'crawl') | 43926+2257 |
LinkScan | 40743 |
Unknown robot (identified by hit on 'robots.txt') | 0+10303 |
Unknown robot (identified by 'robot') | 8403+333 |
MSIECrawler | 2929+440 |
Googlebot | 3323+20 |
Voyager | 2302+204 |
Alexa (IA Archiver) | 180+1645 |
WISENutbot | 1219+14 |
arks | 1157 |
GigaBot | 789+253 |
LinkWalker | 743+3 |
Digger | 697+6 |
AskJeeves | 435+260 |
Turn It In | 605+2 |
Unknown robot (identified by 'spider') | 426+39 |
EchO! | 380+2 |
The Python Robot | 298+4 |
BBot | 214 |
Scooter | 68+124 |
SurveyBot | 136+54 |
LinkChecker | 185 |
Ingrid | 112+62 |
Walhello appie | 85+79 |
Internet Shinchakubin | 145 |
Web Core / Roots | 120 |
larbin | 94+25 |
Combine System | 18+60 |
VWbot | 0+68 |
Nomad | 48 |
Fast-Webcrawler | 38 |
UdmSearch | 27+3 |
UCSD Crawl | 20 |
ht://Dig | 10+10 |
StackRambler | 8+11 |
Yandex bot | 14+5 |
Holmes | 1+8 |
Netcraft | 9 |
InternetSeer | 8 |
LinkBot | 7 |
Checkbot | 0+4 |
Ultraseek | 2+2 |
Bloodhound | 0+3 |
Pompos | 1+2 |
Pita (webmaster@pita.stanford.edu) | 3 |
Fluid Dynamics Search Engine robot | 3 |
Fish search | 3 |
CyberSpyder Link Test | 1+1 |
Atomz.com Search Robot | 0+1 |
Sift | 1 |
BSpider | 1 |
Grub.org | 1 |
Motor | 1 |
The World Wide Web Worm | 1 |