48 different robots | Hits |
Inktomi Slurp | 365068+9535 |
Unknown robot (identified by 'crawl') | 239172+4247 |
Voila | 46315+585 |
Unknown robot (identified by 'robot') | 41788+1389 |
Unknown robot (identified by hit on 'robots.txt') | 0+26558 |
Yandex bot | 16900+656 |
Googlebot | 13481+20 |
LinkScan | 11202 |
arks | 3309 |
MSIECrawler | 2737+130 |
Turn It In | 2845+18 |
StackRambler | 2802+49 |
Voyager | 1036+53 |
Unknown robot (identified by 'spider') | 632+125 |
MSNBot | 705 |
larbin | 387+158 |
Alexa (IA Archiver) | 0+524 |
LinkWalker | 507+3 |
SurveyBot | 349+50 |
Digger | 325+2 |
The Python Robot | 265+22 |
Scooter | 45+125 |
EchO! | 150 |
Grub.org | 79+36 |
ht://Dig | 48+48 |
LinkChecker | 94 |
Motor | 90 |
BBot | 74+3 |
Holmes | 7+50 |
Web Core / Roots | 39+1 |
AskJeeves | 31 |
DNAbot | 13+13 |
Internet Shinchakubin | 26 |
Fish search | 16+1 |
Pita (webmaster@pita.stanford.edu) | 12 |
mnoGoSearch search engine software | 5+5 |
Netcraft | 10 |
Ingrid | 1+8 |
WISENutbot | 6 |
SpiderView(tm) | 4 |
Walhello appie | 2+2 |
Robozilla | 4 |
Combine System | 0+3 |
The World Wide Web Worm | 3 |
NetMechanic | 1+1 |
Muncher | 1 |
LinkBot | 1 |
Harvest | 1 |