75 different robots | Hits |
Googlebot | 136417+1695 |
Inktomi Slurp | 117519+16524 |
Unknown robot (identified by 'spider') | 116570+162 |
Unknown robot (identified by 'crawl') | 91933+1622 |
MSNBot | 73454+4252 |
Pita (webmaster@pita.stanford.edu) | 70110+13 |
AskJeeves | 30559+1722 |
LinkScan | 27415 |
WISENutbot | 19744+75 |
Unknown robot (identified by 'robot') | 11832+226 |
Grub.org | 10314+48 |
InternetSeer | 8193 |
Unknown robot (identified by hit on 'robots.txt') | 0+6171 |
gazz | 5198+18 |
MSIECrawler | 3608+1483 |
Turn It In | 4488+37 |
Alexa (IA Archiver) | 3475+845 |
Fast-Webcrawler | 3725+63 |
GigaBot | 2672+752 |
Voila | 1275+1922 |
psbot | 3045+129 |
Walhello appie | 964+847 |
ComputingSite Robi/1.0 | 1736+33 |
larbin | 1314+186 |
BaiDuSpider | 910+45 |
arks | 861+48 |
ZealBot | 806 |
Digger | 782+1 |
ht://Dig | 480+20 |
LinkWalker | 254+239 |
Yahoo Vertical Crawler | 358+31 |
Speedy Spider | 244+106 |
Pompos | 262+38 |
Ingrid | 236+15 |
LinkChecker | 229 |
Scooter | 152+65 |
Yandex bot | 152 |
InfoSeek Robot 1.0 | 78+73 |
Templeton | 111 |
SurveyBot | 52+49 |
Ultraseek | 73+26 |
UdmSearch | 65+9 |
StackRambler | 24+45 |
Combine System | 33+35 |
BBot | 62+2 |
Fluid Dynamics Search Engine robot | 59+1 |
Holmes | 4+49 |
The Python Robot | 49+2 |
Calif | 42 |
SpiderMan | 21+10 |
LinkBot | 30 |
Netcraft | 28 |
Fish search | 22 |
Mirago | 5+16 |
Hometown Spider Pro | 17 |
UCSD Crawl | 17 |
Openfind data gatherer | 9+7 |
mnoGoSearch search engine software | 10+4 |
ObjectsSearch | 5+3 |
Internet Shinchakubin | 8 |
DeWeb(c) Katalog/Index | 8 |
BSpider | 4+2 |
Lycos | 3+2 |
Magpie | 5 |
Robozilla | 4 |
Checkbot | 1+3 |
ParaSite | 4 |
PhpDig | 2+2 |
ASpider (Associative Spider) | 0+3 |
Muncher | 2 |
spider_monkey | 1+1 |
Perman surfer | 0+2 |
Nomad | 1 |
Harvest | 0+1 |
JoBo Java Web Robot | 0+1 |