|
Unknown! |
498341 |
21.76% |
|
Spider(?), faked user-agent |
496405 |
21.68% |
|
Yahoo! Slurp (yahoo.com) |
271897 |
11.87% |
|
GoogleBot (google.com) |
204110 |
8.91% |
|
BingBot (bing.com) |
202314 |
8.83% |
|
BaiDu Spider (baidu.com) |
186765 |
8.16% |
|
MSNBot (msn.com) |
147460 |
6.44% |
|
Unknown |
135005 |
5.9% |
|
Yandex.ru robot |
20114 |
0.88% |
|
MSNBot Media (msn.com) |
14449 |
0.63% |
|
Sogou web spider (sogou.com) |
13994 |
0.61% |
|
DotBot (dotnetdotcom.org) |
10132 |
0.44% |
|
Spider-on-the-fly (discoverlife.org) |
10117 |
0.44% |
|
Facebookexternalhit (facebook.com) |
7698 |
0.34% |
|
ExaBot (exava.com) |
6267 |
0.27% |
|
Synapse |
6121 |
0.27% |
|
MJ12bot (majestic12.co.uk) |
4503 |
0.2% |
|
Twiceler (cuill.com) |
4200 |
0.18% |
|
Ask Jeeves/Teoma (ask.com) |
3559 |
0.16% |
|
Stack Rambler (rambler.ru) |
3108 |
0.14% |
|
Speedy Spider (entireweb.com) |
2905 |
0.13% |
|
Yahoo! Slurp China (yahoo.com.cn) |
2425 |
0.11% |
|
Yeti (naver.com) |
2180 |
0.1% |
|
80legs.com crawler |
2076 |
0.09% |
|
IA Archiver (alexa.com) |
1955 |
0.09% |
|
LWP Simple |
1870 |
0.08% |
|
Java VM |
1765 |
0.08% |
|
Charlotte (searchme.com) |
1572 |
0.07% |
|
Fast Enterprise Crawler (fast.no) |
1209 |
0.05% |
|
DiscoBot (discoveryengine.com) |
1017 |
0.04% |
|
NV32ts |
894 |
0.04% |
|
DoCoMo (docomo.ne.jp) |
859 |
0.04% |
|
Libwww-perl |
788 |
0.03% |
|
Sosospider (soso.com) |
778 |
0.03% |
|
Voyager (kosmix.com) |
715 |
0.03% |
|
Google Translate (google.com) |
673 |
0.03% |
|
Jakarta Commons-HttpClient |
588 |
0.03% |
|
ZyBorg (wisenut.com) |
582 |
0.03% |
|
Aotech.net E-mail collector |
514 |
0.02% |
|
Findlinks (wortschatz.uni-leipzig.de) |
475 |
0.02% |
|
ODP link checker (pmoz.info) |
465 |
0.02% |
|
AhrefsBot (ahrefs.com) |
448 |
0.02% |
|
Python-urllib fetcher (Naver Japan / Korea) |
439 |
0.02% |
|
Ezooms Bot |
398 |
0.02% |
|
VoilaBot (voila.fr) |
376 |
0.02% |
|
Integrated Computer Solutions (ics.com) |
368 |
0.02% |
|
BecomeBot (become.com) |
352 |
0.02% |
|
BOTW Spider (botw.org) |
352 |
0.02% |
|
Envolk spider (envolk.com) |
339 |
0.01% |
|
Indy Library (E-mail collector) |
329 |
0.01% |
|
Blekkobot (blekko.com) |
325 |
0.01% |
|
DoCoMo (goo.ne.jp) |
314 |
0.01% |
|
Search17Bot (search17.com) |
278 |
0.01% |
|
Goldfire Server |
264 |
0.01% |
|
MojeekBot (mojeek.com) |
255 |
0.01% |
|
ScoutJet (scoutjet.com) |
253 |
0.01% |
|
JyxoBot (jyxo.cz) |
246 |
0.01% |
|
GoogleBot-Mobile (google.com) |
245 |
0.01% |
|
Kalooga Bot (kalooga.com) |
245 |
0.01% |
|
Bender (benderthewebrobot.tumblr.com) |
244 |
0.01% |
|
Fast Web Crawler (fast.no) |
232 |
0.01% |
|
LinkBot Pro (link checking) |
229 |
0.01% |
|
Heritrix (accelobot.com) |
219 |
0.01% |
|
WGet (downloading tool) |
219 |
0.01% |
|
LibraBot (microsoft.com) |
210 |
0.01% |
|
Yahoo Seeker (yahoo.com) |
209 |
0.01% |
|
WordPress (wordpress.com) |
195 |
0.01% |
|
YacyBot (yacy.net) |
193 |
0.01% |
|
GigaBot (gigablast.com) |
192 |
0.01% |
|
www.socialayer.com Agent |
183 |
0.01% |
|
StumbleUpon (stumbleupon.com) |
180 |
0.01% |
|
Xenu Link Sleuth (link cheching) |
176 |
0.01% |
|
TencentTraveler |
175 |
0.01% |
|
MLBot (metadatalabs.com) |
170 |
0.01% |
|
GrubNG (grub.org) |
151 |
0.01% |
|
RetrevoPageAnalyzer (retrevo.com) |
150 |
0.01% |
|
Baidu Transcoder (baidu.com) |
138 |
0.01% |
|
eduCrawler |
131 |
0.01% |
|
YoudaoBot (youdao.com) |
127 |
0.01% |
|
SnapBot (snap.com) |
119 |
0.01% |
|
Archive.org Bot |
113 |
0% |
|
http_load 29jun2005 |
113 |
0% |
|
SPBot (seoprofiler.com) |
112 |
0% |
|
Lynx (file transferring tool) |
111 |
0% |
|
Atrax Solutions atraxbot (atraxsolutions.com) |
106 |
0% |
|
NaverBot (naver.com) |
104 |
0% |
|
Win Http Services (microsoft.com) |
103 |
0% |
|
Raymond Balmès/Biible (bible.info) |
99 |
0% |
|
Curl (file transferring tool) |
96 |
0% |
|
TurnitinBot (turnitin.com) |
89 |
0% |
|
Grub-client (grub.org) |
88 |
0% |
|
NiceBot |
88 |
0% |
|
PSBot (picsearch.com) |
88 |
0% |
|
IRLbot (irl.cs.tamu.edu) |
87 |
0% |
|
Ichiro (help.goo.ne.jp) |
85 |
0% |
|
Convera Crawler (authoritativeweb.com) |
83 |
0% |
|
DataFountains/DMOZ (ucr.edu) |
82 |
0% |
|
JikeSpider (shoulu.jike.com) |
82 |
0% |
|
CShopper Nutch Crawler |
81 |
0% |
|
Pittsburgh Nutch Crawler |
81 |
0% |
|
Total |
2289936 |
|