|
|
| Unknown! |
498341 |
21.76% |
|
| Spider(?), faked user-agent |
496406 |
21.68% |
|
| Yahoo! Slurp (yahoo.com) |
271897 |
11.87% |
|
| GoogleBot (google.com) |
204110 |
8.91% |
|
| BingBot (bing.com) |
202314 |
8.83% |
|
| BaiDu Spider (baidu.com) |
186765 |
8.16% |
|
| MSNBot (msn.com) |
147460 |
6.44% |
|
| Unknown |
135005 |
5.9% |
|
| Yandex.ru robot |
20114 |
0.88% |
|
| MSNBot Media (msn.com) |
14449 |
0.63% |
|
| Sogou web spider (sogou.com) |
13994 |
0.61% |
|
| DotBot (dotnetdotcom.org) |
10132 |
0.44% |
|
| Spider-on-the-fly (discoverlife.org) |
10117 |
0.44% |
|
| Facebookexternalhit (facebook.com) |
7698 |
0.34% |
|
| ExaBot (exava.com) |
6267 |
0.27% |
|
| Synapse |
6121 |
0.27% |
|
| MJ12bot (majestic12.co.uk) |
4503 |
0.2% |
|
| Twiceler (cuill.com) |
4200 |
0.18% |
|
| Ask Jeeves/Teoma (ask.com) |
3559 |
0.16% |
|
| Stack Rambler (rambler.ru) |
3108 |
0.14% |
|
| Speedy Spider (entireweb.com) |
2905 |
0.13% |
|
| Yahoo! Slurp China (yahoo.com.cn) |
2425 |
0.11% |
|
| Yeti (naver.com) |
2180 |
0.1% |
|
| 80legs.com crawler |
2076 |
0.09% |
|
| IA Archiver (alexa.com) |
1955 |
0.09% |
|
| LWP Simple |
1870 |
0.08% |
|
| Java VM |
1765 |
0.08% |
|
| Charlotte (searchme.com) |
1572 |
0.07% |
|
| Fast Enterprise Crawler (fast.no) |
1209 |
0.05% |
|
| DiscoBot (discoveryengine.com) |
1017 |
0.04% |
|
| NV32ts |
894 |
0.04% |
|
| DoCoMo (docomo.ne.jp) |
859 |
0.04% |
|
| Libwww-perl |
788 |
0.03% |
|
| Sosospider (soso.com) |
778 |
0.03% |
|
| Voyager (kosmix.com) |
715 |
0.03% |
|
| Google Translate (google.com) |
673 |
0.03% |
|
| Jakarta Commons-HttpClient |
588 |
0.03% |
|
| ZyBorg (wisenut.com) |
582 |
0.03% |
|
| Aotech.net E-mail collector |
514 |
0.02% |
|
| Findlinks (wortschatz.uni-leipzig.de) |
475 |
0.02% |
|
| ODP link checker (pmoz.info) |
465 |
0.02% |
|
| AhrefsBot (ahrefs.com) |
448 |
0.02% |
|
| Python-urllib fetcher (Naver Japan / Korea) |
439 |
0.02% |
|
| Ezooms Bot |
398 |
0.02% |
|
| VoilaBot (voila.fr) |
376 |
0.02% |
|
| Integrated Computer Solutions (ics.com) |
368 |
0.02% |
|
| BecomeBot (become.com) |
352 |
0.02% |
|
| BOTW Spider (botw.org) |
352 |
0.02% |
|
| Envolk spider (envolk.com) |
339 |
0.01% |
|
| Indy Library (E-mail collector) |
329 |
0.01% |
|
| Blekkobot (blekko.com) |
325 |
0.01% |
|
| DoCoMo (goo.ne.jp) |
314 |
0.01% |
|
| Search17Bot (search17.com) |
278 |
0.01% |
|
| Goldfire Server |
264 |
0.01% |
|
| MojeekBot (mojeek.com) |
255 |
0.01% |
|
| ScoutJet (scoutjet.com) |
253 |
0.01% |
|
| JyxoBot (jyxo.cz) |
246 |
0.01% |
|
| GoogleBot-Mobile (google.com) |
245 |
0.01% |
|
| Kalooga Bot (kalooga.com) |
245 |
0.01% |
|
| Bender (benderthewebrobot.tumblr.com) |
244 |
0.01% |
|
| Fast Web Crawler (fast.no) |
232 |
0.01% |
|
| LinkBot Pro (link checking) |
229 |
0.01% |
|
| Heritrix (accelobot.com) |
219 |
0.01% |
|
| WGet (downloading tool) |
219 |
0.01% |
|
| LibraBot (microsoft.com) |
210 |
0.01% |
|
| Yahoo Seeker (yahoo.com) |
209 |
0.01% |
|
| WordPress (wordpress.com) |
195 |
0.01% |
|
| YacyBot (yacy.net) |
193 |
0.01% |
|
| GigaBot (gigablast.com) |
192 |
0.01% |
|
| www.socialayer.com Agent |
183 |
0.01% |
|
| StumbleUpon (stumbleupon.com) |
180 |
0.01% |
|
| Xenu Link Sleuth (link cheching) |
176 |
0.01% |
|
| TencentTraveler |
175 |
0.01% |
|
| MLBot (metadatalabs.com) |
170 |
0.01% |
|
| GrubNG (grub.org) |
151 |
0.01% |
|
| RetrevoPageAnalyzer (retrevo.com) |
150 |
0.01% |
|
| Baidu Transcoder (baidu.com) |
138 |
0.01% |
|
| eduCrawler |
131 |
0.01% |
|
| YoudaoBot (youdao.com) |
127 |
0.01% |
|
| SnapBot (snap.com) |
119 |
0.01% |
|
| Archive.org Bot |
113 |
0% |
|
| http_load 29jun2005 |
113 |
0% |
|
| SPBot (seoprofiler.com) |
112 |
0% |
|
| Lynx (file transferring tool) |
111 |
0% |
|
| Atrax Solutions atraxbot (atraxsolutions.com) |
106 |
0% |
|
| NaverBot (naver.com) |
104 |
0% |
|
| Win Http Services (microsoft.com) |
103 |
0% |
|
| Raymond Balmès/Biible (bible.info) |
99 |
0% |
|
| Curl (file transferring tool) |
96 |
0% |
|
| TurnitinBot (turnitin.com) |
89 |
0% |
|
| Grub-client (grub.org) |
88 |
0% |
|
| NiceBot |
88 |
0% |
|
| PSBot (picsearch.com) |
88 |
0% |
|
| IRLbot (irl.cs.tamu.edu) |
87 |
0% |
|
| Ichiro (help.goo.ne.jp) |
85 |
0% |
|
| Convera Crawler (authoritativeweb.com) |
83 |
0% |
|
| DataFountains/DMOZ (ucr.edu) |
82 |
0% |
|
| JikeSpider (shoulu.jike.com) |
82 |
0% |
|
| CShopper Nutch Crawler |
81 |
0% |
|
| Pittsburgh Nutch Crawler |
81 |
0% |
|
| Total |
2289937 |
|