|
|
| Session |
78 |
0% |
|
| WebList.ru robot |
72 |
0% |
|
| Ichiro (goo.ne.jp) |
71 |
0% |
|
| Unknown (caching agent) |
71 |
0% |
|
| Powermarks (bookmark manager) |
68 |
0% |
|
| 360Spider |
67 |
0% |
|
| GaisBot (gais.cs.ccu.edu.tw) |
66 |
0% |
|
| DiffBot (diffbot.com) |
65 |
0% |
|
| SimilarPages Nutch Crawler (similarpages.com) |
64 |
0% |
|
| Nutraspace Nutch Crawler (nutraspace.com) |
62 |
0% |
|
| Page Bites Hyper Bot (pagebites.com) |
61 |
0% |
|
| POE-Component-Client-HTTP |
58 |
0% |
|
| NutchCVS (apache.org) |
57 |
0% |
|
| DTS Agent (Beijing E-mail collector) |
56 |
0% |
|
| Discoverybot (discoveryengine.com) |
55 |
0% |
|
| Unknown E-mail collector |
54 |
0% |
|
| SpiderMan (yahoo.com) |
53 |
0% |
|
| NutchCVS (nutch.org) |
51 |
0% |
|
| WebCapture (downloading tool) |
51 |
0% |
|
| Ginger Crawler (gingersoftware.com) |
50 |
0% |
|
| Netcraft Web Server Survey |
50 |
0% |
|
| Factbot (factbites.com) |
49 |
0% |
|
| Heritrix () |
47 |
0% |
|
| SaladSpoon Search Engine crawler (shopsalad.com) |
47 |
0% |
|
| Voyager (lisa.co.jp) |
47 |
0% |
|
| New Sogou Spider |
45 |
0% |
|
| CCBot (commoncrawl.org) |
42 |
0% |
|
| FollowSite Bot (followsite.com) |
42 |
0% |
|
| Microsoft URL Control (spam bot) |
42 |
0% |
|
| Szukacz (szukacz.pl) |
42 |
0% |
|
| Fast MetaWeb Crawler (fastsearch.com) |
41 |
0% |
|
| Nu_tch-princeton crawler (cs.princeton.edu) |
41 |
0% |
|
| AipBot (aipbot.com) |
40 |
0% |
|
| cz32ts |
40 |
0% |
|
| Labome |
40 |
0% |
|
| Almaden Crawler (almaden.ibm.com) |
39 |
0% |
|
| Hailoobot (hailoo.com) |
38 |
0% |
|
| Baypup (baypup.com) |
37 |
0% |
|
| Exalead NG (aol.com) |
37 |
0% |
|
| Yanga World Search Bot (yanga.co.uk/) |
37 |
0% |
|
| Sapphire Web Nutch Crawler (boston.lti.cs.cmu.edu) |
36 |
0% |
|
| MSNBot Products (msn.com) |
35 |
0% |
|
| Net Research Server (loopimprovements.com) |
35 |
0% |
|
| Clj-httpc |
33 |
0% |
|
| ContextAd Bot |
33 |
0% |
|
| Nutch Crawler |
32 |
0% |
|
| Seznam screenshot-generator (fulltext.sblog.cz) |
32 |
0% |
|
| IS Alpha Nutch Crawler |
31 |
0% |
|
| HTMLParser |
30 |
0% |
|
| WBSearchBot (warebay.com) |
30 |
0% |
|
| Sproose bot (sproose.com) |
29 |
0% |
|
| Holmes / Onet Szukaj (szukaj.onet.pl) |
28 |
0% |
|
| Scooter (altavista.com) |
28 |
0% |
|
| WebCorp (webcorp.org.uk) |
28 |
0% |
|
| DeepTrawl |
27 |
0% |
|
| MSIE Crawler (URL check) |
27 |
0% |
|
| czxt2s |
26 |
0% |
|
| Feedfetcher-Google (google.com) |
26 |
0% |
|
| Jewelry City Spider (jewelrycity.org). |
26 |
0% |
|
| JoeDog |
26 |
0% |
|
| NEWT ActiveX (Web Collector) |
26 |
0% |
|
| Covario-IDS (covario.com) |
25 |
0% |
|
| Nutch Crawler (nutch.org) |
25 |
0% |
|
| SemrushBot |
25 |
0% |
|
| Bot-Seeker |
24 |
0% |
|
| CazoodleBot (cazoodle.com) |
24 |
0% |
|
| GenieBot (genieknows.com) |
24 |
0% |
|
| Lycos Spider (lycos.cs.cmu.edu) |
24 |
0% |
|
| YebolBot |
24 |
0% |
|
| Boitho.com Robot |
23 |
0% |
|
| HTTrack (download manager) |
23 |
0% |
|
| Naaya Link Checker |
23 |
0% |
|
| Crawler4j (google.com) |
22 |
0% |
|
| EStyle Search (e-Style ISP, Russia) |
22 |
0% |
|
| MQBot Nutch Crawler (cs.uiuc.edu) |
22 |
0% |
|
| OpenWebIndex Nutch Crawler |
22 |
0% |
|
| Spreadia |
22 |
0% |
|
| WebCollage Syndicator |
22 |
0% |
|
| Ccubee crawler (ccubee.net) |
21 |
0% |
|
| GoForItBot (goforit.com) |
21 |
0% |
|
| Ilial Nutch Crawler (ilial.com) |
21 |
0% |
|
| libiViaCore (ucr.edu) |
21 |
0% |
|
| MSR-ISRCCrawler |
21 |
0% |
|
| NetSeer Crawler (netseer.com) |
21 |
0% |
|
| NoxtrumBot (noxtrum.com) |
21 |
0% |
|
| Isidorus (isidorus.com) |
20 |
0% |
|
| KnowItAll Nutch Crawler (cs.washington.edu) |
20 |
0% |
|
| Suchmaschinenoptimierung.de |
20 |
0% |
|
| TL32Sn |
20 |
0% |
|
| WebAlta Crawler (webalta.net) |
20 |
0% |
|
| Aghaven Nutch Crawler (aghaven.com) |
19 |
0% |
|
| domino.research.imb.com Crawler |
19 |
0% |
|
| Fluid Dynamics Search Engine |
19 |
0% |
|
| Heritrix (archive.org) |
19 |
0% |
|
| RDBot Nutch Crawler |
19 |
0% |
|
| Ruby |
19 |
0% |
|
| Amagit.com robot |
18 |
0% |
|
| Chen Li Nutch Crawler (chenli.com.cn) |
18 |
0% |
|
| Heritrix (adsafemedia.com) |
18 |
0% |
|
| IRP.edu Nutch Crawler |
18 |
0% |
|
| Total |
1735101 |
|