list of: Spiderbots in ASC order (not db order) | ||
No. | Name | User Agent |
1 | 360Spider | 360Spider |
2 | meanpathbot | |
3 | Applebot | |
4 | facebookexternalhit | |
5 | Googlebot | |
6 | ||
7 | X | Twitterbot |
8 | Jetmon | |
9 | slurp | |
10 | YandexBot | |
11 | AddThis | AddThis |
12 | adsbot | adsbot |
13 | Ahrefs | AhrefsBot |
14 | Alexa | ia_archiver |
15 | Amazonbot | Amazonbot |
16 | Archive.org | archive.org |
17 | Axios | axios |
18 | Baidu | Baiduspider |
19 | Barkrowler | Barkrowler |
20 | Bing | bingbot |
21 | Bingpreview | BingPreview |
22 | BLEXbot | BLEXBot |
23 | Bytespider | Bytespider |
24 | CCBot | CCBot |
25 | CheckHost | CheckHost |
26 | Cliqzbot | Cliqzbot |
27 | CloudEndure | CloudEndure |
28 | CloudEndure | CloudEndure Scanner |
29 | COMODO | COMODO SSL Checker |
30 | Companybook | Companybook-Crawler |
31 | DeuSu | DeuSu |
32 | Discord | Discordbot |
33 | DomainSigmaCrawler | DomainSigmaCrawler |
34 | DotBot | DotBot |
35 | DuckDuckGo | DuckDuckBot |
36 | ExaBot | ExaBot |
37 | Glutenfree | Gluten Free Crawler |
38 | Grammarly | Grammarly |
39 | HubPages | HubPages |
40 | James BOT | James BOT |
41 | Linkdex | linkdexbot |
42 | Mail.RU | Mail.RU_Bot |
43 | Majestic 12 | MJ12Bot |
44 | MegaIndex | MegaIndex |
45 | Mojeek | MojeekBot |
46 | MSN | MsnBot |
47 | Nerdy | NerdyBot |
48 | Nimbostratus | Nimbostratus-Bot |
49 | Nutch | Nutch |
50 | oBot | oBot |
51 | Omgili | omgili |
52 | PagesInventory | PagesInventory |
53 | PaperLiBot | PaperLiBot |
54 | PetalBot | PetalBot |
55 | Pingdom | Pingdom.com_bot |
56 | proximic | proximic |
57 | Scrapy | Scrapy |
58 | Semrush | SemrushBot |
59 | Seznam | SeznamBot |
60 | SiteChecker | SiteChecker |
61 | Sogou | Sogou |
62 | SP | spbot |
63 | SurveyBot | SurveyBot |
64 | TKAI | The Knowledge AI |
65 | trendiction | trendictionbot |
66 | TrendsmapResolver | TrendsmapResolver |
67 | UptimeRobot | UptimeRobot |
68 | vBSEO | vBSEO |
69 | WebTarantula | WebTarantula |
70 | Xovi | XoviBot |
71 | XYZbot | XYZbot |
72 | Yak | Yak |
73 | Zoombot | ZoomBot |
74 | ZuperlistBot | ZuperlistBot |