Search crawlers
[ 24.10.30 up!! ]

Adsbot
Ahrefsbot
Amazonbot
Applebot
AwarioBot
archive.org_bot
Baidu
Baiduspider
Barkrowler
bidswitchbot
Bingbot
bingbot
BLEXBot
Botify
BUbiNG
Bytespider
CCBot
Cincraw
ClaudeBot
coccocbot
CognitiveSEO
CognitiveSEO Site Explorer
ContentKing
contxbot
Cotoyogi
CriteoBot
Cyotek WebCopy
DataForSeoBot
Daum
DeepCrawl
Dexi.io
Domains Project
DotBot
dotbot
DuckDuckBot
DuckDuckGo
Exabot
Facebook Crawler
facebookexternalhit
Feedly
FemtosearchBot
Getleft
Gigabot
Googlebot
Google-InspectionTool
GPTBot
GrapeshotCrawler
Helium Scraper
HTTrack
ia_archiver
ias-sg
ICC-Crawler
Import.io
integralads
jet-bot
JetOctopus
Linespider
Linguee
Linguee Bot
linkdexbot
linkfluence
LinkedInBot
ltx71
Lumar
Mail.RU_Bot
Majestic
ManifoldCF
Mappy
MegaIndex.ru
MicroAdBot
MJ12bot
msnbot
Neevabot
netEstate NE Crawler
Netpeak Spider
Octoparse
OnCrawl
Outwit Hub
ParseHub
petalbot
Pinterestbot
Pockey
proximic
psbot
Quantcastbot
Rogerbot
Scrapy
Screaming Frog
Screaming Frog SEO Spider
SeekporBot
SemrushBot
SEOkicks
Sequentum
SerendeputyBot
serpstatbot
SeznamBot
Sitebulb Crawler
Slurp
SMTBot
sogou spider
Sogou head spider
Sogou Orion spider
Sogou Pic Spider
Sogou-Test-Spider
Sogou web spider
Sonic
spbot
special_archiver
spider
Steeler
Superfeedr
Superfeedr bot
Swiftbot
trendictionbot
TurnitinBot
TweetmemeBot
Twitterbot
UptimeRobot
VelenPublicWebCrawler
Visual Scraper
WebHarvy
Webhose.io
WooRank
Wotbox
Yahoo Slurp
YaK
YandexBot
Yeti
Yetibot
YoudaoBot
Y!J
ZoominfoBot
Zyte
360Spider
80legs