很多站点会存在误封了蜘蛛ip,导致抓取收录存在问题,对网站seo造成严重的影响,此文档会未大家持续更新各个搜索引擎的蜘蛛的ip端,方便大家来判断蜘蛛的真实性。很多恶意蜘蛛会伪造UA,假冒搜索引擎蜘蛛来对网站进行抓取。
真假蜘蛛的鉴别
对于大多数蜘蛛可以通过nslookup反查解析来判断蜘蛛的真实性,但是有一些搜索引擎蜘蛛是无法使用此方法鉴别的,这就需要通过搜索引擎蜘蛛公布的ip段来进行识别。
百度蜘蛛
百度蜘蛛UA
百度蜘蛛移动UA:
Mozilla/5.0 (Linux;u;Android 4.2.2;zh-cn;) AppleWebKit/534.46 (KHTML,like Gecko)Version/5.1 Mobile Safari/10600.6.3 (compatible; Baiduspider/2.0;+http://www.baidu.com/search/spider.html)
或
Mozilla/5.0 (iPhone;CPU iPhone OS 9_1 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko)Version/9.0 Mobile/13B143 Safari/601.1 (compatible; Baiduspider-render/2.0;+http://www.baidu.com/search/spider.html)
百度蜘蛛PC UA:
Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)
或
Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
百度蜘蛛小程序UA:
Mozilla/5.0 (iPhone;CPU iPhone OS 9_1 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko)Version/9.0 Mobile/13B143 Safari/601.1 (compatible; Baiduspider-render/2.0;Smartapp; +http://www.baidu.com/search/spider.html)
百度蜘蛛ip段
#持续更新中,最后更新于2022-6-12
123.125.66.*
123.125.68.*
123.125.71.*
220.181.108.*
220.181.108.77
220.181.108.75
220.181.108.80
220.181.108.83
220.181.108.86
220.181.108.89
220.181.108.91
220.181.108.92
220.181.108.94
220.181.108.95
220.181.108.97
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
360蜘蛛
360蜘蛛UA
360搜索的蜘蛛在访问网站时,都会带上带有360Spider签名信息的UA,其形态为:
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36; 360Spider
360搜索蜘蛛暂时不支持nslookup命令的查询,因此建议大家以本页面查询到的IP为准,结合本页提供的信息进行比对,以确认是否为360搜索的蜘蛛。
#持续更新中,最后更新于2022-6-12
42.236.101.*
42.236.102.*
42.236.103.*
42.236.10.*
42.236.12.*
42.236.13.*
42.236.14.*
42.236.15.*
42.236.16.*
42.236.17.*
42.236.46.*
42.236.48.*
42.236.49.*
42.236.50.*
42.236.51.*
42.236.52.*
42.236.53.*
42.236.54.*
42.236.55.*
42.236.99.*
180.153.232.*
180.153.234.*
180.153.236.*
180.163.220.*
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
搜狗蜘蛛
搜狗蜘蛛UA
PC端UA为: Sogou web spider、Sogou inst spider、Sogou spider
无线端UA为: Sogou wap spider
搜狗蜘蛛ip
#持续更新中,最后更新于2022-6-12
123.126.113.110
123.126.113.111
123.126.113.112
123.126.113.113
123.126.113.114
123.126.113.115
123.126.113.116
123.126.113.117
123.126.113.118
123.126.113.119
123.126.113.120
123.126.113.121
123.126.113.122
123.126.113.123
123.126.113.124
123.126.113.125
123.126.113.126
123.126.113.127
123.126.113.128
123.126.113.129
123.126.113.130
123.126.113.131
123.126.113.132
123.126.113.133
123.126.113.134
123.126.113.135
123.126.113.136
123.126.113.137
123.126.113.138
123.126.113.139
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31