Skip to content


禁止微软搜索蜘蛛

禁止微软蜘蛛,爬的太疯狂了,还不带流量…
同时降低频率到60秒间隔.
在web根目录下编辑robots.txt

User-agent: Bingbot
Disallow: /
User-agent: Adidxbot
Disallow: /
User-agent: MSNBot
Disallow: /
User-agent: BingPreview
Disallow: /
User-agent: *
Disallow:
Crawl-delay: 60
Disallow: /api/
Disallow: /data/

参考:
http://www.bing.com/webmaster/help/which-crawlers-does-bing-use-8c184ec0
http://tool.chinaz.com/robots/

Posted in 网站建设, SEO.

Tagged with , , .


No Responses (yet)

Stay in touch with the conversation, subscribe to the RSS feed for comments on this post.



Some HTML is OK

or, reply to this post via trackback.