禁止微软蜘蛛,爬的太疯狂了,还不带流量… 同时降低频率到60秒间隔. 在web根目录下编辑robots.txt
User-agent: Bingbot Disallow: / User-agent: Adidxbot Disallow: / User-agent: MSNBot Disallow: / User-agent: BingPreview Disallow: / User-agent: * Disallow: Crawl-delay: 60 Disallow: /api/ Disallow: /data/参考: http://www.bing.com/webmaster/help/which-crawlers-does-bing-use-8c184ec0 http://tool.chinaz.com/robots/
No Responses (yet)
Stay in touch with the conversation, subscribe to the RSS feed for comments on this post.