定期会查看网站的访问日志
会看到很多的爬虫扫描以及ab测试还有curl
在nginx中有没有办法屏蔽掉呢
我们只需要新建文件
deny_agent.conf
>if ($http_user_agent ~* " SemrushBot|SemrushBot-SA|Bytespider|BLEXBot|CompSpyBot|Exabot|ZoominfoBot|ExtLinksBot|AlphaBot|DotBot|MauiBot|MegaIndex.ru|SiteExplorer|Webdup|AcoonBot|AhrefsBot|Ezooms|EdisterBot|EC2LinkFinder|jikespider|Purebot|MJ12bot|WangIDSpider|WBSearchBot|Wotbox|xbfMozilla|Yottaa|YandexBot|Jorgee|SWEBot|spbot|TurnitinBot-Agent|mail.RU|perl|Python|Wget|Xenu|ZmEu|MSNBot|WinHttpRequest|libssh2|libcurl|curl|ApacheBench") {
return 403;
}
加入网站的配置文件中
`include deny_agent.conf`
重启nginx
>nginx -t
nginx -s reload
即可
