nginx-module-bot-verifier

Description

[BETA] This module has been tested on a handful of production websites. This is an NGINX module designed to validate actors claiming to be search engine indexers. It is right to disable security mechanisms for valid search engine bots to ensure your controls do not interfere with page rankings on any of the search providers. The issue is that the User Agent header cannot be trusted. In order to ensure you are allowing only valid search engine indexers, you must validate according to their published standards. This module performs that validation and caches the results to ensure you do not pay validation penalties on every request. To enable this module after installation, add the following to /etc/nginx/nginx.conf and reload NGINX: load_module modules/ngx_http_bot_verifier_module.so; Alternatively, you can enable all installed modules by placing this line at the top of /etc/nginx/nginx.conf: include /usr/share/nginx/modules/*.conf;

Compatible with all RHEL-based distributions, including CentOS, AlmaLinux, Oracle Linux, Rocky Linux, etc.