Exclude certain requests from the Nginx access log

Logs are nice and all that, but sometimes certain entries are there just to fill up the logs or are cluttering them. Here’s a few ways to exclude requests – by URL or visitor IP – from the Nginx access log.

Exclude specific URLs

In my Nginx configs I usually have a location block like this for static resources. It makes sure correct caching headers are sent, but also turns off logging for static resources – both the access log and the error log when a 404 is returned:

(I handle JS and CSS files in a separate block. Also note that you can very well use a map instead of the awkwardly long regexp)

Exclude specific user agents

If you’re not interested in logging requests from certain bots/crawlers, or you have a monitoring service, like Pingdom.

Note that maps must be created outside of the server block.

Exclude specific IP addresses

If you run a cronjob with curl/wget on your localhost polling your website at certain intervals (e.g. you run WordPress but use a real cronjob to poll wp-cron.php), or you have health checks from downstream.

Combining tests

If you want to match only specific user agents coming for specific IP addresses, it is possible to combine the tests. However, the Nginx config does not support a logical and operator, so we have to resort to a “clever hack”. Here’s a config to check for certain user agents from certain IPs:

You can of course add a “test C” checking for the URL too.

There are 3 comments

I love comments that bring new insights, shares ideas and experiences, and most of all: corrects my mistakes. For support questions, there are other fora, like Stack Overflow, Server Fault and the WordPress support forum.

Your email address will not be published. Required fields are marked *