Fixing Nginx 403 Forbidden: SELinux Permission Denied

Understanding the Error

When configuring Nginx on a Linux distribution that uses Security-Enhanced Linux (SELinux) by default—such as CentOS, RHEL, Rocky Linux, Fedora, or AlmaLinux—you might frequently encounter a 403 Forbidden or 502 Bad Gateway error when trying to serve web pages or proxy requests. If you have already verified that the standard POSIX file permissions (read/write/execute) and ownership (e.g., chown -R nginx:nginx) are correct, the root cause is almost certainly SELinux intervening to prevent unauthorized access.

SELinux implements Mandatory Access Control (MAC). In this system, every process, file, directory, and port has a specific security context (or label). Nginx runs confined within the httpd_t domain. By default, the SELinux policy dictates that the httpd_t domain is only allowed to access files labeled with specific web-related contexts, primarily httpd_sys_content_t for read-only access and httpd_sys_rw_content_t for read-write access.

If you place your website files in a non-standard directory (like /home/user/web, /opt/myapp, or /data/www), those files will inherit the default context of their parent directory. Because Nginx's httpd_t domain is explicitly denied access to arbitrary contexts like user_home_t or default_t, the kernel blocks the read attempt, and Nginx logs a Permission denied error, returning a 403 to the client.

Step 1: Diagnose the Issue

Before executing commands to modify SELinux policies, it is critical to confirm that SELinux is actually responsible for the Permission denied error. This involves correlating Nginx error logs with SELinux audit logs.

1. Check Nginx Error Logs

First, inspect the Nginx error log, typically located at /var/log/nginx/error.log:

tail -n 20 /var/log/nginx/error.log

You will likely see an error message resembling this standard file access denial:

2023-10-25 14:32:10 [error] 12345#0: *1 open() "/var/www/html/index.html" failed (13: Permission denied), client: 192.168.1.100, server: example.com, request: "GET / HTTP/1.1"

Alternatively, if Nginx is acting as a reverse proxy, you might see an upstream connection denial:

2023-10-25 14:45:01 [crit] 12345#0: *5 connect() to 127.0.0.1:8080 failed (13: Permission denied) while connecting to upstream...

2. Check SELinux Audit Logs

If you identify the Permission denied error (code 13) in Nginx, immediately check the SELinux audit log (/var/log/audit/audit.log). You can use the ausearch utility to filter for Nginx-related denials specifically:

sudo ausearch -m AVC,USER_AVC,SELINUX_ERR,USER_SELINUX_ERR -ts recent

If ausearch is not installed, you can use grep:

sudo grep nginx /var/log/audit/audit.log | grep denied

A typical AVC (Access Vector Cache) denial log looks like this:

type=AVC msg=audit(1698244330.123:456): avc: denied { read } for pid=12345 comm="nginx" name="index.html" dev="vda1" ino=67890 scontext=system_u:system_r:httpd_t:s0 tcontext=unconfined_u:object_r:user_home_t:s0 tclass=file permissive=0

Let's break down the critical components of this log entry:

• comm="nginx": The process attempting the action.
• { read }: The action that was denied.
• scontext=system_u:system_r:httpd_t:s0: The source context (the Nginx process running as httpd_t).
• tcontext=unconfined_u:object_r:user_home_t:s0: The target context (the file). Here, the file has a user_home_t label, which httpd_t is not permitted to read.

Step 2: Fix File Access Permissions (403 Forbidden)

The most robust and permanent way to fix SELinux permission issues for Nginx serving static files or PHP scripts is to update the SELinux file context database. This ensures that the correct labels are applied and maintained even if the filesystem undergoes a complete relabeling in the future.

Let's assume your custom web root is /data/www/mywebsite.

1. Define the New Context

You need to tell SELinux that all files and directories within /data/www/mywebsite should possess the httpd_sys_content_t context. The semanage fcontext command achieves this by adding a rule to the policy using regular expressions.

# Install tools if missing (RHEL/CentOS 8+)
sudo dnf install policycoreutils-python-utils

# Add the context rule
sudo semanage fcontext -a -t httpd_sys_content_t "/data/www/mywebsite(/.*)?"

2. Apply the Context (restorecon)

The semanage command only updates the internal SELinux database. To physically apply these changes to the files currently on the filesystem, you must run restorecon.

sudo restorecon -Rv /data/www/mywebsite

The -R flag recursively applies the context to all subdirectories and files, while -v provides verbose output so you can verify the labels changing.

Handling Read/Write Requirements

If your Nginx server (or a backend like PHP-FPM operating under a similar context) needs to write to a specific directory—such as an uploads folder or a cache directory—the standard httpd_sys_content_t label is insufficient as it only grants read privileges.

For directories requiring write access, you must assign the httpd_sys_rw_content_t label:

sudo semanage fcontext -a -t httpd_sys_rw_content_t "/data/www/mywebsite/uploads(/.*)?"
sudo restorecon -Rv /data/www/mywebsite/uploads

Step 3: Fix Reverse Proxy Network Permissions (502 Bad Gateway)

Another highly common scenario involves configuring Nginx as a reverse proxy, forwarding requests to an upstream application server (like Node.js, Gunicorn, Tomcat, or another web server) listening on a local port (e.g., 8080, 3000).

When you access the site, you receive a 502 Bad Gateway, and the Nginx error log shows: [crit] ... connect() to 127.0.0.1:8080 failed (13: Permission denied) while connecting to upstream

By default, SELinux policy prevents the web server domain (httpd_t) from initiating outbound network connections. This is a security measure to limit the blast radius if the web server is compromised. However, it completely breaks reverse proxy functionality.

You can resolve this by enabling a specific SELinux boolean. Booleans are essentially on/off switches that modify the active SELinux policy without requiring you to write or compile custom policy modules.

To permit Nginx to establish network connections to upstream servers:

sudo setsebool -P httpd_can_network_connect 1

The -P flag is crucial; it makes the boolean change persistent across system reboots. If omitted, the permission will revert the next time the server restarts.

Using audit2allow for Complex Scenarios

In some edge cases, you might encounter denials that aren't easily solved by standard labels or booleans. For instance, if Nginx needs to interact with a very specific, non-standard system resource. In these situations, you can generate a custom SELinux policy module using audit2allow.

This tool reads the audit log, identifies the denied actions, and generates the necessary policy rules to allow them.

# Generate a human-readable explanation
sudo grep nginx /var/log/audit/audit.log | audit2allow -w

# Generate and compile a custom policy module named 'nginx_custom'
sudo grep nginx /var/log/audit/audit.log | audit2allow -M nginx_custom

# Install the new policy module
sudo semodule -i nginx_custom.pp

Caution: Only use audit2allow if you completely understand the implications of the allowed rule. It will blindly permit whatever was denied, potentially opening security holes if the denial was actually preventing malicious activity.

Conclusion

While SELinux's strict access controls can initially seem like a hindrance, particularly when troubleshooting Permission denied errors in Nginx, immediately disabling it via setenforce 0 severely degrades your server's security posture. By methodically diagnosing the issue via audit logs and utilizing standard tools like semanage, restorecon, and setsebool, you can precisely grant Nginx the access it needs while maintaining the robust protections SELinux provides. Always verify the exact context conflict before blindly applying labels.