NFS Permission Denied: Complete Troubleshooting Guide (Plus Connection Refused, Slow Mounts & Crashes)

Understanding NFS Errors

NFS (Network File System) errors span four broad categories — permission errors, connectivity failures, performance degradation, and daemon crashes. Although the symptoms differ, they share a common diagnostic path: verify the server export, confirm network reachability, check RPC services, and inspect the client mount options.

Exact Error Messages You Will See

Before diving into fixes, identify which exact error you are hitting:

• mount.nfs: access denied by server while mounting <server>:<path> — server refused the mount request, usually an export mismatch or IP not listed.
• Permission denied on ls, cp, or touch after a successful mount — UID/GID mismatch or wrong export permissions.
• mount.nfs: Connection refused — portmapper (port 111) or nfsd (port 2049) is not reachable.
• mount.nfs: No route to host — network or firewall issue at the IP layer.
• mount.nfs: Stale file handle — server-side path was removed or server rebooted while the client was mounted.
• nfs: server <host> not responding, timed out — NFS server process hung or network congestion.
• Input/output error on a previously working mount — server-side crash or export unexpectedly removed.

Step 1: Verify the Export Is Visible

Always start on the client:

showmount -e <nfs-server-ip>

Expected output:

Export list for 192.168.1.10:
/data  192.168.1.0/24

If you receive clnt_create: RPC: Program not registered or Connection refused, the NFS or portmapper service is not running on the server, or a firewall is blocking port 111.

If the path you want is absent from the list, the export is not configured or the cache is stale.

Server-side fix — refresh exports:

sudo exportfs -rav

If you still don't see the path after editing /etc/exports, confirm the syntax. A common mistake is mixing tabs and spaces or omitting the options parentheses:

# WRONG — space between path and options breaks the line
/data  192.168.1.0/24 (rw,sync,no_subtree_check)

# CORRECT
/data  192.168.1.0/24(rw,sync,no_subtree_check)

Step 2: Diagnose UID/GID Mismatches (Most Common Cause of Permission Denied)

NFSv3 uses AUTH_SYS, which means the client sends its local numeric UID and GID to the server. The server then checks those numbers against the exported directory's ownership. If the user deploy has UID 1001 on the client but UID 1002 on the server, every file operation will fail with Permission denied.

Check UIDs on both hosts:

# On client
id deploy
# uid=1001(deploy) gid=1001(deploy)

# On server
id deploy
# uid=1002(deploy) gid=1001(deploy)  <-- mismatch!

Fix option A — Align UIDs:

# On server, change UID to match client
sudo usermod -u 1001 deploy
# Fix ownership of existing files
sudo find /data -user 1002 -exec chown 1001 {} \;

Fix option B — Use all_squash for shared directories: Add to /etc/exports for a world-writable share where individual ownership does not matter:

/data/shared  192.168.1.0/24(rw,sync,all_squash,anonuid=1001,anongid=1001)

Then sudo exportfs -rav.

Fix option C — NFSv4 with idmapd (name-based mapping): NFSv4 maps usernames rather than raw UIDs. Ensure /etc/idmapd.conf has the same Domain = value on both client and server:

[General]
Domain = example.com

Restart the service:

sudo systemctl restart nfs-idmapd   # server
sudo systemctl restart nfs-idmapd   # client

If files show as owned by nobody, the domain names differ.

Step 3: Fix NFS Connection Refused

Connection refused means a TCP/UDP connection attempt reached the host but no listener exists on the target port.

Verify NFS daemons are running:

sudo systemctl status nfs-server rpcbind
# or on older systems:
sudo service nfs status
sudo rpcinfo -p localhost

Expected rpcinfo output should include entries for portmapper, nfs, mountd, and nlockmgr.

Start the services if down:

sudo systemctl enable --now nfs-server
sudo systemctl enable --now rpcbind

Check firewall rules (firewalld):

sudo firewall-cmd --list-all
sudo firewall-cmd --permanent --add-service=nfs
sudo firewall-cmd --permanent --add-service=rpc-bind
sudo firewall-cmd --permanent --add-service=mountd
sudo firewall-cmd --reload

Check firewall rules (iptables):

sudo iptables -L INPUT -n --line-numbers | grep -E '111|2049|mountd'
# Add rules if missing:
sudo iptables -I INPUT -p tcp --dport 2049 -j ACCEPT
sudo iptables -I INPUT -p udp --dport 2049 -j ACCEPT
sudo iptables -I INPUT -p tcp --dport 111  -j ACCEPT
sudo iptables -I INPUT -p udp --dport 111  -j ACCEPT

For NFSv3, mountd uses a dynamic port — pin it to a fixed port to simplify firewall rules:

# /etc/nfs.conf or /etc/sysconfig/nfs
[mountd]
port=20048

Then restart nfs-server and add port 20048 to the firewall.

Step 4: Recover from NFS Crash and Stale Mounts

When an NFS server crashes or reboots, clients holding mounts will hang on any I/O because the default mount option is hard (retries indefinitely). The kernel log will show:

nfs: server 192.168.1.10 not responding, timed out
nfs: server 192.168.1.10 not responding, still trying

Immediate unmount on client (if server is not coming back quickly):

sudo umount -l /mnt/nfs   # lazy unmount — detaches filesystem
# or force if lazy fails:
sudo umount -f /mnt/nfs

If umount itself hangs, identify and kill the blocking process:

sudo lsof /mnt/nfs
sudo fuser -km /mnt/nfs
sudo umount -l /mnt/nfs

Prevent hangs with soft + timeo mount options (for non-critical mounts):

192.168.1.10:/data  /mnt/nfs  nfs  soft,timeo=30,retrans=3,_netdev  0 0

Note: soft mounts will return I/O errors to applications on timeout, which may cause data corruption for write-heavy workloads. Use hard,intr for databases.

Step 5: Fix Slow NFS Performance

Slow NFS is almost always caused by small rsize/wsize defaults, synchronous writes, or insufficient nfsd threads.

Check current mount options and throughput:

cat /proc/mounts | grep nfs
nfsstat -m
mountstat  # from nfs-utils package

Increase transfer block sizes:

# Remount with larger block sizes (up to 1MB on modern kernels)
sudo mount -t nfs -o rsize=1048576,wsize=1048576,hard,intr \
  192.168.1.10:/data /mnt/nfs

Use async writes for non-critical data:

/data  192.168.1.0/24(rw,async,no_subtree_check)

Warning: async means the server acknowledges writes before they reach disk. Use only when data loss on server crash is acceptable.

Increase nfsd thread count:

# Temporarily
sudo rpc.nfsd 32

# Permanently in /etc/nfs.conf
[nfsd]
threads=32

Baseline benchmark:

# Write test
dd if=/dev/zero of=/mnt/nfs/testfile bs=1M count=1024 oflag=direct
# Read test
dd if=/mnt/nfs/testfile of=/dev/null bs=1M

Step 6: Validate the Entire Stack

After applying any fix, run a full validation:

# 1. Server: confirm export is live
exportfs -v

# 2. Client: confirm RPC services are reachable
rpcinfo -p <server-ip>

# 3. Client: mount and verify permissions
mount -t nfs4 <server-ip>:/data /mnt/test
ls -la /mnt/test
touch /mnt/test/probe_file && echo "Write OK" && rm /mnt/test/probe_file

# 4. Check kernel logs for NFS errors
dmesg | grep -i nfs | tail -20
journalctl -u nfs-server --since '10 minutes ago'