systemctl failed: How to Diagnose and Fix Service Failures on Linux

Understanding 'systemctl failed'

When systemd reports a service in a failed state you will see output like this from systemctl status:

● nginx.service - A high performance web server and a reverse proxy server
     Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Mon 2026-02-23 10:14:32 UTC; 3s ago
    Process: 18432 ExecStartPre=/usr/sbin/nginx -t (code=exited, status=1/FAILURE)
   Main PID: 18432 (code=exited, status=1/FAILURE)
        CPU: 42ms

The Result field tells you the failure category:

• exit-code — the process started but returned a non-zero exit status
• signal — the process was killed by a signal (e.g., SIGSEGV, SIGKILL from the OOM killer)
• core-dump — the process crashed and produced a core dump
• timeout — the service did not become active within TimeoutStartSec
• resources — systemd could not allocate the resources specified in the unit
• start-limit-hit — the service restarted too many times and hit the StartLimitBurst threshold

Understanding the Result field narrows the search space dramatically before you even read the logs.

Step 1: Read the Journal Immediately

The single most important command after any systemctl failure is:

journalctl -xe -u <service-name> --no-pager

Flags:

• -x — adds explanatory catalog messages that describe what the log entry means
• -e — jumps to the end of the journal (most recent entries)
• -u <service-name> — filters to only messages from that unit

For persistent failures or to see entries across reboots:

journalctl -u <service-name> --since "10 minutes ago"
journalctl -u <service-name> -b -1   # previous boot

If the service logs to its own file rather than the journal, check:

systemctl cat <service-name>          # shows unit file with StandardError/StandardOutput paths
tail -100 /var/log/<service>/<service>.log

Step 2: Identify the Root Cause

Scenario A: ExecStart binary not found or not executable

Journal will show:

Feb 23 10:14:32 host systemd[1]: nginx.service: Control process exited with error code.
Feb 23 10:14:32 host systemd[1]: Failed to start A high performance web server.

And closer inspection reveals:

exec /usr/sbin/nginx: no such file or directory

Fix:

which nginx                           # find actual binary path
ls -la /usr/sbin/nginx                # verify it exists and is executable
chmod +x /usr/sbin/nginx              # if missing execute bit
systemctl cat nginx.service           # compare ExecStart path to actual binary

Scenario B: systemctl permission denied

This error appears in two distinct contexts:

1. The user running systemctl lacks privileges:

Failed to connect to bus: Permission denied
Failed to disable unit: Access denied

Fix: Run with sudo systemctl ... or grant polkit rules for the specific action.

2. The service process itself gets permission denied on a file or socket:

nginx: [emerg] open() "/var/run/nginx.pid" failed (13: Permission denied)

Fix:

ls -la /var/run/nginx.pid             # check owner
chown root:root /var/run/nginx.pid    # or the user specified in nginx.conf
systemctl show nginx.service -p User  # check which user systemd starts nginx as

Scenario C: Missing dependency or ordered unit

Journal shows:

ConditionPathExists=/etc/myapp/config.yaml was not met

Or a dependency like network.target or a database socket was not ready.

Fix:

systemctl list-dependencies <service-name>   # show full dependency tree
systemctl is-active network-online.target    # check if network dependency is met
# Add to unit [Unit] section:
# After=network-online.target postgresql.service
# Requires=postgresql.service

Scenario D: Start limit hit

Journal shows:

systemd[1]: myapp.service: Start request repeated too quickly.
systemd[1]: myapp.service: Failed with result 'start-limit-hit'.

This happens when a service crashes immediately on startup and systemd has tried restarting it beyond StartLimitBurst times within StartLimitIntervalSec.

Fix the underlying crash first, then reset the failed state:

systemctl reset-failed myapp.service
systemctl start myapp.service

To raise the limit while debugging (not for production):

systemctl edit myapp.service
# Add:
# [Unit]
# StartLimitBurst=0

Scenario E: Environment variables or working directory missing

myapp[18501]: Error: DATABASE_URL environment variable is required

Fix — add to unit file with systemctl edit:

systemctl edit myapp.service
# [Service]
# Environment=DATABASE_URL=postgres://user:pass@localhost/db
# EnvironmentFile=/etc/myapp/env
# WorkingDirectory=/opt/myapp

Step 3: Apply the Fix and Reload

After editing any unit file:

systemctl daemon-reload              # re-read all unit files
systemctl start <service-name>       # attempt start
systemctl status <service-name>      # verify active (running)

Verify the service will survive a reboot:

systemctl is-enabled <service-name>  # should return 'enabled'
systemctl enable <service-name>      # if it was disabled

Step 4: When systemctl Is Not Working At All

If systemctl itself errors:

System has not been booted with systemd as init system (PID 1). Can't operate.
Failed to connect to bus: Host is down

This means either:

• You are in a Docker container or chroot without systemd (use service or direct binary management)
• D-Bus is not running: systemctl start dbus or apt install dbus
• systemd is corrupted: reinstall with apt install --reinstall systemd

Check what PID 1 actually is:

ps -p 1 -o comm=
ls -la /sbin/init

Preventive Practices

1. Always validate unit files before deploying: systemd-analyze verify /path/to/my.service
2. Test with --dry-run when available: nginx -t, apache2ctl configtest
3. Use systemctl edit for overrides rather than editing vendor unit files in /lib/systemd/system/ — your changes survive package upgrades
4. Set Restart=on-failure and RestartSec=5 for long-running services so transient failures self-heal
5. Capture full logs with StandardOutput=journal and StandardError=journal in the [Service] section