Fixing Apache Crash and 'Connection Refused' Errors: A Complete Guide

Understanding the Error

When an Apache web server crashes, fails to start, or becomes unresponsive, it can bring down critical infrastructure. The symptoms vary widely: you might see a complete process failure (apache2.service: Main process exited, code=killed, status=9/KILL), clients receiving Connection refused, or the server grinding to a halt with apache slow symptoms.

At the core, these issues usually stem from resource exhaustion (RAM, file descriptors, or CPU), misconfiguration in the Multi-Processing Module (MPM), or segmentation faults caused by unstable third-party modules (like buggy PHP extensions).

Common Error Messages

You will typically encounter one of the following in your system logs (journalctl -xe) or Apache error logs:

• [mpm_prefork:error] [pid 12345] AH00161: server reached MaxRequestWorkers setting, consider raising the MaxRequestWorkers setting
• [core:notice] [pid 1234] AH00052: child pid 5678 exit signal Segmentation fault (11)
• Out of memory: Kill process 1234 (apache2) score 500 or sacrifice child
• Warning: DocumentRoot [/var/www/html] does not exist (causing startup failure)
• (24)Too many open files: AH00046: child process 5678 still did not exit, sending a SIGTERM

Step 1: Diagnose the Root Cause

The first step is always to consult the logs. Do not guess; let the logs tell you why the apache service not starting or crashing.

1. Check the Systemd Journal: If Apache fails to start, systemd holds the key.

   sudo journalctl -u apache2.service -e --no-pager
   # or on RHEL/CentOS:
   sudo journalctl -u httpd.service -e --no-pager
   

2. Inspect the Apache Error Log: This is the most critical file for diagnosing active crashes or connection limits.

   sudo tail -n 100 /var/log/apache2/error.log
   # or on RHEL/CentOS:
   sudo tail -n 100 /var/log/httpd/error_log
   

3. Check for OOM Kills: If the server is experiencing apache out of memory, the Linux kernel's OOM killer will terminate the process. Check the kernel ring buffer:

   dmesg -T | grep -i 'out of memory'
   dmesg -T | grep -i 'killed process'
   

4. Verify Configuration Syntax: Always ensure your configuration isn't broken before attempting a restart.

   sudo apache2ctl configtest
   # or
   sudo apachectl -t
   

Step 2: Fixing Resource Exhaustion (MaxRequestWorkers)

If your logs show server reached MaxRequestWorkers, your server is receiving more concurrent requests than it is configured to handle. This leads to queued connections, apache slow performance, and eventually apache connection refused.

To fix this, you must tune your MPM settings (usually located in /etc/apache2/mods-enabled/mpm_prefork.conf or mpm_event.conf).

Calculating the Correct MaxRequestWorkers: Do not arbitrarily increase this number, or you will cause an OOM crash. You must calculate it based on your available RAM.

1. Find the average memory usage of an Apache process:

   ps -ylC apache2 | awk '{x += $8;y += 1} END {print "Average Process Size (KB): "x/y}'
   

2. Determine available memory for Apache (Total RAM minus OS and database needs).
3. Divide available RAM by the average process size. For example, if you have 2000MB available for Apache, and each process takes 50MB, your MaxRequestWorkers should be roughly 40.

Update your configuration:

<IfModule mpm_prefork_module>
    StartServers             5
    MinSpareServers          5
    MaxSpareServers         10
    MaxRequestWorkers       40
    MaxConnectionsPerChild   500
</IfModule>

Note: Setting MaxConnectionsPerChild to a non-zero value helps prevent memory leaks in mod_php by recycling processes.

Step 3: Resolving OOM (Out of Memory) Kills

If dmesg confirms the OOM killer is terminating Apache, you have two choices: increase RAM or reduce Apache's memory footprint.

1. Lower MaxRequestWorkers: As calculated above, ensure Apache cannot spawn enough processes to consume all RAM.
2. Add Swap Space: As a temporary buffer (though it will cause slowness):

   sudo fallocate -l 2G /swapfile
   sudo chmod 600 /swapfile
   sudo mkswap /swapfile
   sudo swapon /swapfile
   

3. Disable Unused Modules: Every loaded module increases the base memory footprint of every Apache child process. Use a2dismod to disable modules you don't need (e.g., a2dismod status, a2dismod autoindex).

Step 4: Troubleshooting Segmentation Faults (Core Dumps)

If the error log shows child pid exit signal Segmentation fault (11), a module (often PHP, SSL, or a custom caching extension) is crashing.

To debug this, you need an apache core dump.

1. Enable core dumps in Apache. In /etc/apache2/apache2.conf, add:

   CoreDumpDirectory /tmp/apache-cores
   

2. Create the directory and set permissions:

   sudo mkdir /tmp/apache-cores
   sudo chown www-data:www-data /tmp/apache-cores
   

3. Ensure the OS allows core dumps (ulimit -c unlimited).
4. Restart Apache and wait for the crash. Once a core file is generated, analyze it with gdb:

   gdb /usr/sbin/apache2 /tmp/apache-cores/core.<pid>
   (gdb) bt full
   

   The backtrace (bt full) will reveal the exact function and module causing the crash. You can then update, recompile, or disable that specific module.

Step 5: Fixing 'Too Many Open Files'

If Apache cannot open network sockets or read files, it will crash. This happens when it hits the OS file descriptor limit.

1. Check the current limits for the Apache process:

   cat /proc/$(pgrep -u root apache2 | head -n 1)/limits | grep 'Max open files'
   

2. Increase the limit via systemd. Edit the service file:

   sudo systemctl edit apache2.service
   

3. Add the following lines to increase the limit to 65536:

   [Service]
   LimitNOFILE=65536
   

4. Reload and restart:

   sudo systemctl daemon-reload
   sudo systemctl restart apache2
   

By systematically checking logs, tuning MPM values based on available memory, and addressing OS-level limits, you can restore stability and resolve Apache crashes effectively.