Resolving Cloudflare API Timeout (Error 524: A Timeout Occurred)
Fix Cloudflare Error 524 API timeouts. Learn how to bypass the 100-second limit using asynchronous workers, chunking, and Enterprise configurations.
- Cloudflare enforces a strict 100-second idle timeout for all HTTP requests on Free, Pro, and Business plans, resulting in an Error 524.
- Error 504 (Gateway Timeout) differs from 524; a 504 means your origin server timed out before Cloudflare's limit, whereas 524 is Cloudflare actively terminating the connection.
- The most robust fix is decoupling long-running API tasks using asynchronous background workers (e.g., Celery, Redis Queue) and polling for status.
- Enterprise customers can explicitly increase the proxy_read_timeout up to 6000 seconds via Cloudflare Support or API.
| Method | When to Use | Time to Implement | Risk/Complexity |
|---|---|---|---|
| Asynchronous Workers (Queues) | Heavy processing (PDF generation, ML inference, large db queries) | Days | Medium |
| Data Pagination / Chunking | Large data exports or bulk API imports | Hours | Low |
| Bypass Cloudflare (Grey Cloud) | Internal API endpoints not requiring DDoS protection or caching | Minutes | High (Exposes Origin IP) |
| Increase Timeout (Enterprise Only) | Legacy monolithic apps that cannot be easily rewritten | Minutes | Low (Requires Enterprise Plan) |
Understanding the Error
When working with APIs routed through Cloudflare, developers frequently encounter sudden connection drops on long-running requests. The exact symptom is usually an HTTP 524 A Timeout Occurred status code.
Unlike standard HTTP timeouts, a 524 error is entirely specific to Cloudflare. It indicates that Cloudflare successfully established a TCP connection to the origin web server, but the origin did not reply with an HTTP response before the connection timed out. By default, Cloudflare waits exactly 100 seconds for a response. If your API endpoint takes 101 seconds to process a massive payload or execute a complex database query, Cloudflare drops the connection, returning the 524 error to the client, even if your origin server eventually completes the task successfully in the background.
It is crucial to differentiate this from a 504 Gateway Timeout. A 504 error means your origin server (or a load balancer/reverse proxy in front of it like Nginx or HAProxy) timed out the request before Cloudflare's 100-second limit was reached.
Step 1: Diagnose the Timeout
Before refactoring your API, verify exactly where the timeout is occurring.
- Check the HTTP Status Code: A 524 confirms Cloudflare killed the connection. A 504 means your origin's configuration (e.g., Nginx
proxy_read_timeout, Gunicorn timeout, or PHPmax_execution_time) is too low. - Analyze Origin Logs: Correlate the
CF-Rayheader from the client request with your origin server logs. If the origin log shows the request completing successfully after 110 seconds, but the client received a 524 at the 100-second mark, you have confirmed the Cloudflare 100s limit is the culprit. - Network Tab Profiling: Use your browser's dev tools or
curl -vto measure the exact Time to First Byte (TTFB). If it hangs at exactly 100s, it's Cloudflare.
Step 2: Fix - Architectural Solutions
If you are on a Free, Pro, or Business plan, you cannot change the 100-second limit. You must change your application architecture.
Solution A: Implement Asynchronous Processing (The Right Way)
For tasks like generating reports, processing video, or bulk database operations, synchronous HTTP requests are an anti-pattern.
- The client sends a
POSTrequest to the API to initiate the job. - The API immediately returns a
202 Acceptedstatus with ajob_idand aLocationheader pointing to a status endpoint. - A background worker (e.g., Celery, Sidekiq, AWS SQS) processes the task.
- The client polls the status endpoint (e.g.,
GET /api/jobs/{job_id}) every few seconds. - Once complete, the status endpoint returns a
303 See Otherredirecting to the result, or returns the result payload directly.
Solution B: Data Chunking and Pagination
If the timeout occurs during bulk data retrieval or insertion, break the payload into smaller chunks. Instead of requesting 100,000 records in one API call, request 5,000 records per page. Use cursor-based pagination to ensure consistent reads across chunks.
Step 3: Fix - Configuration Solutions
Solution C: Bypass Cloudflare for Specific Subdomains
If the API endpoint is for internal use only (e.g., an admin dashboard triggering a heavy backup), you can route the traffic directly to the origin, bypassing Cloudflare's proxy.
- Create a new DNS A record (e.g.,
direct-api.yourdomain.com). - Set the proxy status to "DNS Only" (Grey Cloud).
- Update your internal tools to use this unproxied endpoint. Warning: This exposes your origin IP to the public internet and bypasses WAF and DDoS protection. Secure this endpoint heavily via firewall rules (e.g., AWS Security Groups or iptables) restricting access to known IPs.
Solution D: Enterprise Timeout Adjustments
If you are a Cloudflare Enterprise customer, you can increase the default timeout. This is often the quickest fix for legacy applications that cannot be easily rewritten.
- Go to your Cloudflare Dashboard.
- Navigate to Network -> Proxy Read Timeout.
- Increase the value (up to 6000 seconds).
Alternatively, you can adjust this via the Cloudflare API or Terraform using the
cloudflare_zone_settings_overrideresource.
Frequently Asked Questions
# Diagnostic command to measure exact TTFB and verify the 100s drop
curl -o /dev/null -s -w "Time Connect: %{time_connect}s\nTime TTFB: %{time_starttransfer}s\nTotal Time: %{time_total}s\nHTTP Code: %{http_code}\n" https://api.yourdomain.com/heavy-endpoint
# Example output indicating a Cloudflare 524 timeout:
# Time Connect: 0.045s
# Time TTFB: 0.000s
# Total Time: 100.052s
# HTTP Code: 524
# --- Example of Asynchronous Polling implementation (Client Side) ---
#!/bin/bash
# 1. Start the job
JOB_RESPONSE=$(curl -s -X POST https://api.yourdomain.com/v1/export)
JOB_ID=$(echo $JOB_RESPONSE | jq -r '.job_id')
echo "Started job $JOB_ID. Polling for completion..."
# 2. Poll until complete
STATUS="pending"
while [ "$STATUS" != "completed" ]; do
sleep 5
STATUS_RESPONSE=$(curl -s https://api.yourdomain.com/v1/jobs/$JOB_ID)
STATUS=$(echo $STATUS_RESPONSE | jq -r '.status')
echo "Current status: $STATUS"
if [ "$STATUS" == "failed" ]; then
echo "Job failed."
exit 1
fi
done
echo "Job complete! Downloading result..."
RESULT_URL=$(echo $STATUS_RESPONSE | jq -r '.result_url')
curl -O $RESULT_URLError Medic Editorial
Error Medic Editorial comprises senior DevOps engineers and Site Reliability Experts dedicated to demystifying complex cloud infrastructure and network errors.