Azure API Timeout: Fix 504 Gateway Timeout and RequestTimeout Errors in Azure API Management, Functions, and ARM

Understanding Azure API Timeout Errors

Azure API timeouts surface at multiple layers of the request pipeline. Correct diagnosis requires identifying where the clock ran out before changing any configuration. The exact error messages differ by layer:

• Azure API Management: HTTP 504 Gateway Timeout, body: {"statusCode": 504, "message": "Backend service timeout"}
• Azure API Management (forward-request): Microsoft.Azure.ApiManagement.Gateway.BackendRequestException: The request to the backend service timed out.
• Azure Functions (Consumption plan): Microsoft.Azure.WebJobs.Host.FunctionTimeoutException: Timeout value of 00:05:00 exceeded by function 'FunctionName'
• ARM REST API: {"code": "OperationTimeout", "message": "The operation 'Microsoft.Compute/virtualMachines/write' timed out"} — or a 202 Accepted that never transitions to Succeeded
• Azure Load Balancer: Silent TCP RST after 4 minutes of idle — no HTTP error code, connection simply drops
• Application Gateway / WAF: HTTP 502 Bad Gateway or HTTP 504 Gateway Timeout with x-appgw-trace-id header present

Step 1: Identify Which Layer Is Timing Out

Always capture full HTTP response headers before changing configuration.

1a. Inspect the Via and x-ms-request-id headers.

If the response contains Via: 1.1 apim-gateway-xxxxx (Azure API Management), the request reached APIM and the timeout occurred between APIM and your backend. If no Via header is present and you see a 504, the timeout happened upstream (Application Gateway or Load Balancer).

1b. Query APIM gateway logs in Log Analytics.

ApiManagementGatewayLogs
| where TimeGenerated > ago(1h)
| where ResponseCode == 504 or DurationMs > 28000
| project TimeGenerated, OperationName, DurationMs, BackendResponseCode, LastErrorReason, Url
| order by TimeGenerated desc
| take 20

A DurationMs clustering near 30000 confirms the default 30-second APIM backend timeout is the culprit. A DurationMs near 20000 with no backend response code points to Application Gateway.

1c. Check Application Insights for end-to-end traces.

requests
| where timestamp > ago(1h)
| where resultCode == "504" or duration > 25000
| project timestamp, name, url, duration, resultCode, cloud_RoleName
| order by timestamp desc

Step 2: Fix Azure API Management Timeout

The default backend timeout is 30 seconds and applies to all operations unless overridden. Fix at the operation, API, or global scope using the timeout attribute on <set-backend-service>.

Option A — Increase timeout (synchronous operations up to 240s):

<policies>
  <inbound>
    <base />
    <set-backend-service
      base-url="https://your-backend.example.com"
      timeout="120" />
  </inbound>
  <backend>
    <base />
  </backend>
  <outbound><base /></outbound>
  <on-error><base /></on-error>
</policies>

The timeout attribute is in seconds (integer). The platform maximum is 240 seconds. If you need longer, you must use an async pattern.

Option B — Async LRO pattern for operations over 4 minutes:

Kick off work asynchronously and return a 202 with a polling URL:

<inbound>
  <base />
  <send-request mode="new" response-variable-name="jobResponse" timeout="10">
    <set-url>https://your-backend.example.com/api/jobs</set-url>
    <set-method>POST</set-method>
    <set-body>@(context.Request.Body.As<string>(preserveContent: true))</set-body>
  </send-request>
  <return-response>
    <set-status code="202" reason="Accepted" />
    <set-header name="Location" exists-action="override">
      <value>@(((IResponse)context.Variables["jobResponse"]).Headers.GetValueOrDefault("Location", ""))</value>
    </set-header>
    <set-header name="Retry-After" exists-action="override">
      <value>5</value>
    </set-header>
  </return-response>
</inbound>

Clients poll the Location URL every Retry-After seconds until the status transitions to Succeeded or Failed.

Step 3: Fix Azure Function Timeout

Timeout limits are hard-enforced by plan tier:

Update host.json to extend timeout up to the plan ceiling:

{
  "version": "2.0",
  "functionTimeout": "00:10:00"
}

If your function requires more than 10 minutes, upgrade the plan:

# Upgrade to Premium EP1
az functionapp update \
  --name <function-app-name> \
  --resource-group <rg> \
  --plan-name <premium-plan-name>

For complex multi-step workflows, refactor to Durable Functions using the fan-out/fan-in pattern — this removes timeout concerns entirely because the orchestrator checkpoints state between activities.

Step 4: Fix Application Gateway Timeout

Application Gateway WAF v2 defaults to 20 seconds for the backend request timeout. Standard v2 defaults to 30 seconds. Update backend HTTP settings:

az network application-gateway http-settings update \
  --gateway-name <agw-name> \
  --resource-group <rg> \
  --name <backend-http-settings-name> \
  --timeout 120

# Verify the change
az network application-gateway http-settings show \
  --gateway-name <agw-name> \
  --resource-group <rg> \
  --name <backend-http-settings-name> \
  --query requestTimeout

Step 5: Handle ARM API Long-Running Operations

ARM returns HTTP 202 Accepted for operations that run longer than a few seconds. The response includes a polling URL:

HTTP/1.1 202 Accepted
Azure-AsyncOperation: https://management.azure.com/subscriptions/.../operations/abc123?api-version=2024-01-01
Retry-After: 30

Your client must poll Azure-AsyncOperation until status is Succeeded or Failed. Azure SDKs handle this automatically. For raw HTTP clients:

import requests, time

def poll_arm_lro(token, operation_url, retry_after=30, max_polls=40):
    headers = {"Authorization": f"Bearer {token}", "Content-Type": "application/json"}
    for _ in range(max_polls):
        resp = requests.get(operation_url, headers=headers, timeout=10)
        resp.raise_for_status()
        body = resp.json()
        status = body.get("status", "")
        if status == "Succeeded":
            return body
        if status in ("Failed", "Canceled"):
            err = body.get("error", {})
            raise RuntimeError(f"ARM LRO failed [{err.get('code')}]: {err.get('message')}")
        time.sleep(retry_after)
    raise TimeoutError(f"ARM operation did not complete after {max_polls} polls")

Step 6: Add a Retry Policy as a Safety Net

Transient 504 errors from backend cold starts or brief spikes should be retried before surfacing to callers. Add this APIM retry policy around the <forward-request> element:

<backend>
  <retry condition="@(context.Response.StatusCode == 504 || context.Response.StatusCode == 502)"
         count="3"
         interval="2"
         max-interval="10"
         delta="2"
         first-fast-retry="false">
    <forward-request timeout="60" />
  </retry>
</backend>

This retries up to 3 times with exponential backoff (2s, 4s, 8s) without masking genuine persistent timeouts.

Verification

After applying fixes, run a targeted latency test:

curl -v -w "\nDNS: %{time_namelookup}s | Connect: %{time_connect}s | TTFB: %{time_starttransfer}s | Total: %{time_total}s\n" \
  --max-time 150 \
  -H "Ocp-Apim-Subscription-Key: <your-key>" \
  https://<your-apim>.azure-api.net/<api-path>

Then confirm in Application Insights that p95 latency is below your new timeout threshold:

requests
| where timestamp > ago(30min)
| where url contains "<api-path>"
| summarize p50=percentile(duration,50), p95=percentile(duration,95), p99=percentile(duration,99)
  by bin(timestamp, 5m)
| render timechart