Azure API Timeout: Fixing 'The request timed out' and 408/504 Errors in Azure APIs

Understanding Azure API Timeout Errors

Azure surfaces timeout failures across several layers, each with distinct error messages and HTTP status codes:

• 408 Request Timeout — the client or an upstream proxy gave up waiting before the server sent a full response.
• 504 Gateway Timeout — Azure's front-end gateway (ARM, APIM, Application Gateway, or Front Door) forwarded your request but the backend did not reply within the gateway's limit.
• CloudException: The request timed out. — thrown by the Azure .NET SDK when HttpClient.Timeout (default 100 seconds) elapses.
• azure.core.exceptions.ServiceResponseError: ('Connection aborted.', timeout('timed out')) — Python SDK equivalent.
• Error: connect ETIMEDOUT / Error: socket hang up — Node.js @azure/arm-* SDK or axios-based calls.
• DeploymentFailed: The resource operation completed with terminal provisioning state 'Failed'. Error: Gateway Timeout — ARM deployment calling a slow custom script extension or external endpoint.

The Azure Timeout Stack

Requests traverse multiple timeout boundaries before reaching your backend:

Client SDK timeout (default: 100 s, HttpClient)
  └─ Azure API Management timeout (default: 240 s, configurable)
      └─ Azure Application Gateway / Front Door (default: 60–600 s)
          └─ ARM gateway synchronous limit (60 s hard)
              └─ Your backend / Function App / Container

A timeout fires at whichever boundary is hit first. Increasing only the APIM policy helps nothing if the client SDK times out at 30 seconds.

Step 1: Identify Which Layer Is Timing Out

Check the HTTP status code and response headers.

An APIM-generated timeout includes x-ms-request-id and an APIM policy error body:

{
  "statusCode": 504,
  "message": "The backend service did not respond in time."
}

An ARM gateway timeout looks like:

{
  "error": {
    "code": "GatewayTimeout",
    "message": "The gateway did not receive a response from the backend service within the time period."
  }
}

A client-side SDK timeout (C#):

Microsoft.Rest.HttpOperationException: Operation returned an invalid status code 'RequestTimeout'
  ---> System.Threading.Tasks.TaskCanceledException: A task was canceled.

Use curl -v with --max-time to isolate the layer:

# Test direct backend (bypassing APIM)
curl -v --max-time 120 \
  -H "Authorization: Bearer $(az account get-access-token --query accessToken -o tsv)" \
  https://management.azure.com/subscriptions/{sub}/resourceGroups/{rg}/providers/Microsoft.Web/sites/{name}?api-version=2022-03-01

# Test through APIM gateway
curl -v --max-time 120 \
  -H "Ocp-Apim-Subscription-Key: {key}" \
  https://{apim-name}.azure-api.net/{api-path}

If the direct call succeeds and the APIM call times out, the issue is in APIM policy or backend URL configuration. If both fail, the problem is in the backend itself.

Check Azure Monitor / Application Insights for the request duration:

# Query Application Insights via Azure CLI
az monitor app-insights query \
  --app {app-insights-name} \
  --resource-group {rg} \
  --analytics-query "
    requests
    | where timestamp > ago(1h)
    | where resultCode in ('408','504')
    | summarize count(), avg(duration), max(duration) by operation_Name
    | order by max_duration desc
  "

Step 2: Fix APIM Backend Timeout

Azure API Management has a default backend timeout of 240 seconds set at the API level. Individual operations can override this. The maximum configurable value is 240 seconds for the Consumption tier and up to 3600 seconds (1 hour) for Developer, Basic, Standard, and Premium tiers.

In the APIM policy editor, add or modify the <backend> section:

<policies>
  <inbound>
    <base />
  </inbound>
  <backend>
    <!-- Increase timeout to 3 minutes for slow analytics endpoints -->
    <forward-request timeout="180" follow-redirects="true" />
  </backend>
  <outbound>
    <base />
  </outbound>
  <on-error>
    <base />
  </on-error>
</policies>

Deploy via Azure CLI:

az apim api operation policy create \
  --resource-group {rg} \
  --service-name {apim-name} \
  --api-id {api-id} \
  --operation-id {operation-id} \
  --policy-format xml \
  --value @policy.xml

Step 3: Fix Azure Function Timeout

Azure Functions timeouts are controlled by functionTimeout in host.json:

host.json change:

{
  "version": "2.0",
  "functionTimeout": "00:10:00"
}

For Premium/Dedicated plans, set "functionTimeout": "-1" (no timeout) or a specific duration like "02:00:00".

Redeploy after the change:

func azure functionapp publish {function-app-name} --build remote

Step 4: Fix Client-Side SDK Timeout (C# / .NET)

The Azure SDK for .NET uses HttpClient under the hood. The default HttpClient.Timeout is 100 seconds. When using DefaultAzureCredential and ARM management clients, set timeout via RetryOptions or a custom HttpPipeline:

var options = new ArmClientOptions();
options.Retry.MaxRetries = 3;
options.Retry.Delay = TimeSpan.FromSeconds(2);
options.Retry.MaxDelay = TimeSpan.FromSeconds(30);
options.Retry.Mode = RetryMode.Exponential;

// For custom network timeout, inject HttpClientTransport
var httpClient = new HttpClient { Timeout = TimeSpan.FromSeconds(300) };
var transport = new HttpClientTransport(httpClient);
options.Transport = transport;

var client = new ArmClient(new DefaultAzureCredential(), subscriptionId, options);

Step 5: Handle Long-Running Operations (LRO) Correctly

ARM operations that take longer than 60 seconds must follow the LRO pattern. ARM will return a 202 Accepted with a Location or Azure-AsyncOperation header. You must poll this URL until the state is Succeeded or Failed.

Azure SDKs handle this automatically via WaitForCompletionAsync():

// Correct: await the full LRO
var operation = await resourceGroupCollection.CreateOrUpdateAsync(
    WaitUntil.Completed,   // <-- polls until done
    resourceGroupName,
    new ResourceGroupData(AzureLocation.EastUS));

var resourceGroup = operation.Value;

If you call WaitUntil.Started (or the REST API directly), you receive a 202 and must poll manually:

# Poll the async operation URL
STATUS_URL=$(curl -sI -X PUT \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"location":"eastus"}' \
  "https://management.azure.com/subscriptions/{sub}/resourceGroups/{rg}?api-version=2021-04-01" \
  | grep -i 'azure-asyncoperation:' | awk '{print $2}' | tr -d '\r')

while true; do
  STATE=$(curl -s -H "Authorization: Bearer $TOKEN" "$STATUS_URL" | jq -r '.status')
  echo "State: $STATE"
  [[ "$STATE" == "Succeeded" || "$STATE" == "Failed" ]] && break
  sleep 10
done

Step 6: Verify Application Gateway / Front Door Timeouts

If requests pass through Azure Application Gateway, its default backend request timeout is 30 seconds. Increase it:

# Update Application Gateway backend HTTP settings timeout
az network application-gateway http-settings update \
  --gateway-name {gateway-name} \
  --resource-group {rg} \
  --name {http-settings-name} \
  --timeout 120

For Azure Front Door (Standard/Premium), the origin response timeout defaults to 60 seconds and is configurable up to 240 seconds in the origin group settings.