Error Medic

GitLab CI Timeout: Fix Job Timeouts, Permission Denied & Pipelines Not Working

Resolve GitLab CI timeout errors and permission denied failures: increase job timeout in .gitlab-ci.yml, fix runner Docker permissions, and debug stuck pipeline

Last updated:
Last verified:
2,369 words
Key Takeaways
  • GitLab CI jobs time out after 1 hour by default — increase per-job with 'timeout: 3h' in .gitlab-ci.yml or raise the project ceiling under Settings > CI/CD > General pipelines
  • Permission denied on scripts means the executable bit is missing from git — fix permanently with 'git update-index --chmod=+x script.sh'; Docker socket permission denied requires adding gitlab-runner to the docker group
  • Pipelines stuck in 'pending' almost always mean no runner is online, runner tags don't match the job, or the runner's max timeout is lower than the job needs — effective timeout is min(project_timeout, runner_max_timeout)
  • Enable CI_DEBUG_TRACE: 'true' as a CI/CD variable to get verbose step-by-step runner output for any silent failure or hang
Fix Approaches Compared
MethodWhen to UseTime to ImplementRisk
Add 'timeout: 3h' to job in .gitlab-ci.ymlSingle job consistently exceeds the 1h limit< 5 minLow
Raise project timeout in Settings > CI/CDAll jobs need a higher ceiling< 2 minLow
usermod -aG docker gitlab-runnerRunner reports 'permission denied' on /var/run/docker.sock5–10 minMedium — grants Docker daemon access
git update-index --chmod=+xScript exits with '/bin/bash: ./script.sh: Permission denied'< 5 minLow
Re-register runner as correct userRunner installed as root or wrong system user15–30 minLow
Mount Docker socket in config.toml volumesDocker builds inside jobs fail with socket errors10 minMedium — shared socket
CI_DEBUG_TRACE: 'true' variablePipeline hangs silently with no visible error< 5 minLow — exposes env vars in logs

Understanding GitLab CI Timeout, Permission Denied, and Pipeline Failures

GitLab CI/CD pipelines fail in predictable, fixable ways. This guide covers the three most common failure modes — job timeout, permission denied, and pipeline not triggering at all — with exact error messages, root cause analysis, and step-by-step remediation.


Part 1: GitLab CI Timeout

Exact error messages

When a job exceeds its configured timeout you will see one of the following in the job log:

ERROR: Job failed: execution took longer than 1h0m0s seconds
Job's activity exceeded the timeout
Running on runner-abc1234 via build-host-01
...
ERROR: Job failed (system failure): aborted: timeout
Root causes

Default 1-hour project timeout not adjusted. GitLab applies a 60-minute job timeout by default to every project. Large Rust or C++ compilations, slow integration test suites, multi-stage Docker builds with many layers, and database migration jobs frequently exceed this without any indication until they hit the wall.

Runner-level max timeout is lower than the job needs. Each registered runner has its own "Maximum job timeout" setting. The effective timeout for any job is min(project_timeout, runner_max_timeout). Raising the project setting to 3 hours does nothing if the runner is capped at 1 hour.

The job is hanging, not actually slow. Many apparent timeouts are really deadlocks: a test suite waiting on a network service that never responded, an interactive apt prompt blocking the script, or a parallel test runner that orphaned child processes.

Step 1: Determine whether the job is slow or hanging

Add coarse-grained timestamps to your script to pinpoint where time is spent:

build:
  script:
    - date && echo "[START] dependency install"
    - npm ci --prefer-offline
    - date && echo "[END] dependency install"
    - date && echo "[START] build"
    - npm run build
    - date && echo "[END] build"

If the log stops mid-step with no further output until the timeout fires, the job is hanging. Enable CI_DEBUG_TRACE for verbose executor-level output:

variables:
  CI_DEBUG_TRACE: "true"
Step 2: Increase the job-level timeout

Add timeout: directly under the job key in .gitlab-ci.yml. Accepted formats: 3h, 3h 30m, 210 minutes.

build-heavy:
  stage: build
  timeout: 3h
  script:
    - make all

This value cannot exceed the project-level ceiling or the runner's max timeout.

Step 3: Raise the project timeout ceiling

Go to Settings > CI/CD > General pipelines > Timeout and set a value that covers your worst-case job. For GitLab.com free tier, shared runners are hard-capped at 60 minutes. Paid tiers allow up to 3 hours on shared runners.

Step 4: Adjust the runner's max timeout

In Settings > CI/CD > Runners, expand the specific runner and update "Maximum job timeout". If you manage the runner yourself, you can also set this at registration time:

sudo gitlab-runner register \
  --url https://gitlab.com \
  --registration-token YOUR_TOKEN \
  --executor docker \
  --docker-image alpine:latest \
  --maximum-timeout 10800
Step 5: Fix hanging jobs

For package managers that prompt interactively, force non-interactive mode:

before_script:
  - export DEBIAN_FRONTEND=noninteractive
  - apt-get update -y && apt-get install -y curl git
  - npm ci --prefer-offline

For test suites, add explicit timeouts and a force-exit flag:

variables:
  JEST_TIMEOUT: "30000"
script:
  - npx jest --testTimeout=30000 --forceExit --bail

Part 2: GitLab CI Permission Denied

Exact error messages
/bin/bash: ./deploy.sh: Permission denied
dial unix /var/run/docker.sock: connect: permission denied
mkdir: cannot create directory '/cache': Permission denied
fatal: could not read Username for 'https://gitlab.com': No such device or address
Root causes

Script missing executable bit in git. Git tracks file permissions. If deploy.sh was added with git add without first running chmod +x, the file mode stored in git is 0644 (not executable). The runner checks out exactly what git has.

gitlab-runner user not in the docker group. On shell executor setups, the runner process runs as the gitlab-runner system user. By default this user is not in the docker group, so any command that opens /var/run/docker.sock fails immediately.

Artifact files owned by root. If a previous stage ran a Docker container that wrote files as uid 0, the next stage — running as gitlab-runner (typically uid 999 or similar) — cannot overwrite those files.

Missing deploy credentials for git push. CI jobs clone the repository read-only by default. Any script that calls git push without configuring credentials will fail with a permission error that looks like a network error.

Fix 1: Make scripts executable in git

This is the permanent fix — change the file mode stored in the git index:

git update-index --chmod=+x deploy.sh
git commit -m "fix: make deploy.sh executable"
git push

Verify the mode was stored correctly:

git ls-files -s deploy.sh
# Should show mode 100755, not 100644
Fix 2: Add gitlab-runner to the docker group

On the runner host:

sudo usermod -aG docker gitlab-runner
sudo systemctl restart gitlab-runner
# Verify membership
id gitlab-runner
# Expected output includes: groups=...,docker,...
# Spot-check access
sudo -u gitlab-runner docker ps
Fix 3: Mount the Docker socket in config.toml

Edit /etc/gitlab-runner/config.toml on the runner host:

[[runners]]
  name = "my-runner"
  executor = "docker"
  [runners.docker]
    image = "alpine:latest"
    volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"]

Restart after changes: sudo systemctl restart gitlab-runner

Fix 4: Correct artifact ownership for multi-stage pipelines

When a build stage uses Docker to produce artifacts, fix ownership before the artifacts block:

build:
  script:
    - docker run --rm -v "$PWD:/app" -w /app node:20 npm ci
    - docker run --rm -v "$PWD:/app" -w /app node:20 npm run build
    # Ensure next stage can read these files
    - sudo chown -R "$(id -u):$(id -g)" dist/ node_modules/
  artifacts:
    paths:
      - dist/
Fix 5: Git push from CI using a deploy token

Create a deploy token in Settings > Repository > Deploy tokens with write_repository scope, then store it as a masked CI/CD variable:

before_script:
  - git remote set-url origin
      "https://deploy-token:${CI_DEPLOY_PASSWORD}@${CI_SERVER_HOST}/${CI_PROJECT_PATH}.git"
  - git config user.email "ci@example.com"
  - git config user.name "GitLab CI"

Part 3: GitLab CI Not Working — Pipeline Stuck or Not Triggering

Symptom: Pipeline stays in 'Pending' indefinitely

This is almost always a runner availability problem. Check in order:

  1. Settings > CI/CD > Runners — is there at least one runner with a green online dot?
  2. If no online runners, SSH to the runner host and check the service:
    sudo systemctl status gitlab-runner
    sudo gitlab-runner verify
    
  3. Check for tag mismatch — if your job specifies tags:, the runner must have matching tags AND "Run untagged jobs" must be disabled or the tags must match exactly (case-sensitive).
Symptom: Jobs never appear despite push

Validate your .gitlab-ci.yml against the GitLab lint API:

curl --silent \
  --header "PRIVATE-TOKEN: $GITLAB_TOKEN" \
  --header "Content-Type: application/json" \
  --data "{\"content\": $(cat .gitlab-ci.yml | python3 -c 'import sys,json; print(json.dumps(sys.stdin.read()))')}" \
  "https://gitlab.com/api/v4/ci/lint" | python3 -m json.tool

Also check for rules: or only: blocks that never evaluate to true for your branch. A common mistake:

# Wrong — pushes to 'master' never match
deploy:
  rules:
    - if: '$CI_COMMIT_BRANCH == "main"'

# Correct — uses the project's configured default branch
deploy:
  rules:
    - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'

Frequently Asked Questions

bash
#!/usr/bin/env bash
# gitlab-ci-diagnose.sh — run on the GitLab Runner host
# Diagnoses timeout, permission denied, and pipeline-not-working issues

set -euo pipefail

HEADER() { echo ""; echo "=== $* ==="; }

HEADER "GitLab Runner Service Status"
sudo systemctl status gitlab-runner --no-pager -l

HEADER "Registered Runners"
sudo gitlab-runner list 2>&1

HEADER "Runner Connectivity (verify against GitLab)"
sudo gitlab-runner verify 2>&1

HEADER "Runner User Identity"
id gitlab-runner

HEADER "Docker Group Membership"
getent group docker || echo "docker group does not exist"

HEADER "Docker Socket Permissions"
ls -la /var/run/docker.sock

HEADER "Can gitlab-runner Access Docker?"
if sudo -u gitlab-runner docker info > /dev/null 2>&1; then
  echo "OK: gitlab-runner can reach Docker daemon"
else
  echo "FAIL: permission denied on Docker socket"
  echo "FIX:  sudo usermod -aG docker gitlab-runner && sudo systemctl restart gitlab-runner"
fi

HEADER "Runner Configuration"
sudo cat /etc/gitlab-runner/config.toml

HEADER "Effective Timeout Settings (from config.toml)"
sudo grep -E 'maximum_timeout|timeout' /etc/gitlab-runner/config.toml || echo "No explicit timeout set (runner default applies)"

HEADER "Disk Space (full disk causes silent failures)"
df -h

HEADER "Recent Runner Logs (last 60 lines)"
sudo journalctl -u gitlab-runner -n 60 --no-pager

HEADER "Git Executable Bits in .gitlab-ci.yml scripts (local repo)"
if [ -f .gitlab-ci.yml ]; then
  # Extract script paths that start with ./ and check their git mode
  grep -oP '\./.+\.sh' .gitlab-ci.yml | sort -u | while read -r script; do
    mode=$(git ls-files -s "$script" 2>/dev/null | awk '{print $1}')
    if [ "$mode" = "100644" ]; then
      echo "WARN: $script is NOT executable in git (mode 100644)"
      echo "FIX:  git update-index --chmod=+x $script"
    elif [ "$mode" = "100755" ]; then
      echo "OK:   $script is executable (mode 100755)"
    else
      echo "INFO: $script not tracked by git or not found"
    fi
  done
else
  echo "No .gitlab-ci.yml found in current directory"
fi

HEADER "YAML Validation"
if command -v python3 &>/dev/null && python3 -c 'import yaml' 2>/dev/null; then
  python3 - <<'PYEOF'
import yaml, sys
try:
    with open('.gitlab-ci.yml') as f:
        doc = yaml.safe_load(f)
    jobs = [k for k,v in doc.items() if isinstance(v, dict) and 'script' in v]
    print(f'OK: .gitlab-ci.yml is valid YAML with {len(jobs)} job(s): {", ".join(jobs)}')
except FileNotFoundError:
    print('SKIP: .gitlab-ci.yml not found in current directory')
except yaml.YAMLError as e:
    print(f'FAIL: YAML parse error: {e}')
    sys.exit(1)
PYEOF
else
  echo "Skipping — install python3-yaml: sudo apt-get install -y python3-yaml"
fi

echo ""
echo "Diagnostic complete."
E

Error Medic Editorial

The Error Medic Editorial team is composed of senior DevOps and SRE engineers with hands-on experience managing GitLab CI/CD at scale across cloud-native, on-premises, and hybrid environments. Our guides are written from production incident retrospectives and peer-reviewed for technical accuracy before publication.

Sources

Related Guides