Keep an API error body while failing a curl job

John Burns

An API job can fail in two different ways at once: the HTTP status says the request was rejected, and the response body contains the only useful explanation. A plain curl --fail handles the first part well. It returns a nonzero status for an HTTP response of 400 or higher, so a shell script, timer, or CI job does not report success. By design, though, it does not save the response body. That can leave an operator with an exit code but no server-provided error identifier, retry advice, or validation detail.

--fail-with-body makes a narrower trade-off. It preserves curl’s failure status while allowing the response body to be written to an explicit file. The file is diagnostic evidence, not a success payload and not something a later step should parse as if the request completed.

A disposable local validation using curl 8.14.1 on Linux AMD64 compared the two modes against a synthetic JSON response with status 403. --fail returned exit 22 and did not create its output file. --fail-with-body returned the same exit 22 while saving the synthetic error JSON. Against a 200 response, --fail-with-body returned zero and saved the expected success JSON. The paired result matters because it proves that preserving a body does not turn an HTTP rejection into a successful job.

This is a useful pattern for an approved API endpoint where a small error body is safe to retain. Do not apply it blindly to an endpoint that can return access tokens, personal data, customer records, large HTML error pages, or internal topology. Decide where error output belongs before adding it to unattended automation.

Keep the exit status as the decision

For a request whose successful response is useful to a later step, start with an explicit output location and capture the HTTP status separately:

response_file=<approved-output-file>
status=$(curl --silent --show-error --fail-with-body \
  --output "$response_file" \
  --write-out '%{http_code}' \
  <approved-url>)

--silent suppresses the progress meter, while --show-error keeps curl’s own diagnostic visible when the transfer fails. --output directs the response body to a file rather than mixing it with shell output. --write-out prints the final HTTP response code after the transfer, so the command substitution records a small status value instead of trying to infer success from the body.

The command’s exit status remains the primary gate. In a script that must stop on a failed request, do not hide that status by placing the request in an unchecked command substitution or by ending the script with a successful logging command. Use an explicit branch:

response_file=<approved-output-file>
if ! status=$(curl --silent --show-error --fail-with-body \
  --output "$response_file" \
  --write-out '%{http_code}' \
  <approved-url>); then
    printf 'request failed with HTTP status %s\n' "$status" >&2
    exit 1
fi

printf 'request succeeded with HTTP status %s\n' "$status"

The ! makes the failure branch clear, but it also means $? inside that branch describes the negated test rather than curl’s original result. Record the HTTP status and any restricted curl error output instead of trying to reconstruct an exact curl exit code after more commands have run. If the caller needs differentiated handling for timeout, DNS, TLS, and HTTP errors, run curl without !, save its exit status immediately, and branch on that saved value.

A successful curl transfer does not by itself prove that the response contains the expected application result. For a JSON API, validate the schema or the required field before treating the file as input. Do that only on the success path; an error document can also be valid JSON.

Reproduce the two failure modes with synthetic data

The validation uses a tiny disposable HTTP responder with two paths: one returns a JSON success document and the other returns JSON with HTTP 403. Keep the listener confined to an isolated test boundary and use invented data. The following commands assume that <test-url> is the base URL of such a responder.

First, compare ordinary --fail with a rejected response:

curl --silent --show-error --fail \
  --output fail-body.json \
  --write-out 'status=%{http_code} exit=%{exitcode}\n' \
  <test-url>/denied

The validation returned status=403 exit=22. No fail-body.json file was created. That is the expected behavior for --fail: curl treats the HTTP failure as an error and does not preserve the body. A job that only needs a pass/fail signal may prefer this lower-retention behavior.

Now change only the failure option:

curl --silent --show-error --fail-with-body \
  --output fail-with-body.json \
  --write-out 'status=%{http_code} exit=%{exitcode}\n' \
  <test-url>/denied

The same synthetic response returned status=403 exit=22, and fail-with-body.json contained the responder’s error document. The result is deliberately two-part: the body is available for a restricted diagnostic record, while exit 22 still prevents the script from advancing to a deployment, deletion, or success notification.

Finally, send a request that is meant to succeed:

curl --silent --show-error --fail-with-body \
  --output ok.json \
  --write-out 'status=%{http_code} exit=%{exitcode}\n' \
  <test-url>/ok

The local check returned status=200 exit=0 and wrote the synthetic success JSON to ok.json. This confirms the intended distinction: a successful HTTP result creates usable output and a zero status; an HTTP rejection can retain a body but still fails the command.

Do not key automation on the exact English text of curl’s error message. The durable interface is the nonzero exit status, the HTTP status recorded with --write-out, and a deliberately bounded response file. Authentication exchanges deserve additional care. curl documents that --fail is not fail-safe for every 401 or 407 scenario, so test the actual authentication flow and avoid logging authorization headers, cookies, credentials, or unredacted response bodies.

Bound the diagnostic artifact

An error body can be more sensitive than an exit code. Place it in a job-specific directory with restrictive permissions, give it a short retention period, and do not upload it as a public CI artifact. Before saving it, consider the largest expected response and whether a proxy or application can return a verbose stack trace. --max-filesize can limit a declared response size, but it is not a complete storage policy and may not protect every transfer mode. For a strict API contract, add an application-level size check before copying the body into long-lived evidence.

Keep normal output and error output separate. --output captures the HTTP body; curl diagnostics still go to standard error when --show-error is present. Redirect standard error only to an approved restricted log. Never solve a missing error body by removing --fail, appending || true, or allowing a later command to overwrite curl’s result. Each of those changes can make a rejected API call appear healthy.

The operational boundary is simple: --fail-with-body is for a response that must remain available for diagnosis without weakening failure handling. The validation showed the rejected response remained an exit-22 failure while its synthetic JSON body was retained, and the successful response remained exit zero. Put that distinction in the job before the response file reaches any workflow that assumes success.

Sources