Compressing HTTP Request Bodies

How request compression differs from response compression

Response compression is negotiated: the client lists what it can decode in Accept-Encoding and the server picks. For request bodies there is no equivalent handshake before the request is sent. A client that sends Content-Encoding: gzip on a POST is betting that the server can decompress it.

RFC 7694 fills part of the gap. A server that receives a request body in a coding it does not support should answer 415 Unsupported Media Type, and it may include an Accept-Encoding header in the response to list the codings it does accept for request bodies. Clients can use that to retry or to remember the server's capability. Many APIs simply document whether they accept compressed uploads instead.

When it is worth doing

• Large JSON, XML, CSV or NDJSON uploads: telemetry batches, log shipping, bulk imports, SOAP payloads.
• Mobile or metered clients where upstream bandwidth is the bottleneck.
• Server-to-server pipelines that both sides control.

For small requests (a few hundred bytes) the CPU cost and the risk of incompatibility outweigh the savings. Binary uploads that are already compressed (images, video, ZIP files) gain nothing.

Sending a compressed body

curl

gzip -c payload.json > payload.json.gz
curl -X POST https://api.example.com/ingest \
  -H "Content-Type: application/json" \
  -H "Content-Encoding: gzip" \
  --data-binary @payload.json.gz

Use --data-binary, not -d: -d strips newlines and carriage returns from file input and corrupts the gzip stream.

Python

import gzip, json, requests

body = gzip.compress(json.dumps(data).encode("utf-8"))
r = requests.post(
    "https://api.example.com/ingest",
    data=body,
    headers={"Content-Type": "application/json", "Content-Encoding": "gzip"},
)

Node.js

import { gzipSync } from "node:zlib";

const res = await fetch("https://api.example.com/ingest", {
  method: "POST",
  headers: { "Content-Type": "application/json", "Content-Encoding": "gzip" },
  body: gzipSync(JSON.stringify(data)),
});

Load-testing tools

k6 can compress the request body for you with the compression request parameter (for example compression: "gzip"), which also sets Content-Encoding. Setting 'Accept-Encoding': 'gzip' in a k6 or Postman request only affects the response; it does not compress what you send.

Decompressing on the server

# Apache: accept gzip-compressed uploads on /ingest
<Location "/ingest">
    SetInputFilter DEFLATE
</Location>

// Go: decompress if needed, and cap the decompressed size
func handler(w http.ResponseWriter, r *http.Request) {
    var body io.Reader = r.Body
    if strings.EqualFold(r.Header.Get("Content-Encoding"), "gzip") {
        zr, err := gzip.NewReader(r.Body)
        if err != nil {
            http.Error(w, "request body is not valid gzip", http.StatusBadRequest)
            return
        }
        defer zr.Close()
        body = io.LimitReader(zr, 10<<20) // 10 MiB after decompression
    }
    // decode JSON from body ...
}

Why the server answers 400 or 415

• The header says gzip but the body is not. The most common bug: the code sets Content-Encoding: gzip but sends plain JSON, or a library compresses the body a second time. The server's decompressor fails and returns 400 Bad Request, often with a message such as "request body is not valid gzip".
• The server does not support request compression. It may try to parse gzip bytes as JSON (400, JSON parse error) or reject the coding outright (415).
• The body was corrupted in transit or in the client. Text-mode file reads, string conversions and curl -d all mangle binary data.
• A proxy or WAF in front of the application inspects or rewrites the body, or strips Content-Encoding while leaving the compressed bytes.
• Wrong format under the right name. Raw DEFLATE sent as deflate, or zlib data sent as gzip. Check the first bytes: gzip starts with 1f 8b.

A quick way to check what you are actually sending is to point the client at a request-inspection endpoint you control and save the raw body, then run file or gzip -t on it. The decoding errors guide lists the equivalent checks for responses.

Security: limit what you decompress

A compressed request body can expand by a factor of a thousand or more. Any server that accepts compressed uploads must enforce a limit on the decompressed size, not just on the bytes received; otherwise a small request becomes a memory-exhaustion attack ("decompression bomb"). The Go example above shows the pattern with io.LimitReader; most frameworks expose a similar limit. See also the security notes in best practices.