S3-Compatible Backends: Requirements and Gaps#

The S3 backend addresses S3-compatible endpoints, not only AWS S3 — MinIO and other implementations are supported through custom endpoint configuration. “S3-compatible” is a spectrum, though, and the gaps show up as capability failures rather than connection failures.

Use this runbook when a backend connects and authenticates but some operations behave differently from AWS S3.

Symptom#

The endpoint is reachable and credentials work, but:

  • addressing fails for some buckets and not others,

  • version history is empty or errors,

  • conditional writes are rejected,

  • object keys past a certain length are refused,

  • copy or multipart operations fail where they succeed on AWS.

What the backend must provide#

Requirement

Why it matters

S3 signature authentication

Static keys, or an identity mechanism the adapter supports. Anonymous access is supported for public buckets.

Path or virtual addressing

Both are supported, but the endpoint must be declared with the right one — see below.

Object versioning, if version history is used

Version-aware operations depend on it. There is no runtime detection.

Conditional operations, if optimistic locking is used

Concurrency control depends on the backend honouring preconditions.

Multipart upload, for large objects

Large writes use multipart; a backend without it is limited to smaller objects.

Presigned URLs, for redirect reads and writes

Redirected transfers are fetched by the client directly from the endpoint.

Configuration that matters for clones#

Addressing style. When omitted, the style is inferred from the endpoint name: an endpoint name with a non-empty path uses path addressing, otherwise virtual. Most self-hosted S3-compatible servers require path addressing, and inference does not always produce it — so declare it explicitly rather than relying on the default:

config:
  storage:
    s3:
      endpoints:
        "minio.example.internal:9000":
          addressingStyle: "path"

Versioning. The adapter does not probe the backend for versioning support. Declare it, so a bucket without versioning fails fast with a clear precondition error instead of a confusing backend error:

versioningEnabled: false

Key length. The default maximum object key size matches the S3 specification. Backends that allow longer keys need the limit raised, otherwise writes with long keys are rejected by the adapter before they reach the backend.

Unconfigured endpoints. Strict mode refuses endpoints that are not declared. Leaving it permissive means a typo in an address is served as a new endpoint with default settings rather than refused — which is much harder to debug than an explicit rejection.

Known compatibility gaps#

These are behaviours where implementations diverge from AWS S3 in ways that surface as product failures. Treat this as the starting list to validate against any new backend.

Server-side copy with no request body. Some implementations reject a copy request that carries no body over HTTP/1.1, responding 411 Length Required, where AWS tolerates it. This has been observed against Google Cloud Storage’s S3-compatible endpoint. It presents as copy and move failing while read and write succeed.

Conditional-write semantics. Support for preconditions varies, and where the header is accepted but not enforced, a precondition that appears to pass has no effect. Concurrency control silently degrades rather than erroring, which makes this the most dangerous gap in this list: validate it deliberately rather than assuming.

ETag semantics. ETags are not guaranteed to be content hashes, and multipart-uploaded objects have composite ETags on AWS. Any logic that treats an ETag as an MD5 of the content is wrong on AWS and differently wrong on clones. Use them only as opaque validators.

Versioning completeness. A backend may accept versioning configuration while not implementing the full version listing surface, which presents as empty or partial history rather than an error.

Checksum and trailer support. Newer AWS SDK checksum behaviour is not implemented everywhere; mismatches appear as signature or checksum failures on write.

Triage#

# Does the endpoint answer, and with which addressing?
curl -sS -o /dev/null -w '%{http_code}\n' https://<endpoint>/<bucket>/

# What does the adapter say the backend refused?
kubectl -n <namespace> logs deployment/storage-service --since=15m | grep -i -E 's3|denied|411|signature|checksum'

Match the failing operation to the table above rather than treating it as a generic backend error — the adapter reports what the backend returned, and these gaps have distinctive responses.

Fix#

  • Declare addressingStyle explicitly for any non-AWS endpoint.

  • Declare versioningEnabled to match reality on that bucket.

  • Raise the key-length limit only if the backend genuinely supports longer keys.

  • Where a capability is absent, disable the feature that depends on it for that endpoint rather than leaving it to fail per request.

Verify#

Exercise each capability against the endpoint, not just read and write: stat, list, versioned read, conditional write, copy, move, multipart write, and a presigned redirect read. A backend that passes read and write can still fail four of the remaining six, and those failures reach users as product bugs rather than configuration errors.