Bandwidth

What this layer does

Phase 12 keeps push and pull cheap. Tiny files go raw. Mid-size files get gzipped before upload with Content-Encoding: gzip. Large files hit a hard 10 MB ceiling that surfaces as a clear rejection. Append-only ndjson logs are eligible for tail-only delta uploads (v2 feature; v1 just gzips and re-uploads).

Files involved

state/bin/sync/gzip-helper.sh — gzip wrapper invoked by

bin/cli.js push for files in the 100 KB - 10 MB band.

bin/cli.js — enforces the 10 MB ceiling and the gzip threshold.
~/projects/snappy-skills/src/push.ts — Worker side; honors

Content-Encoding: gzip on upload, stores compressed.

state/bin/sync/sync-rules.json — append-only ndjson allowlist

used by the v2 tail-only future feature.

Size bands

Size	Treatment
<100 KB	Raw `POST /_push`, negligible cost
100 KB - 10 MB	gzip body before push, `Content-Encoding: gzip` header to Worker, Worker stores compressed
>10 MB	REJECT with clear error: "file too large for snappy-os sync; should this be in `sources/`?"

Append-only ndjson special case

Files in SYNC_ALLOW that are append-only (changelog.ndjson, evals.aggregate.ndjson, snapshots.ndjson) are good candidates for tail-only delta uploads. v2 feature: ship a sidecar <file>.last-pushed-offset so push only uploads the new tail.

v1 just gzips and re-uploads the full file. Acceptable until any single ndjson exceeds ~1 MB; the gzip ratio on ndjson is typically 8-12x so 1 MB compresses to ~100 KB and stays under the negligible-cost band.

Operational gotchas

The 10 MB hard ceiling exists because Worker request bodies have

practical limits and because anything that large probably belongs in a different store (DO Spaces direct via Robert admin path, R2, or sources/ if it's content the human owns).

Gzip threshold of 100 KB is a balance: smaller files don't benefit

enough to justify the CPU cost; larger files start showing visible egress savings.

Worker MUST serve gzipped responses with the same

Content-Encoding: gzip header so client pull can decompress on the way down. Mixing compressed-store with uncompressed-serve burns bandwidth.

The error message on rejection MUST mention sources/ as the

alternative. Without that hint, Joes hit the ceiling and don't know the workaround.

v1's full-file ndjson re-upload becomes a problem if any tenant's

evals.aggregate.ndjson grows past 1 MB. Monitor and prioritize the v2 tail-only feature accordingly.

How to verify it's working

A 50 KB push uploads raw; state/log/sync-events.ndjson

shows bytes close to file size.

A 500 KB push uploads gzipped; bytes shows compressed size,

Worker wrangler tail shows Content-Encoding: gzip on the request.

An 11 MB push fails fast with the documented error message before

any network call.

A pull of a 500 KB file decompresses correctly on the client side

and the local SHA matches the manifest entry.

Sustained push/pull cost stays under DO Spaces' $5/mo flat for the

expected v1 tenant count.