Docker Desktop 4.70.0 httpproxy severs long HTTPS CONNECT tunnels mid-upload (breaks large docker push)

[mikerross]

Docker Desktop 4.70.0 httpproxy severs long HTTPS CONNECT tunnels mid-upload (breaks large docker push)

Version

• Docker Desktop: 4.70.0 (Docker Engine 29.4.0, Client 29.4.0, API 1.54)
• macOS: 26.4.1 (Tahoe, Build 25E253, Darwin 25.4.0)
• Hardware: Intel Core i9-9980HK (x86_64)

Summary

Docker Desktop's internal httpproxy component (running on the VM gateway at 192.168.65.1:3128) silently closes the write pipe for long-lived HTTPS CONNECT tunnels after roughly 7–10 minutes or ~500–800 MB of data transferred. This breaks docker push and docker buildx build --push for any image containing a large layer.

The dockerd daemon inside the VM reports:

write tcp 192.168.65.3:<port>->192.168.65.1:3128: write: broken pipe

…which surfaces to the user as either use of closed network connection (buildx) or failed to copy: failed to do request: Put <registry-url> (classic push).

This is a regression — the same workflow worked on Docker Desktop 4.69.x (engine 29.3.0) against the same registry over the same network.

Prior art

• 'Broken Pipe' Error When Pushing Image #7349 ('Broken Pipe' Error When Pushing Image, 4.31.0, closed) — same exact error fingerprint (write tcp 192.168.65.3:<port>->192.168.65.1:3128: write: broken pipe) reported against Docker Hub. Closed without a real fix because the reporter switched networks and it went away. That report was never properly root-caused; this is likely the same underlying httpproxy pipe-handling bug resurfacing in 4.70.0.
• Push to registry keeps re-trying then fails (Mac OS Intel 12.7.1) #7077 (Push to registry keeps re-trying then fails, 4.25.0, open) — similar symptom family (push retries then fails), no diagnostic trail, still open and untriaged.

Filing this new issue rather than reopening #7349 because (a) different version, (b) we have a confirmed bypass workaround, (c) we have a specific reproducer and narrowed root cause to the httpproxy component.

Expected behavior

docker push of a multi-GB image to a remote registry completes successfully, limited only by upload bandwidth.

Actual behavior

1. Small blobs (config, manifests, <~100 MB layers) push successfully.
2. Large blobs (~500 MB+) fail after 7–10 minutes regardless of the actual available bandwidth.
3. httpproxy.log reports the CONNECT tunnel as successful after <N>m with <bytes> transferred — but the byte count never reaches the full layer size, and dockerd sees a broken pipe at the same wall-clock moment.
4. Separately (likely a distinct bug in the 29.4.0 client), progress output for large blobs stays on Waiting indefinitely — the Pushing <percent>% lines never appear. This isn't the root cause but makes the problem look worse than it is during diagnosis.

Minimal reproduction

# Generate a fresh 1 GB random-data image (won't dedupe in any registry)
mkdir -p /tmp/push-test && cd /tmp/push-test
dd if=/dev/urandom of=blob.bin bs=1m count=1024
cat > Dockerfile <<EOF
FROM scratch
COPY blob.bin /
EOF
docker buildx build --platform linux/amd64 --load -t <your-registry>/push-test:latest .

# Push — will fail after 7–10 min with broken pipe
docker push <your-registry>/push-test:latest

Expected: completes in <image-size> / <upload-bandwidth> seconds.
Actual: fails with broken pipe or closed network connection before completion.

Registry used when filing: Oracle Cloud Infrastructure Registry (phx.ocir.io). Presumably reproduces against any remote registry; please test others.

Evidence

dockerd client error

failed to copy: failed to do request: Put "https://<registry>/v2/.../blobs/uploads/<uuid>?digest=sha256:...":
  write tcp 192.168.65.3:61564->192.168.65.1:3128: write: broken pipe

httpproxy log (~/Library/Containers/com.docker.docker/Data/log/host/httpproxy.log)

req 443 HTTP CONNECT <registry>:443: successful after 6m46.966798846s with 536301749 bytes transferred (0 remaining in-progress HTTP requests)

The "successful after" entry appears at the exact moment dockerd reports the broken pipe, with only 536 MB transferred of a 1 GB layer.

dockerd VM log

Shows matching use of closed network connection entries for grpc and api-proxy streams at the same timestamp.

Workaround

Add the destination registry to Docker Desktop's Containers Proxy bypass list:

1. Settings → Resources → Proxies
2. Toggle Manual proxy configuration ON
3. Under Containers Proxy (not "Docker Desktop Proxy")
4. Leave HTTP/HTTPS fields blank
5. In Bypass proxy settings for these hosts & domains, enter the registry hostname(s), e.g. *.ocir.io,phx.ocir.io
6. Apply & Restart

After this, httpproxy.log emits container via direct connection because matches container settings exclude and dockerd connects directly to the registry without routing through 192.168.65.1:3128. The same 1 GB push completes in ~5 min 20 s on the same machine / network / registry.

What's NOT the cause (ruled out during investigation)

I spent several hours narrowing this down; hopefully saves triage time:

• Registry outage / rate limiting — Ruled out: small HTTPS requests to same registry (<200 ms TTFB) and image pulls succeed during the failing window. Other laptops in the same region push fine.
• Buildx / BuildKit specifically — Ruled out: classic docker push (no buildx, no exporter) has the same failure. It's daemon-wide, not buildx-specific.
• gvisor VM network stack — Ruled out: created a buildkit container with --driver-opt network=host (bypasses gvisor for the container); same failure pattern persisted at the httpproxy layer.
• macOS Tahoe ECN default (net.inet.tcp.ecn_initiate_out=1) — Ruled out: sudo sysctl -w net.inet.tcp.ecn_initiate_out=0 did not change the failure.
• macOS TCP stack broadly — Ruled out: curl uploads from the Mac host (bypassing Docker entirely) to Cloudflare complete successfully; SSH uploads to a VM in the same region complete successfully. Only traffic flowing through dockerd → Docker Desktop httpproxy → destination fails.
• Upload bandwidth limit — Ruled out: Cloudflare speed-test upload at 11 Mbps works cleanly; no ISP throttle.
• Docker Desktop state corruption — Ruled out: quit/relaunch Docker Desktop, restart Mac; same failure.

Signal for the fix

The bypass workaround confirms the bug is specifically in how httpproxy handles long HTTPS CONNECT tunnels when it's in the path. It's not a general networking issue in Docker Desktop — only this specific code path. A likely candidate area is an idle-timer or write-buffer limit in the CONNECT-tunnel handling code that regressed between 4.69.x and 4.70.0.

Happy to provide additional diagnostics or test a candidate fix.