Skip to content

ci(release): switch PyPI watch to JSON API instead of pip index#70

Merged
lukeocodes merged 1 commit intomainfrom
fix/pypi-watch-json-api
May 8, 2026
Merged

ci(release): switch PyPI watch to JSON API instead of pip index#70
lukeocodes merged 1 commit intomainfrom
fix/pypi-watch-json-api

Conversation

@lukeocodes
Copy link
Copy Markdown
Member

Problem

The "Wait for new deepctl version on PyPI" step in .github/workflows/release.yml polls pip index versions deepctl. That endpoint queries PyPI's simple index, which sits behind a CDN cache that can lag 5-15 minutes after a release is actually published.

Two consecutive releases hit this:

In both cases PyPI confirmed the release was installable (any user could pip install deepctl==X.Y.Z), but the cached index hadn't picked it up before the 10-minute window (30 × 20s) ran out. The bump-brew-formula job failed → we had to regenerate the formula manually.

Fix

GET https://pypi.org/pypi/deepctl/<version>/json — the PyPI JSON API endpoint flips to HTTP 200 the instant a release is published, no CDN cache between us and the source-of-truth catalog.

URL="https://pypi.org/pypi/deepctl/${VERSION}/json"
if [ "$(curl -fsS -o /dev/null -w '%{http_code}' "${URL}")" = "200" ]; then
  echo "deepctl==${VERSION} is live on PyPI"
  exit 0
fi

Same 30 × 20s loop shape, total worst-case budget unchanged at 10 minutes. Typical wait should drop from 5-15 minutes to a single iteration.

Verification

While diagnosing v0.2.22 today:

$ curl -fsS -o /dev/null -w "HTTP %{http_code}\n" https://pypi.org/pypi/deepctl/0.2.22/json
HTTP 200

$ pip index versions deepctl
deepctl (0.2.21)        # ← still didn't see 0.2.22

JSON API: live. Simple index: lagging. The fix is to probe the source of truth.

What it doesn't change

  • The 30 × 20s budget shape is preserved so we don't accidentally let bad releases through faster than the existing flow.
  • bump-brew-formula's downstream pipeline (homebrew-pypi-poet resource generation, etc.) is unchanged. That step's own pip install deepctl==<version> may still hit the simple-index lag for ~30-60s after this watch step succeeds, but that's a much smaller window and the install retry is implicit in the script's own behaviour.

The 'Wait for new deepctl version on PyPI' step in the release workflow
polled `pip index versions deepctl`, which queries PyPI's simple index
behind a CDN cache that can lag 5-15 minutes after a successful publish.
v0.2.21 (PR #3 on homebrew-tap) and v0.2.22 (PR #4 on homebrew-tap) both
hit this — PyPI confirmed the release was installable, but the cached
simple index hadn't picked it up by the time the 10-minute window
(30 × 20s) ran out, so bump-brew-formula failed and we had to re-run
the formula bump manually.

Switch to GET https://pypi.org/pypi/deepctl/<version>/json. That endpoint
flips to HTTP 200 the instant a release is published. Same 30 × 20s loop
shape, just a different probe — total runtime budget unchanged at
10 minutes worst case but the typical wait should drop to a single
iteration.

Verified locally that the JSON endpoint already returned 200 for 0.2.22
while `pip index versions` still missed it. No other changes needed —
release-please outputs and downstream jobs are untouched.
@lukeocodes lukeocodes merged commit 63903ea into main May 8, 2026
39 checks passed
@lukeocodes lukeocodes deleted the fix/pypi-watch-json-api branch May 8, 2026 19:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant