docs: port v0.16 upgrade guide onto refactored docs#818
Merged
alexluong merged 1 commit intofeat/refactor-docsfrom Apr 10, 2026
Merged
docs: port v0.16 upgrade guide onto refactored docs#818alexluong merged 1 commit intofeat/refactor-docsfrom
alexluong merged 1 commit intofeat/refactor-docsfrom
Conversation
Ports the v0.16 upgrade guide from #811 into the new Markdoc-based docs structure. Adapts "operation events" prose to "operator events" to match the naming decision on feat/refactor-docs; env vars remain OPERATION_EVENTS_* as defined in code. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
alexbouchardd
added a commit
that referenced
this pull request
Apr 15, 2026
* wip: Add new docs content * chore: Add new content, cleanup old content * chore: Finish nav and documents structure * chore: Update nav order * chore: Change docs root URL * fix: Fix tabs * chore: Document markdoc components * fix: Remove left-over use of old markdoc tags * feat: add redirects * docs: port v0.16 upgrade guide to refactored docs structure (#818) Ports the v0.16 upgrade guide from #811 into the new Markdoc-based docs structure. Adapts "operation events" prose to "operator events" to match the naming decision on feat/refactor-docs; env vars remain OPERATION_EVENTS_* as defined in code. Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add Kafka destination page (#821) * docs: add Kafka destination page Adds docs/content/destinations/kafka.mdoc mirroring the other destination pages, and wires it into nav.json. Overview and concepts pages already linked to this slug. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: note default partition key in table Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: Hookdeck Outpost quickstarts and agent onboarding prompt (#815) * docs: add Hookdeck Outpost managed quickstarts and agent prompt Add self-contained quickstarts for curl, TypeScript, Python, and Go against the managed API, with Settings → Secrets, env-based examples, and verification via Hookdeck Console and project logs. Nest Quickstarts nav under Hookdeck Outpost (above Self-Hosted) and add an agent prompt template page for dashboard copy/paste. Include TEMP-hookdeck-outpost-onboarding-status.md for GA tracking. Made-with: Cursor * docs: add Outpost agent evaluation harness and scenarios - Claude Agent SDK runner with explicit --scenario/--scenarios/--all, per-run workspace - Heuristic + LLM scoring vs scenario Success criteria; score-transcript 01-10 - Scenarios: basics, minimal apps, existing-app integration baselines - CI slice (eval:ci), SCENARIO-RUN-TRACKER, prompt template Files on disk guidance - Allow committing docs/**/.env.example under docs/.gitignore - TEMP status and README updates Made-with: Cursor * docs(agent-eval): prompt mapping, scenarios, harness; reset scenario tracker - Agent prompt: language implies SDK; simplest path defaults to curl; option 2/3 framework mapping; warn on sdks.mdx vs per-language quickstarts. - Curl quickstart: shell script notes (HTTP 202, portable body/status split). - run-agent-eval: PreToolUse write guard, default EVAL_MAX_TURNS 80, local docs block aligned with prompt; scenario heuristic fix for publish data key escaping. - Scenarios 01-10: realistic short user turns; success-criteria fixes where needed. - SCENARIO-RUN-TRACKER: cleared run results for a fresh pass; action items reset. - README and .env.example updates for eval harness as applicable. Made-with: Cursor * docs(agent-eval): record fresh scenario 01 eval run in tracker Made-with: Cursor * fix(agent-eval): remove harness-only 202/head hints from local docs block - Point local EVAL_LOCAL_DOCS guidance at full curl quickstart instead - Reword scenario 01 execution criteria to reference quickstart/OpenAPI * docs(agent-eval): update scenario 01 tracker after re-run and execution pass * docs(agent-eval): record scenario 02 run and execution pass * docs(agent-eval): fix tracker table formatting and artifact markdown * docs(agent-eval): record scenario 03 run and execution pass * docs(agent-eval): record scenario 04 run and execution pass * docs(agent-eval): record scenario 05 run and execution pass Made-with: Cursor * docs: Outpost mental model, UI guide agnostic URLs, agent prompt links - Expand concepts with SaaS/platform flow; refine building-your-own-ui (API root, paths, no localhost:3333 in examples) - Agent prompt: link concepts, UI guide, topics; tighten option-2 guidance - Eval harness: local docs list includes concepts, building-your-own-ui, topics - SCENARIO-RUN-TRACKER: scenario 05 assessment for 17-21-22 run, heuristic notes - Minor scenario 05 doc tweak Made-with: Cursor * docs(agent-eval): record scenario 06–07 runs and execution passes Made-with: Cursor * docs: fix List Topics UI example for string[] API response GET /topics returns a JSON array of topic names (OpenAPI). The React snippet incorrectly treated items as objects with id and name, which misled readers and agent integrations. Use the string as key, value, and label to match the API and TypeScript SDK (topicsList → Array<string>). Made-with: Cursor * feat(agent-eval): declarative pre-steps via Eval harness section - Add eval-harness.ts to parse eval-harness fenced JSON (git_clone + agentCwd). - Runner applies pre-steps per scenario, sets agent cwd and write guard to the run directory, passes scenario markdown once into runOneScenario. - Transcript meta includes evalHarness summary; document EVAL_SKIP_HARNESS_PRE_STEPS. Made-with: Cursor * docs(agent-eval): harness blocks for existing-app scenarios 08–10 - Add ## Eval harness JSON (git_clone + agentCwd) for Next.js, FastAPI, Go baselines. - Turn 1 stays in-user voice (repo present) without naming the eval harness. - Align Automated eval and success criteria with pre-cloned workspace model. Made-with: Cursor * chore(agent-eval): update SCENARIO-RUN-TRACKER for recent runs Made-with: Cursor * docs(pages): guide Option 3 full-stack Outpost integration Expand the copy-paste agent template so existing apps with a product UI wire backend (BFF, server SDK) and frontend (calls own API only). Point to Concepts and Building your own UI before destination screens; allow API-only path when there is no customer UI. Made-with: Cursor * docs(agent-eval): scenario 09 uses full-stack FastAPI template Pin scenario 09 to fastapi/full-stack-fastapi-template (React + Pydantic v2). Update scoreScenario09 baseline check, README index, TEMP onboarding status, and SCENARIO-RUN-TRACKER notes. Optional clone URL override: EVAL_FASTAPI_BASELINE_URL. Made-with: Cursor * docs(agent-eval): record scenario 09 re-eval after prompt update Run 2026-04-09T22-16-54-750Z-scenario-09: heuristic 6/6, LLM pass. Point execution notes to prior Docker smoke on 20-48 stamp. Made-with: Cursor * docs: scenario 09 tracker, agent prompt, BYO UI events/retry guidance Made-with: Cursor * docs: refresh Building Your Own UI guide Reword for customer-facing UI builders: clearer tenant/auth framing, configurable API base URL, less internal jargon and emphasis noise. Add implementation checklists for planning, destinations, activity, and safe rendering without duplicating the OpenAPI mapping tables. Made-with: Cursor * docs(eval): align scenarios 08–10, prompt, and heuristics - Agent prompt: topic reconciliation, domain vs test publish, full-stack UI guidance; remove eval-flavored Turn 0 / next-run wording in template. - score-transcript: publish_beyond_test_only for 08/09/10 (domain publish). - Scenarios + README: success criteria and Turn 1 nudges match prompt. - SCENARIO-RUN-TRACKER: scenario 09 review notes marked resolved. Made-with: Cursor * docs(eval): de-meta user turns in scenarios 8–10 Rewrite Turn 1 blockquotes as natural operator speech; drop Option 3, Turn 0, and prompt-section references. Align success-criteria wording with configured onboarding topics. Tracker references user-turn scripts. Made-with: Cursor * feat(eval): extend scenario 09 transcript heuristics Add no_client_bundled_outpost_key and readme_or_env_docs checks to scoreScenario09 (align with full-stack success criteria). Made-with: Cursor * feat(eval): persist run lifecycle sidecars Write eval-run-started.json at scenario start; eval-failure.json on uncaught errors; eval-aborted.json on SIGTERM/SIGINT. Register signal handlers so interrupted runs leave a trace (SIGKILL still silent). Made-with: Cursor * docs(eval): authoring AGENTS, README, shared Cursor rule Add docs/agent-evaluation/AGENTS.md (anti-leakage checklist), root AGENTS.md pointer, and a Cursor rule scoped to docs/agent-evaluation/. Document run sidecars, re-scoring, integration verification wording, and scenario 09 heuristic summary. Fix placeholder fixtures markdown. Made-with: Cursor * feat(agent-evaluation): read/bash sandbox and sibling harness sidecars Restrict PreToolUse Read/Glob/Grep to the run directory (and docs/ when EVAL_LOCAL_DOCS). Block Bash that touches the monorepo root outside those areas; deny Agent unless EVAL_ALLOW_AGENT_TOOL. Split read vs write guard env vars. Write eval-started, eval-failure, and eval-aborted next to the run folder under results/runs/ so the agent cannot read harness metadata. SIGTERM/ SIGINT abort payload includes runDirectory. Made-with: Cursor * docs(agent-evaluation): document sidecars, sandbox, and env vars Describe sibling *.eval-*.json harness files and expanded PreToolUse permissions (read guard, bash, Agent tool). Made-with: Cursor * docs(agent-evaluation): update scenario 01 tracker row Record 2026-04-10 run, quickstart.sh artifact, execution smoke test, and sibling harness sidecar layout. Made-with: Cursor * docs(agent-evaluation): update scenario 02 tracker row Record 2026-04-10 run: heuristic 9/9 pass, LLM fail (script vs Next.js mismatch), execution pass via outpost-quickstart.ts. Made-with: Cursor * docs: scope-router Outpost agent prompt and refresh basics tracker rows Restructure hookdeck-outpost-agent-prompt.mdx with Quick path / new minimal app / existing app guidance, default-to-smallest behavior, language vs architecture, doc list split, mapping hints, and explicit anti-over-build rules. Update SCENARIO-RUN-TRACKER for scenarios 01–03 with recent eval runs (heuristic, LLM, execution notes, sibling harness sidecars). Made-with: Cursor * fix(api): add DestinationSchemaField.key to OpenAPI spec The API and registry metadata always returned key on config_fields and credential_fields; the published schema omitted it and examples did not validate against the corrected shape. Align DestinationSchemaField and embedded destination-types examples with the wire format. Made-with: Cursor * docs: refine Building your own UI guide and onboarding agent prompt Rebalance audience and IA (SDK-first server usage, wire JSON in later sections). Shorten prompt invariants with links; align with integration checklist. Made-with: Cursor * docs(eval): tighten scenarios 08–10 and transcript heuristics Stricter success criteria with guide/prompt references; align placeholders. Add heuristic checks for activity and test-publish signals where applicable. Made-with: Cursor * docs(eval): document wall time for heavy baseline scenarios Explain 08–10 clone/install cost, sparse console output, and operator knobs (Ctrl+C, EVAL_SKIP_HARNESS_PRE_STEPS, EVAL_MAX_TURNS, --no-score-llm). Mirror a short note in run-agent-eval --help output. Made-with: Cursor * docs(eval): update scenario run tracker for scenario 08 Record primary run 2026-04-10T14-29-04-214Z-scenario-08, heuristic 10/10, execution pass, and execution notes (seed/dev, schema key vs SDK). Made-with: Cursor * docs: drop destinations overview hub; clarify OSS hosting in concepts - Remove redundant destinations/overview.mdoc; link to overview#supported-destinations from quickstarts, building-your-own-ui, nav, and redirects (/destinations) - Document MAX_DESTINATIONS_PER_TENANT and DESTINATIONS_METADATA_PATH under self-hosting configuration - Concepts: Hookdeck hosts same open-source Outpost; GitHub feature requests for all - Ignore docs/dist and docs/TEMP-*.md; remove temp onboarding status file Made-with: Cursor * docs: use hookdeck.com/docs/outpost for production doc links Update README, OpenAPI contact URL, entrypoint migration hint, and example READMEs so public links match Outpost docs on Hookdeck. Made-with: Cursor * docs(eval): Hookdeck prod as default {{DOCS_URL}}; fix harness doc paths - Default EVAL_DOCS_URL to https://hookdeck.com/docs/outpost - Replace invalid destinations directory path with overview + webhook mdoc - Document placeholder examples in agent prompt and fixtures Made-with: Cursor * docs(agent-evaluation): refresh tracker, scenarios, and harness docs - Point scenario and script links at docs/content paths (.mdoc) - Update SCENARIO-RUN-TRACKER for latest heuristic-pass runs - Revise README and AGENTS for current layout - Remove SKILL-UPSTREAM-NOTES (obsolete) Made-with: Cursor * docs(eval): record scenario 10 pass in run tracker Log 2026-04-10T22-14-20-704Z-scenario-10 with heuristic/LLM/execution results and execution notes (Go baseline, signup smoke, Hookdeck probe). Made-with: Cursor * ci(docs): agent eval workflow with live Outpost execution Add docs-agent-eval-ci.yml: scenarios 01+02 with EVAL_LOCAL_DOCS, heuristic + LLM judge, then execute-ci-artifacts.sh (curl + TypeScript) using OUTPOST_API_KEY. Trigger on docs content/apis, agent-evaluation harness (ignoring tracker/results README noise), TypeScript SDK, and workflow edits. Ignore .env.ci for local secret template; document secrets and execution in README. Made-with: Cursor * ci(docs): allow workflow_dispatch for manual agent eval runs Made-with: Cursor * ci(docs): fix workflow YAML (paths vs paths-ignore); document dispatch GitHub rejects paths + paths-ignore on the same event; drop paths-ignore. README: manual workflow_dispatch; note broader path matches. Made-with: Cursor * fix(agent-eval): eval:ci argv — drop stray -- before --scenarios Node parseArgs treats a bare -- as starting positionals; --scenarios then failed with ERR_PARSE_ARGS_UNEXPECTED_POSITIONAL in CI. Made-with: Cursor * fix(agent-eval): execution defaults, smoke test, CI env for live Outpost - execute-ci-artifacts: EVAL_TEST_DESTINATION_URL fallback for webhook URL; default OUTPOST_API_BASE_URL with := (empty .env no longer strips version path); clearer errors on shell/ts failure - Add smoke-test-execute-ci-artifacts.sh + npm run smoke:execute-ci (topics *, loads .env then .env.ci) - CI execution step: OUTPOST_API_BASE_URL + OUTPOST_CI_PUBLISH_TOPIC - README troubleshooting (404) and .env.example OUTPOST_CI_PUBLISH_TOPIC Made-with: Cursor * chore: Move agent promot * chore: Add missing docs descriptions * docs: add v0.17 upgrade guide and update migration docs (#831) Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: Change nav docs order * fix: Fix schema for managed config * fix: Fix gramatical errors * fix: Fix eval script --------- Co-authored-by: Alex Luong <alex.luong@hookdeck.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Alex Luong <alex@alexluong.com> Co-authored-by: Phil Leggetter <phil@leggetter.co.uk>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Ports the v0.16 upgrade guide from #811 (merged to
main) into the new Markdoc-based docs structure onfeat/refactor-docs(#814).docs/content/self-hosting/upgrade-v0.16.mdocfollowing theupgrade-v0.14.mdocpattern (title + description frontmatter, plain Markdoc).docs/content/nav.jsonunder the Changelog section, after v0.14.feat/refactor-docs. Env var names (OPERATION_EVENTS_*) are preserved as-is since they're defined in code.Notes
feat/refactor-docs(notmain), so it merges into feat: Add new docs content #814 before that PR lands.docs/content/features/operator-events.mdocalready uses "operator events" in prose withOPERATION_EVENTS_*env vars — no changes needed there.features.mdx,event-delivery.mdx,alerts.mdx,zudoku.config.ts) — they're all rewritten or deleted by feat: Add new docs content #814.Test plan
feat/refactor-docs+ this branch locally and confirm the v0.16 upgrade page loads/features/operator-eventsresolves