Protocol guide

AGENTS.md

@@ -74,3 +74,17 @@ All three tools are pre-configured in `pyproject.toml` and can be run without ex
 - **ruff**: `uv run ruff check` (excludes `docs/`, configured in `[tool.ruff]`)
 - **mypy**: `uv run mypy` (targets `src/`, configured in `[tool.mypy]`)
 - **pytest**: `uv run pytest` (targets `test/`, configured in `[tool.pytest.ini_options]`)
+
+## Agent skills
+
+### Issue tracker
+
+Issues live as GitHub issues on the canonical upstream `toolsforexperiments/labcore` (not `origin`). See `docs/agents/issue-tracker.md`.
+
+### Triage labels
+
+Default canonical label names (`needs-triage`, `needs-info`, `ready-for-agent`, `ready-for-human`, `wontfix`). See `docs/agents/triage-labels.md`.
+
+### Domain docs
+
+Single-context layout: one `CONTEXT.md` + `docs/adr/` at the repo root. See `docs/agents/domain.md`.

CONTEXT.md

@@ -0,0 +1,68 @@
+# labcore — Domain Context
+
+The vocabulary used across the codebase and docs. Update entries as terms are
+clarified; remove or rewrite entries that go stale.
+
+## Protocols subsystem
+
+- **Protocol** — the top-level entity a lab user runs end-to-end (e.g. a qubit
+  tune-up). A protocol holds a tree of branches and operations that execute in
+  sequence, with optional conditional branching. Implemented as a subclass of
+  `ProtocolBase` whose `__init__` builds `self.root_branch`.
+
+- **Operation** — a single measurement step inside a protocol (e.g. a resonator
+  spectroscopy, a power Rabi). Each operation follows a fixed lifecycle:
+  `measure → load_data → analyze → evaluate → correct`. Implemented as a
+  subclass of `ProtocolOperation`.
+
+- **Parameter** — a named handle that an operation reads from or writes to.
+  Sits between operations and two concerns the operation does not want to know
+  about:
+  1. **Persistence across processes.** Lab work runs in many processes — a
+     notebook for ad-hoc operations, a script for a full protocol — and
+     parameter values must survive process boundaries. Each parameter holds a
+     `params` proxy to whatever persistence layer is in use (typically the
+     `instrumentserver` parameter manager, but a config file or any other
+     store works equally well).
+  2. **Hardware translation.** Different platforms speak different languages.
+     QICK can program a qubit frequency in GHz directly; OPX has to split the
+     same value into IF + LO and mix. Each platform-specific getter/setter
+     (`_qick_getter`, `_opx_getter`, `_dummy_getter`) carries whatever
+     conversion logic that platform needs.
+
+  The analysis layer only sees the resolved value via `param()`; it does not
+  care how it was produced. Operations register parameters via
+  `_register_inputs`, `_register_outputs`, and `_register_correction_params`.
+
+- **Correction parameter** — a parameter that controls a *correction strategy*
+  rather than hardware state (e.g. a noise tolerance, a step count). Subclass
+  of `CorrectionParameter`. Excluded from hardware verification; otherwise
+  identical to `ProtocolParameterBase`.
+
+- **Check** — a pure, side-effect-free assessment performed during `evaluate()`,
+  producing a `CheckResult(name, passed, description)`. An operation can
+  register multiple checks; the default `evaluate()` runs them all and returns
+  RETRY if any fail.
+
+- **Correction** — a strategy applied *between retries* when a specific check
+  fails. One instance per operation, created in `__init__` and reused across
+  retries so stateful strategies (e.g. stepping through a list of windows) work
+  correctly. A correction declares which check it is `triggered_by`.
+
+- **Branch** — a named sequence of operations and conditions inside a protocol.
+  Implemented as `BranchBase`. The simplest protocol is one root branch
+  containing a flat list of operations (see `QubitTuneup`).
+
+- **Platform** — the hardware backend a protocol runs against (`DUMMY`, `QICK`,
+  `OPX`). Selected globally via the `PLATFORMTYPE` module variable in
+  `labcore.protocols.base`; parameters and operations dispatch to
+  platform-specific code (`_dummy_getter`, `_qick_getter`, …) based on it.
+
+- **Report** — a self-contained HTML document assembled by
+  `ProtocolBase._assemble_report()` after a protocol runs. Each operation
+  contributes by appending strings (markdown) and figure paths to
+  `self.report_output`; figures are embedded as base64 data URIs so the
+  resulting file stands on its own. The default `correct()` adds a check
+  table; `_register_success_update` adds parameter-improvement lines.
+  SuperOperations aggregate their sub-operations' contributions. Saved under
+  `report_path / "{ProtocolName}_report"`.

docs/adr/0001-parameters-abstract-persistence-and-hardware.md

@@ -0,0 +1,46 @@
+# Parameters abstract persistence and hardware translation
+
+Parameters in `labcore.protocols` are dataclass subclasses of
+`ProtocolParameterBase` that hold a `params` proxy to a persistence backend
+and implement platform-specific getter/setter methods (`_qick_getter`,
+`_opx_getter`, `_dummy_getter`). Operations interact with parameters through
+a uniform `param() / param(value)` call, never with the backing store or the
+target platform directly. We chose this shape because parameters sit between
+operations and two concerns the operation must not be coupled to: a
+persistence layer that survives Python-process boundaries (notebook running
+one operation, script running a full protocol — same parameter values), and
+hardware platforms that handle parameters in non-equivalent ways (QICK takes
+a qubit frequency in GHz directly; OPX has to split it into IF + LO and mix).
+
+## Considered Options
+
+- **Flat dict-of-values.** A `dict[str, float]` shared via a module global
+  or passed into operations. Rejected: provides no place for hardware
+  translation logic, and forces persistence to be solved in user code.
+- **Direct coupling to `instrumentserver`.** Make every parameter call
+  `instrumentserver.helpers.nestedAttributeFromString` directly, no
+  abstraction. Rejected: hard-codes one persistence backend; users wanting
+  config files or other stores would have to fork. Also still leaves the
+  hardware-translation problem unsolved.
+- **Per-operation hardcoding.** Each operation reads/writes hardware in its
+  own `_measure_*` body. Rejected: parameters are typically reused across
+  many operations (a `QubitFrequency` shows up in spectroscopy, Rabi, T1, …)
+  and duplicating the read/write/translate logic per operation is a
+  maintenance hazard.
+
+## Consequences
+
+- **More boilerplate per parameter.** A parameter that needs to support
+  three platforms is ~30 lines of dataclass + getter/setter pairs even when
+  the logic is trivial. Mitigated by the "implement only the platforms you
+  use" pattern — most parameters today implement DUMMY + QICK only and let
+  the others raise `NotImplementedError`.
+- **Persistence backend is swappable.** A toolbox can use the
+  `instrumentserver` parameter manager (the common choice today), a config
+  file, or any other store, without changes to operations or to labcore.
+- **New platforms add zero churn to existing operations.** Adding OPX
+  support for a parameter is a localized change — implement
+  `_opx_getter`/`_opx_setter`. Operations and the analysis layer don't move.
+- **Analysis layer stays clean.** Analysis only ever calls `param()` and
+  receives the resolved value; it does not see the platform-specific
+  conversion logic.

docs/agents/domain.md

@@ -0,0 +1,41 @@
+# Domain Docs
+
+How the engineering skills should consume this repo's domain documentation when exploring the codebase.
+
+This repo uses a **single-context** layout: one `CONTEXT.md` at the repo root and one `docs/adr/` directory.
+
+> Note: `labcore` is one of four packages in the [toolsforexperiments ecosystem](https://toolsforexperiments.github.io/guides/software_map.html) — alongside `instrumentserver`, `plottr`, and `CQEDToolbox`. Each package lives in its own git repo with its own single-context setup. Cross-package vocabulary (e.g. how `labcore` relates to `instrumentserver`) belongs as a short "Ecosystem position" section in this repo's eventual `CONTEXT.md`, not as a separate context.
+
+## Before exploring, read these
+
+- **`CONTEXT.md`** at the repo root.
+- **`docs/adr/`** — read ADRs that touch the area you're about to work in.
+
+If any of these files don't exist yet, **proceed silently**. Don't flag their absence; don't suggest creating them upfront. The producer skill (`/grill-with-docs`) creates them lazily when terms or decisions actually get resolved.
+
+## File structure
+
+```
+/
+├── CONTEXT.md              ← domain glossary (sweep, DataDict, DDH5Writer, …)
+├── docs/
+│   ├── adr/                ← architectural decisions
+│   │   ├── 0001-….md
+│   │   └── 0002-….md
+│   └── …                   ← existing Sphinx docs (unrelated; coexists)
+└── src/labcore/
+```
+
+The existing Sphinx site under `docs/` is unrelated to `CONTEXT.md` and `docs/adr/` — they coexist. Sphinx will ignore `docs/adr/` unless you explicitly include it in `conf.py`.
+
+## Use the glossary's vocabulary
+
+When your output names a domain concept (in an issue title, a refactor proposal, a hypothesis, a test name), use the term as defined in `CONTEXT.md`. Don't drift to synonyms the glossary explicitly avoids.
+
+If the concept you need isn't in the glossary yet, that's a signal — either you're inventing language the project doesn't use (reconsider) or there's a real gap (note it for `/grill-with-docs`).
+
+## Flag ADR conflicts
+
+If your output contradicts an existing ADR, surface it explicitly rather than silently overriding:
+
+> _Contradicts ADR-0007 (storage format) — but worth reopening because…_

docs/agents/issue-tracker.md

@@ -0,0 +1,22 @@
+# Issue tracker: GitHub
+
+Issues and PRDs for this repo live as GitHub issues on the **canonical upstream**: [`toolsforexperiments/labcore`](https://github.com/toolsforexperiments/labcore). Use the `gh` CLI for all operations.
+
+> **Important:** This clone has two remotes — `origin` (your fork) and `upstream` (`toolsforexperiments/labcore`). Issues live on `upstream`, not `origin`. **Always pass `--repo toolsforexperiments/labcore`** to `gh issue` commands so they don't default to `origin`.
+
+## Conventions
+
+- **Create an issue**: `gh issue create --repo toolsforexperiments/labcore --title "..." --body "..."`. Use a heredoc for multi-line bodies.
+- **Read an issue**: `gh issue view <number> --repo toolsforexperiments/labcore --comments`.
+- **List issues**: `gh issue list --repo toolsforexperiments/labcore --state open --json number,title,body,labels,comments --jq '[.[] | {number, title, body, labels: [.labels[].name], comments: [.comments[].body]}]'` with appropriate `--label` and `--state` filters.
+- **Comment on an issue**: `gh issue comment <number> --repo toolsforexperiments/labcore --body "..."`
+- **Apply / remove labels**: `gh issue edit <number> --repo toolsforexperiments/labcore --add-label "..."` / `--remove-label "..."`
+- **Close**: `gh issue close <number> --repo toolsforexperiments/labcore --comment "..."`
+
+## When a skill says "publish to the issue tracker"
+
+Create a GitHub issue on `toolsforexperiments/labcore`.
+
+## When a skill says "fetch the relevant ticket"
+
+Run `gh issue view <number> --repo toolsforexperiments/labcore --comments`.

docs/agents/triage-labels.md

@@ -0,0 +1,24 @@
+# Triage Labels
+
+The skills speak in terms of five canonical triage roles. This file maps those roles to the actual label strings used in this repo's issue tracker (`toolsforexperiments/labcore` on GitHub).
+
+| Label in mattpocock/skills | Label in our tracker | Meaning                                  |
+| -------------------------- | -------------------- | ---------------------------------------- |
+| `needs-triage`             | `needs-triage`       | Maintainer needs to evaluate this issue  |
+| `needs-info`               | `needs-info`         | Waiting on reporter for more information |
+| `ready-for-agent`          | `ready-for-agent`    | Fully specified, ready for an AFK agent  |
+| `ready-for-human`          | `ready-for-human`    | Requires human implementation            |
+| `wontfix`                  | `wontfix`            | Will not be actioned                     |
+
+When a skill mentions a role (e.g. "apply the AFK-ready triage label"), use the corresponding label string from this table.
+
+Of these, only `wontfix` currently exists on `toolsforexperiments/labcore`. The other four will be created on the upstream the first time the `triage` skill applies them. Create them ahead of time with:
+
+```bash
+gh label create needs-triage     --repo toolsforexperiments/labcore --description "Maintainer needs to evaluate this issue"
+gh label create needs-info       --repo toolsforexperiments/labcore --description "Waiting on reporter for more information"
+gh label create ready-for-agent  --repo toolsforexperiments/labcore --description "Fully specified, ready for an AFK agent"
+gh label create ready-for-human  --repo toolsforexperiments/labcore --description "Requires human implementation"
+```
+
+Edit the right-hand column of the table above if you ever decide to remap to existing labels (e.g. reuse `question` as `needs-info`).

docs/api/index.md

@@ -10,5 +10,6 @@ Complete API documentation for Labcore, generated from docstrings.
    labcore.data
    labcore.measurement
    labcore.analysis
+   labcore.protocols
    labcore.utils
 ```

docs/conf.py

@@ -13,8 +13,8 @@
 # https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information
 
 project = 'Labcore'
-copyright = '2025-2026, Marcos Frenkel, Wolfgang Pfaff, Cynthia Nolan, Oliver Wolff'
-author = 'Marcos Frenkel, Wolfgang Pfaff, Cynthia Nolan, Oliver Wolff'
+copyright = '2025-2026, Tools for Experiments'
+author = 'Tools for Experiments'
 
 # -- General configuration ---------------------------------------------------
 # https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration

docs/user_guide/index.md

@@ -5,5 +5,6 @@ This user guide is organized by different topics, each having their own guides.
 ```{toctree}
 measurement/index
 data/index
+protocols/index
 instruments/index
 ```

docs/user_guide/protocols/index.md

@@ -0,0 +1,114 @@
+# Protocols
+
+A **protocol** ties several experiments together to achieve a complete goal that no
+single one of them can — *calibrating a qubit*, rather than just finding
+its frequency. Each experiment is wrapped as an **operation**: a
+self-contained unit that measures, analyses, and defines for itself what
+counts as success, usually to nail down some number, or provides next steps with an attempt to solve its failures.
+The protocol runs its operations in sequence (more complex protocols can have more complex execution flows), 
+lets each one retry itself with adjusted settings if needed, and records the whole run as a self-contained HTML report. 
+The result is the calibrated system, with a report that shows how you got there.
+
+A protocol is built out of three concepts, one per sub-page:
+
+- {doc}`parameters` — the named handles operations read from and write to
+- {doc}`operations` — a single experiment, including its checks and corrections
+- {doc}`protocols` — composing operations into a runnable protocol
+
+## How protocols are organized
+
+Every protocol is a tree of branches and operations.
+
+```
+Protocol
+└── Branch                    a named sequence of items
+    ├── Operation             a single measurement step
+    │   ├── Parameters        named handles for inputs and outputs
+    │   ├── Checks            pure assessments after analysis
+    │   └── Corrections       strategies applied between retries
+    └── Condition (optional)  routes execution to one of two branches
+```
+
+The simplest shape — and the one most protocols use — is a single root
+branch with a flat list of operations. See {doc}`protocols` for
+super-operations, conditions, and the assembled report.
+
+## The lifecycle of an operation
+
+Every operation runs the same five steps in order, on every attempt:
+
+```
+  ◀── platform-specific ──▶ ◀───── platform-agnostic ──────▶
+
+  measure ──▶ load_data ──▶ analyze ──▶ evaluate ──▶ correct
+     │            │            │            │            │
+   write       pull and     compute      check        parameter
+   hardware    normalize    (fitting,    results      writes;
+   / save      shape and    statistics)  (pure        apply any
+   raw data    names                     assessment)  correction
+               across
+               platforms
+```
+
+- `measure` — performs the measurement (or generates fake data on `DUMMY`) and saves the raw data to disk.
+- `load_data` — reads the raw data back into memory and normalizes its shape and field names so the rest of the lifecycle is platform-agnostic.
+- `analyze` — runs fits and statistics over the loaded data and attaches the results to the operation.
+- `evaluate` — returns named check results and an overall status; pure assessment, no side effects.
+- `correct` — the only place parameters get written: fitted outputs on success, a correction strategy on retry.
+
+See {doc}`operations` for how each step is implemented and customized.
+
+## Run a protocol in 10 lines
+
+```python
+from labcore.protocols import select_platform, ProtocolBase, BranchBase
+from labcore.testing.protocol_dummy.gaussian_with_correction import (
+    GaussianWithCorrectionOperation,
+)
+
+select_platform("DUMMY")
+
+class HelloProtocol(ProtocolBase):
+    def __init__(self):
+        super().__init__()
+        self.root_branch = BranchBase("hello")
+        self.root_branch.extend([GaussianWithCorrectionOperation()])
+
+HelloProtocol().execute()
+```
+
+This protocol has one operation. The operation runs a noisy Gaussian fit
+and assesses its own signal-to-noise ratio. The first attempt fails, a
+**correction** fires that lowers the simulated noise level, and the operation
+retries. After two corrections the SNR check passes, the fit succeeds, and
+the protocol writes an HTML report to the current directory.
+
+A few things to notice:
+
+- {py:func}`select_platform <labcore.protocols.select_platform>` is required
+  before any protocol can be instantiated. It tells parameters and operations
+  which hardware backend to dispatch to. `"DUMMY"` is the in-memory backend
+  used for testing.
+- The protocol is just a class with a `root_branch`. The branch holds a
+  flat list of operations.
+- The correction strategy lives **inside** the operation. The protocol does
+  not know or care that this particular operation retries itself.
+
+:::{note}
+At the moment, protocols only support the `DUMMY`, `QICK`, and `OPX`
+platforms. Adding a new platform is a small change — if you need one,
+please [open an issue on GitHub](https://github.com/toolsforexperiments/labcore/issues).
+:::
+
+## Where to read next
+
+Read in order: {doc}`parameters` → {doc}`operations` → {doc}`protocols`.
+Each page builds on the previous one.
+
+```{toctree}
+:hidden:
+
+parameters
+operations
+protocols
+```