Skip to content

feat: add ChainAnalyzer for offline schema-compatibility analysis (#77)#161

Open
dgenio wants to merge 1 commit into
feat/148-cli-diff-subcommandfrom
feat/77-analyzer-foundation
Open

feat: add ChainAnalyzer for offline schema-compatibility analysis (#77)#161
dgenio wants to merge 1 commit into
feat/148-cli-diff-subcommandfrom
feat/77-analyzer-foundation

Conversation

@dgenio
Copy link
Copy Markdown
Owner

@dgenio dgenio commented May 16, 2026

Summary

Adds chainweaver/analyzer.py — the static "what could be compiled?" companion to the deterministic runtime. Given a set of Tool objects, ChainAnalyzer answers three questions: pairwise compatibility, chain enumeration, and flow suggestion. Pure-Python, no LLM, no I/O.

This is the foundation for the next PR in the cluster (#155 chainweaver suggest CLI), which extends this module with suggest_optimizations() and adds a CLI verb.

Stacked on top of #160 (diff CLI); base cascades through #160 → #159 → #158 → #157 → main as those merge.

Closes #77.

Changes

  • chainweaver/analyzer.py — new module (~260 LoC). ChainAnalyzer class + ToolChain type alias + three private helpers (_schema_field_types, _is_compatible, _auto_input_mapping). Replaces the entry in architecture.md § Planned modules.
  • chainweaver/__init__.py — exports ChainAnalyzer and ToolChain via __all__.
  • tests/test_analyzer.py — new, 23 test cases.
  • examples/chain_analyzer.py — runnable standalone demo.
  • AGENTS.md — repo-map row added.
  • docs/agent-context/architecture.md — module-boundaries table row added; analyzer.py moves from "Planned" to "delivered."

Compatibility rule

Tool A → Tool B is compatible iff every required field of B's input_schema appears in A's output_schema with an equal type annotation. Optional consumer fields (those with a Pydantic default) are tolerated when missing. Conservative — no coercion, no subtype inference.

Invariants

Mirrors the executor's three hard rules — analyzer is invoked offline, but the discipline still applies:

  • No LLM, no network, no randomness. Pure-Python static pass.
  • Cycle-free enumeration: a tool may appear at most once per chain.
  • Depth-bounded: every traversal entry point requires max_depth.

Public API

from chainweaver import ChainAnalyzer

analyzer = ChainAnalyzer(tools=[double, add_ten, format_result])
analyzer.compatibility_matrix()
analyzer.find_chains(max_depth=3, start="double", end="format_result")
analyzer.suggest_flows(max_depth=3, min_depth=3)

Testing

  • Linting passes (ruff check chainweaver/ tests/ examples/)
  • Formatting check passes (ruff format --check chainweaver/ tests/ examples/)
  • Type checking passes (python -m mypy chainweaver/ tests/)
  • All existing tests pass — 527/527 passed in 2.01s (504 pre-existing + 23 new)
  • New tests added for new functionality
$ ruff check chainweaver/ tests/ examples/
All checks passed!
$ ruff format --check chainweaver/ tests/ examples/
54 files already formatted
$ python -m mypy chainweaver/ tests/
Success: no issues found in 46 source files
$ python -m pytest tests/ -q --no-cov
527 passed in 2.01s
$ python examples/chain_analyzer.py
Compatibility matrix:
  double         → ['add_ten', 'format_result']
  ...

The performance sanity check in tests/test_analyzer.py::TestPerformance runs 50 tools at max_depth=3 → ~117 k chains in well under the issue's 1-second bar (the test allows up to 5 s for slow CI hosts).

Diff stat: 6 files changed, 773 insertions(+), 1 deletion(-) (analyzer module + tests + example + docs).

Related Issues

Closes #77. The companion chainweaver suggest CLI (#155) lands as the next PR in this stack and extends analyzer.py with suggest_optimizations().

Checklist

  • Code follows project conventions (see AGENTS.md and docs/agent-context/)
  • Public API changes are documented — ChainAnalyzer + ToolChain exported in __all__; AGENTS.md + architecture.md updated in same PR
  • No secrets or credentials included

Tradeoffs / risks

  • Exact-annotation type match. Two tools whose schemas use semantically-equivalent but syntactically different annotations (e.g. typing.List[int] vs list[int] on Python ≥ 3.9) won't compose. Pydantic v2 normalizes these in model_fields, so in practice it's rarely an issue — but worth calling out. A future enhancement could compute pydantic.TypeAdapter equivalence.
  • No semantic field-name matching. double produces value; add_ten consumes value — that works. But if a tool produces result_value and another consumes value, the analyzer won't connect them. The issue body explicitly scopes this to name-equality; semantic similarity (embeddings, aliases) is out of scope.
  • suggest_flows() returns Flow.version="0.0.0". Auto-generated flows should not be confused with real registered flows, so they ship with a deliberately invalid-looking version that signals "review me before promoting."
  • Reserved name handling. architecture.md reserved analyzer.py for Offline computation of valid tool combinations from schemas #77; this PR delivers it and updates the reservation table. The forthcoming Add chainweaver suggest <flow> static optimizer (coordinates with #77) #155 (suggest CLI) coordinates scope by extending the same module.

Scope notes

Closes #77 only. The "candidate flow proposal from traces" stretch idea on #12 is intentionally not bundled here — analyzer.py's public surface stays focused on schema-time analysis. #155 will add suggest_optimizations(flow, traces=None) as a sibling capability.

https://claude.ai/code/session_01QcSJ3NWhe5B4k1EP25Hx3n


Generated by Claude Code

)

Closes #77.

Introduces chainweaver/analyzer.py — the static "what *could* be
compiled?" companion to the deterministic runtime. Given a set of Tool
objects, ChainAnalyzer answers:

- Pairwise compatibility — for each tool, which tools can directly
  follow it? (compatibility_matrix)
- Chain enumeration — what N-step sequences are valid, optionally
  filtered by start/end tool? (find_chains)
- Flow suggestion — promote discovered chains to ready-to-register
  Flow objects with auto-wired input_mapping. (suggest_flows)

Compatibility rule: Tool A → Tool B is compatible iff every required
field of B's input_schema appears in A's output_schema with an equal
type annotation. Optional consumer fields (those with a Pydantic
default) are tolerated when missing from A's output. Conservative by
design — no coercion, no subtype inference.

Invariants (mirrors the executor's three hard rules):
- No LLM, no network, no randomness — pure-Python static pass.
- Cycle-free enumeration: a tool may appear at most once per chain.
- Depth-bounded: every traversal entry point requires max_depth.

Public API additions (exported in __init__.py __all__):
- ChainAnalyzer
- ToolChain = tuple[str, ...]

Tests: 23 cases in tests/test_analyzer.py covering compatibility
matrix (empty / single / pair / 3-tool / type mismatch / optional vs
required fields / duplicate-name rejection), find_chains (length-1,
length-2/3, start/end filters, cycle guard, validation), suggest_flows
(min_depth, auto-wiring, validation), and a performance sanity check
(50 tools, max_depth=3 → 100k+ chains, <5s budget).

Example: examples/chain_analyzer.py is runnable standalone and prints
compatibility matrix + chain list + auto-suggested Flow for a 3-tool
toy set.

Docs:
- AGENTS.md repo map gains chainweaver/analyzer.py.
- docs/agent-context/architecture.md module-boundaries table adds
  the new module; Planned modules section moves #77 to delivered.

Verification:
  $ ruff check chainweaver/ tests/ examples/      # All checks passed
  $ ruff format --check chainweaver/ tests/ ...   # 54 files already formatted
  $ python -m mypy chainweaver/ tests/            # Success: no issues
  $ python -m pytest tests/ -q --no-cov           # 527 passed in 2.01s

Stacked on top of #160 (diff CLI); chains through
#160#159#158#157 → main as those merge.

The next PR in the cluster (#155 chainweaver suggest CLI) extends this
module with suggest_optimizations() and adds the CLI surface.

https://claude.ai/code/session_01QcSJ3NWhe5B4k1EP25Hx3n
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants