fix: Distiller agent stdout output — eliminate idea.md filename ambiguity

[colehurwitz]

Factory experiment 21. Fixes the Distiller agent writing specs to a file instead of stdout by removing ambiguous idea.md references from distiller.md and ceo.md prompts.

Closes #522

Changes

• distiller.md: Replace 3 idea.md references with specification, change Write to Print, add anti-instruction
• ceo.md: Update 2 Distiller task strings and 1 spec persistence reference

[colehurwitz]

✅ Factory Review: KEEP

Verdict: KEEP
Reason: Prompt-only changes correctly eliminate idea.md filename ambiguity; guard check clean, no code/test changes, CEO review CLEAN

Experiment: #21
Hypothesis: Fix Distiller agent stdout output — eliminate idea.md filename ambiguity in distiller.md and ceo.md prompts (closes #522)

Score Comparison

Metric	Value
Before	0.0000
After	0.0000
Delta	+0.0000
Threshold	0.6000

Guard Checks

Check	Result
eval_immutable	✅ PASS
scope	✅ PASS
git_clean	✅ PASS
experiment_branch	✅ PASS

Precheck Gate

Guard check returned clean. No eval scores available — experiment not yet evaluated. Prompt-only changes do not affect eval dimensions.

Code Review Notes

• All 4 distiller.md changes correctly replace idea.md references with 'specification' and add anti-instruction preventing file creation
• All 3 ceo.md changes correctly update Distiller task strings and spec persistence reference
• Anti-instruction in distiller.md Output section is good defense-in-depth against the file-creation bug
• No functional code modified — changes are exclusively agent prompt text

---

Posted by Factory CEO

[colehurwitz]

✅ Factory Review: KEEP

Verdict: KEEP
Reason: Prompt-only fix eliminates idea.md filename ambiguity — all 3 distiller.md refs renamed, anti-instruction added, 2 ceo.md refs updated. Score +0.0054, no regressions.

Experiment: #21
Hypothesis: Fix Distiller agent stdout output — eliminate idea.md filename ambiguity

Score Comparison

Metric	Value
Before	0.7395
After	0.7449
Delta	+0.0054
Threshold	0.6000

Guard Checks

Check	Result
scope	✅ PASS
eval_immutable	✅ PASS

---

Posted by Factory CEO

[colehurwitz]

CEO Review — Experiment #21

Verdict: ✅ KEEP

Summary

Prompt engineering fix across two files (distiller.md and ceo.md) to eliminate the ambiguous "idea.md" filename references that caused the Distiller agent to write specs to a file instead of printing to stdout. This broke the CEO's ideation pipeline because the agent runner captures stdout to .factory/reviews/distiller-latest.md — when the Distiller writes to a file, the full spec is lost.

Closes #522

Changes

factory/agents/prompts/distiller.md (4 changes):

1. Task section (line 23): Produce a complete idea.md → Produce a complete project specification
2. Output section (line 62): Write the idea.md content to stdout → Print your specification to stdout
3. Output section: Added anti-instruction — **IMPORTANT: Print ALL output to stdout. Do NOT create or write to any files (no idea.md, no spec files). The factory captures your stdout automatically — writing to files means the output is lost.**
4. Exit condition (line 158): Complete idea.md printed to stdout → Complete specification printed to stdout

factory/agents/prompts/ceo.md (3 changes):

1. Interactive ideation task string (line 420): Produce a complete idea.md specification → Print the complete specification to stdout
2. Research ideation task string (line 472): same pattern
3. Spec persistence reference (line 574): idea.md content → spec content (clarity — this is the CEO's own action, not a Distiller instruction)

Root Cause

The Distiller is the only factory agent expected to output exclusively to stdout. Every other agent writes to explicit file paths (Strategist → current.md, Researcher → research.md, Archivist → archive/). The model has a strong prior toward file-writing because:

• "Write the idea.md" pattern-matches to file creation
• Other agents in the same system all write to files
• The name "idea.md" reads as a filename target, not a format label

Review Pipeline

Check	Result
CEO structured review (6 categories)	CLEAN — iteration 1
Guard check (scope, eval_immutable)	PASS
Eval score	0.7395 → 0.7449 (+0.0054)
E2E verification	PASS
Precheck (score, scope, anti-pattern)	PASS (smoke_test: pre-existing Bob auth failure on main)
Final headless review	CLEAN — iteration 1

Score

	Before	After
Composite	0.7395	0.7449
Delta		+0.0054
Threshold		0.6000

No dimension regressions. All 12 dimensions passed.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.94%. Comparing base (17e224f) to head (919c8d3).
⚠️ Report is 4 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #535      +/-   ##
==========================================
+ Coverage   87.92%   87.94%   +0.02%     
==========================================
  Files          70       70              
  Lines       10120    10180      +60     
==========================================
+ Hits         8898     8953      +55     
- Misses       1222     1227       +5     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[colehurwitz]

@xukai92 could you review and merge? Pretty small but useful change