Sub-issue of #72 (phase 3 of the suggested sequencing — longer-horizon).
Problem. Model combination is a real source of score gains, but evolving it inside the same flat loop as base models doesn't work — the search space blows up and combo-search has a very different cost/value profile from base-search. Combo gains are mostly unrealized today.
Directions.
- Two-stage / hierarchical evolution: evolve base models first, then a separate evolution stage on combination strategies over a frozen or curated pool of bases.
- Give combination its own population, budget, and selection criteria — don't mix it into the per-model attempt budget.
- Larger architectural change; tracking as a longer-horizon direction rather than a quick fix.
Best taken on once #73 and the gradient meta-evolve work are stable.
Sub-issue of #72 (phase 3 of the suggested sequencing — longer-horizon).
Problem. Model combination is a real source of score gains, but evolving it inside the same flat loop as base models doesn't work — the search space blows up and combo-search has a very different cost/value profile from base-search. Combo gains are mostly unrealized today.
Directions.
Best taken on once #73 and the gradient meta-evolve work are stable.