weigh — weighted-trajectory selection

Run sample → attach signals → weight → select over one task. The model side proposes candidate trajectories (this is the Workflow tool's substrate — its judge-panel / adversarial-verify sampling); the harness owns the weight function that picks among them: scripts/score-trajectories.mjs, a pure, zero-dep, frozen-weights scorer. See references/workflow-shape.md for the seam and references/scoring.md for the weighting contract.

Core principle: the model proposes, the scorer disposes. Selection is a deterministic, version-controlled, reconstructable weighted sum — not a vibe.

disable-model-invocation: true suppresses auto-invocation only — the model never fires /weigh on its own (it spawns N agents and burns tokens). A user-typed /weigh runs the full procedure below, including the Step-2 sampling fan-out. allowed-tools includes Agent because Step 2 spawns the N sampling agents — without it the sampling step cannot run.

Result tag

Announce exactly one human result tag at the end of the run:

RESULT: SELECTED | NO-SELECTION | DRY-RUN

Tag	Meaning
SELECTED	The scorer ran and returned a selected id (or top-K for synthesis).
NO-SELECTION	Every trajectory broke the hard floor — the scorer returned { selected: null, reason: "NO-SELECTION", floorViolations }. Report the floor that killed them; do not promote a failure.
DRY-RUN	--dry-run was passed: the sampling plan was printed and the run stopped after Step 1.

Steps

Step 1 — Resolve config (and --dry-run gate)

Parse $ARGUMENTS:

Setting	Default	Notes
task / --cohort <path>	(required)	A task string to sample for, OR a pre-assembled cohort JSON to score directly (skip Steps 2–3).
--n N	4	Number of trajectories to sample. Hard cap 8 (mirrors /delegate wave discipline + the cost risk). Clamp N into [1, 8].
--method	best-of-n	One of best-of-n (argmax), vote (largest self-consistency cluster), softmax, synthesis (top-K).
--weights <json>	frozen DEFAULT_WEIGHTS	Passed through to the scorer verbatim (single shell token); the scorer's validateWeights() rejects bad vectors. --weights '{"consistency":30,"evalPass":20,"auditPass":15,"cost":10,"judge":0}' → a fully deterministic, judge-free weight.
--soft	off	Convert the hard eligibility floor into a down-weight (least-bad pick allowed).
--dry-run	off	Print the sampling plan (N, angles, method, weights) and stop here — spawn no agents. Announce RESULT: DRY-RUN, run Step 6's log, stop.

If a --cohort <path> is supplied, the cohort is already assembled — skip Steps 2–3 and go straight to Step 4.

Step 2 — Sample N trajectories (the Workflow-tool substrate)

Launch N Agent tool calls in ONE message (parallel), each attacking the task from a different angle (e.g. minimal-diff, defensive/edge-case-first, refactor-toward-simplicity, performance-first) so the cohort spans the solution space rather than re-deriving one path N times. Each worker returns structured output conforming to TRAJECTORY_SCHEMA — the named JSON-Schema object exported by scripts/score-trajectories.mjs (the single source of truth for the trajectory record shape). At minimum each trajectory emits:

• id (required, stable + unique), output (the candidate answer/artifact), costTokens (tokens consumed).

The signal fields (evalRc, auditVerdict, clusterId, clusterSize, judgeScore, judgeReason) are attached in Step 3 — workers leave them null.

Cite the schema source to the workers so their output is machine-consumable:

# the published contract the sampling agents target:
node -e 'import("'"${CLAUDE_SKILL_DIR}"'/scripts/score-trajectories.mjs").then(m => process.stdout.write(JSON.stringify(m.TRAJECTORY_SCHEMA, null, 2)))'

--dry-run already returned in Step 1; if it reaches here it was not set.

Step 3 — Attach signals (compose /eval + /audit implementation by reference)

For each sampled trajectory, attach the deterministic signals the scorer weights — compose the existing gates, do not re-derive them:

• evalRc — run the trajectory's change through /eval (the regression floor); record the runner rc (0 PASS · 1 REGRESSION · 2 SKIPPED · null N/A). evalRc === 1 is a hard-floor breaker.
• auditVerdict — run /audit implementation <slug> (per-unit promotability) for trajectories that touch code; record "PASS" · "FAIL" · null. "FAIL" is a hard-floor breaker.
• clusterId / clusterSize — one clustering pass: group the N outputs by semantic equivalence (a model step) and stamp each with its cluster id and the cluster's member count. consistency (the 30-weight signal) is clusterSize / N.
• judgeScore / judgeReason — optional verifier LM scoring each output 0..1. Leave null to disable the judge (the scorer treats null as a neutral 0.5); set judge: 0 in --weights to remove its influence entirely.

Write the assembled cohort (array of TRAJECTORY_SCHEMA records, or a { "trajectories": [...] } wrapper) to a gitignored working file, e.g. $TMPDIR/oh-weigh/<UTC-date>/weigh-<slug>-<HHMMSS>.cohort.json.

Step 4 — Weight + select (the harness-owned step)

Run the owned scorer over the assembled cohort. --now is required (the scorer has no Date.now() fallback — purity); supply it from the shell so the scorer itself stays pure:

node "${CLAUDE_SKILL_DIR}/scripts/score-trajectories.mjs" \
  --cohort "${TMPDIR:-/tmp}/oh-weigh/$(date -u +%F)/weigh-<slug>-<HHMMSS>.cohort.json" \
  --method "<best-of-n|vote|softmax|synthesis>" \
  --now "$(date -u +%s)" \
  [--weights '<json>'] [--soft]

The scorer prints a JSON report: the resolved config, the per-trajectory scored[] rows (each with a fully reconstructable weightBreakdown), any floorViolations, and selected (an id, or an array of ids for synthesis). If no trajectory is eligible it returns { selected: null, reason: "NO-SELECTION", floorViolations: [...] } — surface the floor that killed the cohort and announce RESULT: NO-SELECTION. Never hand-pick a least-bad trajectory the scorer rejected (use --soft if a least-bad pick is genuinely wanted).

Step 5 — Aggregate (for synthesis)

For --method synthesis the scorer returns the top-K eligible ids (selected is an array, topK carries their weights) rather than one winner. Spawn one synthesis Agent that reads the top-K outputs and grafts them into a single best answer (take the strongest element of each). For the other three methods selected is already the single chosen id — no aggregation step.

Step 6 — Persist

Write the run artifacts to ephemeral scratch under ${TMPDIR:-/tmp}/oh-weigh/<UTC-date>/, outside the repo — never inside .agro/:

• weigh-<slug>-<HHMMSS>.json — the full scorer report (config + scored rows + selection), the audit trail.
• weigh-<slug>-<HHMMSS>.md — a short human summary: the task, N, method, weights, the selected id(s) + why (cite the winning weightBreakdown), and any floorViolations.

Announce the RESULT: tag once Step 6 completes.

Anti-patterns

• Hand-picking the scorer's reject. If select() returns NO-SELECTION, the hard floor killed every trajectory — report it, do not promote a failure. Use --soft deliberately if a least-bad pick is wanted.
• Sampling the same path N times. Step 2's angles must differ, or the cohort has no diversity to select over and consistency is meaningless.
• Calling Date.now() for --now. The scorer is pure by design; the skill supplies --now "$(date -u +%s)" from the shell. Never patch a fallback into the scorer.
• Exceeding the cap. N > 8 is rejected — cost grows linearly with N (plus /eval + /audit implementation <slug> per trajectory). Preview with --dry-run first.
• Writing the artifacts into the repo. They belong in $TMPDIR; never stage a cohort, report, or summary.
• Editing the scorer to change weights inline. DEFAULT_WEIGHTS is Object.freezed and probe-pinned; pass --weights for a one-off, and route any durable change through review (as /benchmark).

References

• references/scoring.md — the weighting formula, sub-signal table, validateWeights rules, hard-floor + --soft semantics, and the judge:0-vs-judge:25 tuning signal.
• references/workflow-shape.md — the seam: which steps are the Workflow tool's sampling/judging substrate and which step (the scorer) is the part we own, plus the illustrative runtime workflow-script shape.
• scripts/score-trajectories.mjs — the harness-owned scorer (exports validateWeights, weight, select, clamp, DEFAULT_WEIGHTS, TRAJECTORY_SCHEMA).