second-opinion

Overview

Independent, multi-model review of any material the user provides, turned into tiered decisions. Uses Amazon Quick's parallel task system to spawn diverse reviewer agents, orchestrate anonymized cross-review, and synthesize peer-validated findings into a chair verdict.

Workflow

<Definition - Council> A group of 1-5 independent reviewer agents, each spawned as an isolated task with no shared context. Each reviewer operates on a different model tier or with a different briefing perspective to maximize diversity of critique. </Definition - Council>

<Definition - Cross-Review> Stage where reviewer agents critique each other's reviews anonymously. Each judge ranks all reviews and adjudicates every finding (agree/dispute/neutral). No judge knows which review is theirs, preventing self-favoritism. </Definition - Cross-Review>

<Definition - Finding Tiers> Classification of findings based on cross-review consensus:

• Confirmed: agrees > disputes, at least 2 judges engaged
• Contested: split opinion among judges
• Singleton: only 1 judge engaged, unvalidated </Definition - Finding Tiers>

<Definition - Street Cred> A reviewer's rolling average rank position across judges' FINAL RANKING blocks. Lower is better. Used to track reviewer reliability over time. </Definition - Street Cred>

<Definition - Model Tiers> Quick's built-in model selection:

• smart: deep reasoning, 120s timeout
• balanced: practical all-rounder, 90s timeout
• fast: quick pattern-matching, 60s timeout </Definition - Model Tiers>

<Workflow - Council Review description="End-to-end multi-model council review of user-provided material." tools=[start_task, create_task_group, get_task_group_result, file_read, file_write, run_python, get_current_time] triggers=["second opinion", "council review", "multi-model review", "cross-check this", "red-team this", "stress-test this", "what would other models think", "peer review this", "independent review"] preferred_model=smart preferred_thinking=high

1. [Ask user] Gather inputs: {{source_material}}, {{analysis_request}}, {{criteria}}. Ask only for what's missing. Validate: All three inputs present. If fails: Ask for missing inputs. Suggest defaults from context if possible.

2. [Agent] Read the source material (fetch links, extract text from docs). If content exceeds 15K chars, save to a workspace file and note the path for reviewer briefings. Validate: Source material text extracted successfully. If fails: Ask user for alternative format or direct paste.

3. [Think] Select council composition. Default: 3 agents across model tiers (smart, balanced, fast), each with a different review lens (e.g., strategic, operational, technical precision). Consider user's {{council_size}} input if provided.

4. [Agent] Compute time estimate. Check for scripts/run-history.jsonl. If 3+ entries with stage-durations exist, compute p50/p90 from recent 20 entries. Otherwise fall back to static estimate: "~10 task spawns across 3 waves, ~5-8 minutes." Validate: Estimate computed or fallback used. If fails: Use static estimate.

5. [Ask user] Present run shape: reviewer composition, model tiers, estimated time, and task spawn count. Wait for explicit confirmation before launching. Validate: User confirms. Per Rule 1, never proceed without this. If fails: Adjust composition per user feedback and re-present.

6. [Agent] Launch all reviewers in parallel using create_task_group. Each reviewer gets: full source material (inline or file path), analysis request, criteria, perspective-specific briefing, and strict output format (findings list with id, claim, severity, location, rationale). Use different model tiers per Rule 11. Validate: Each task completes with structured findings. If fails: Per Rule 9, degrade gracefully. If fewer than 2 succeed, abort.

7. [Agent] Save each review to artifacts/{stem}-council/review-{label}.md as it arrives. Measure pairwise diversity using Jaccard similarity on normalized claim sets (see references/workflow-details.md for algorithm). Emit warning if any pair exceeds 0.7 threshold. Validate: Reviews saved. Diversity matrix computed. If fails: Proceed without diversity measurement. Note in report.

8. [Decide] If council_size is 1, skip to step 12 (orchestrator synthesizes directly). If 2+, proceed to cross-review.

9. [Agent] Build anonymized bundle: assign stable labels (Review X, Y, Z), shuffle order (Rule 10), strip identifying markers. Keep private label-to-model map for later de-anonymization. Validate: Bundle contains all reviews with anonymized labels. If fails: Retry bundle construction.

10. [Agent] Launch cross-review judges via create_task_group (same N agents). Each judge receives the same bundle and must: RANK all reviews best-to-worst, and ADJUDICATE every finding (agree/dispute/neutral + one-line reason). Validate: Each judge produces FINAL RANKING + adjudications. Need at least 2 valid judge responses. If fails: Proceed with available judges. If fewer than 2, use available rankings only.

11. [Agent] Score cross-review: calculate avg rank per reviewer (street cred), count agrees/disputes per finding, classify findings into tiers (Confirmed/Contested/Singleton). Select chair: fresh smart-tier agent that did NOT serve as reviewer. Launch chair with full de-anonymized context. Validate: Chair produces verdict with overall assessment, grouped recommendations, contested items, and APPROVE/REVISE/REJECT rating. If fails: Per Rule 2, orchestrator synthesizes directly and discloses this.

12. [Ask user] Present findings in tiers: Confirmed (bulk accept), Contested (individual with dissent context), Singleton (noted but unvalidated). Wait for user decisions on each tier. Validate: User responds to each tier. If fails: Save verdict as-is if user doesn't engage.

13. [Agent] Write outputs to artifacts/{stem}-council/: verdict.md, review-{A,B,C}.md, cross-review-bundle.md, report.md. For editable sources, offer to apply accepted changes to a -reviewed copy. Validate: Files written successfully. If fails: Retry write. If workspace issue, print report inline.

14. [Agent] Capture lessons: generate run-id (UUID v4), record timestamp, compute per-stage durations, append record to scripts/run-history.jsonl. Check MODEL-NOTES.md growth cap (default 300 lines). See references/workflow-details.md for full Stage 6 procedure. Validate: JSONL record written. If fails: Skip gracefully. Never block on lessons capture.

15. [Ask user] Present proposed MODEL-NOTES.md updates (if any). Write nothing without approval (Rule 13). Validate: User approves or rejects. If fails: Skip MODEL-NOTES update. JSONL record already written.

</Workflow - Council Review>

<Template - Reviewer Briefing>

You are an independent reviewer evaluating material for a structured council review.

PERSPECTIVE: {{perspective_lens}}
MATERIAL: {{source_material_or_path}}
ANALYSIS REQUEST: {{analysis_request}}
CRITERIA: {{criteria}}

Begin immediately with your findings. No preamble, no "I will now analyze...", no meta-commentary.

OUTPUT FORMAT (produce EXACTLY this structure):
## Findings

### Finding 1
- **ID**: F1
- **Claim**: [one-sentence statement of the issue]
- **Severity**: [critical / major / minor / nit]
- **Location**: [section or line reference in the material]
- **Rationale**: [2-3 sentences explaining why this matters]

[repeat for each finding]

## Overall Take
[3-4 sentences: overall quality assessment, key strengths, primary concerns]

</Template - Reviewer Briefing>

<Template - Cross-Review Judge Briefing>

You are a judge in an anonymous peer review. Below are {{N}} independent reviews of the same material. You do NOT know which review is yours (if any).

{{anonymized_bundle}}

TASK:
1. RANK all reviews from best to worst for the target audience. Justify in one line per rank.
2. ADJUDICATE every finding from every review: agree / dispute / neutral + one-line reason.

OUTPUT FORMAT:
## FINAL RANKING
1. Review [X] - [one-line justification]
2. Review [Y] - [one-line justification]
3. Review [Z] - [one-line justification]

## ADJUDICATIONS
### Review [X]
- F1: [agree/dispute/neutral] - [reason]
- F2: [agree/dispute/neutral] - [reason]
[repeat for all findings in all reviews]

</Template - Cross-Review Judge Briefing>