Audit Scoring

Purpose

This directory provides the quantitative scoring framework for measuring audit quality. Use these metrics to objectively evaluate performance, track improvement over time, and identify areas needing attention.

Available Files

File	Description
AUDIT_SCORING.md	Complete scoring system — detection, precision, severity accuracy, coverage, efficiency metrics, composite score formula, reward schema, tracking templates, and industry benchmarks

Core Metrics at a Glance

Metric	Weight	What It Measures
Detection Score	35%	Vulnerabilities correctly identified vs. total real vulnerabilities
Precision Score	25%	Valid findings vs. total findings submitted (false positive rate)
Severity Accuracy	15%	Correct severity classification vs. total findings
Coverage Score	15%	Functions/entry points audited vs. total codebase
Efficiency Score	10%	Weighted findings produced per hour spent

Composite Score = (0.35 × Detection) + (0.25 × Precision) + (0.15 × Severity) + (0.15 × Coverage) + (0.10 × Efficiency)

Severity Weights for Efficiency Scoring

These weights connect the scoring system to the severity classification:

Severity	Points	Reference
Critical	10	Escalation required (not in standard severity files)
High	5	high-severity.md
Medium	2	medium-severity.md
Low	1	low-severity.md
Informational	0.5	Best-practice suggestions
Gas	0	gas-optimizations.md

How to Use

1. After an audit → Fill out the Score Card Template in AUDIT_SCORING.md
2. Classify findings → Use severity/ files + severity-scoring decision tree
3. Track monthly → Use the Monthly Score Tracking template
4. Identify gaps → Category-specific scores highlight weak areas
5. Improve → Low category scores → update checklists and patterns

Related Skills

• Severity Classification — HIGH / MEDIUM / LOW / GAS finding databases
• Severity Scoring Decision Tree — How to assign severity levels
• Feedback Loop — Scores feed back into skill improvement
• Audit Report Templates — Report structure with severity sections
• Prompt Evolution — Higher-scoring prompts get promoted

Prerequisites

Scoring requires completed audit findings with severity classifications. The Severity Classification skill MUST be applied before scoring.

Validation

To verify scoring accuracy, compare computed composite scores against known benchmarks:

# Example composite score calculation
detection = 0.85   # 85% of real vulns found
precision = 0.80   # 80% valid findings
severity_acc = 0.90 # 90% correct severity
coverage = 0.75    # 75% codebase covered
efficiency = 0.70  # Weighted findings per hour

composite = (0.35 * detection + 0.25 * precision + 0.15 * severity_acc + 0.15 * coverage + 0.10 * efficiency)
print(f"Composite Score: {composite:.2f}")  # Expected: 0.81

# Score thresholds for audit quality tiers
tiers:
  elite: 0.90+       # Top-tier competitive auditor
  proficient: 0.75+  # Solid professional auditor
  developing: 0.60+  # Learning auditor
  needs_work: <0.60  # Consider additional training

# Validate scoring data integrity
python scripts/quality-check.py skills/scoring/SKILL.md

Behavior Guidelines

• Detection and Precision scores are required for every engagement
• Coverage tracking is optional for quick scans but MUST be included in full audits
• Efficiency scoring should be used for self-improvement, never to rush audits

References

• Scoring References - Industry benchmarks and calibration data