Benchmark Optimization Loop
SkillDev toolsThis skill lets your AI make code faster by testing instead of guessing. It benchmarks several versions of an implementation against each other and keeps the best-performing one that is still safe to use. Each round of measurement feeds the next, so improvements come from real numbers.
Available today. Use it from your connected AI after setup.
No other account needed.
After adding it, ask your AI to make a piece of code faster or to compare a few ways of doing the same task. It will run measured tests and report which version performs best.
Then ask your AI: use the Benchmark Optimization Loop skill
What your AI can do with it
- Benchmark how quickly different code versions run
- Compare latency, throughput, and cost across variants
- Try many implementations of the same task
- Run repeated optimization rounds to keep improving performance
- Adopt the best-performing version that is still safe
What this skill tells your AI
The instructions your AI receives, as published by affaan-m/ecc in skills/benchmark-optimization-loop/SKILL.md and read by ahel’s review.
Use this skill to convert "make it 20x faster" or "try 50 recursive optimizations" into a bounded measured loop that can actually improve a system.
Required Baseline
Do not optimize until these exist:
- the operation being optimized;
- the correctness gate that must stay green;
- the metric: wall time, p95 latency, rows/sec, cost/run, memory, error rate;
- the current baseline;
- the search budget: max variants, max time, max spend, max data impact.
If the user asks for an unrealistic target, keep the ambition but make the loop bounded and measurable.
Loop
- Measure the baseline.
- Identify bottlenecks from evidence.
- Generate variants that test one hypothesis each.
- Run variants with the same input shape.
- Reject variants that fail correctness, safety, or reproducibility.
- Promote the fastest safe variant.
- Codify the winning path in a script, command, test, config, or doc.
- Rerun the baseline and winner to confirm the delta.
Variant Table
Track variants like this:
Variant | Hypothesis | Command | Time | Correct? | Notes
baseline | current path | npm run job | 120s | yes | stable
batch-500 | fewer round trips | npm run job -- --batch 500 | 42s | yes | winner
parallel-8 | more workers | npm run job -- --workers 8 | 31s | no | rate limited
Recursive Search
For recursive or hyperparameter work:
- persist every run to a ledger;
- compare against the prior accepted winner, not only the previous run;
- keep a holdout or replay check;
- stop when improvement is within noise, correctness fails, cost exceeds the budget, or the search starts changing more variables than it can explain.
Use phrases like "best measured safe variant" instead of "global optimum" unless the search space was actually exhaustive.
Promotion Gate
A variant cannot become the new default until:
- correctness tests pass;
- the performance delta is repeated or explained;
- rollback is obvious;
- the change is encoded in source control or a durable runbook;
- the final summary includes exact commands and measurements.
Signals
- GitHub stars
- 256k
- Forks
- 38k
- Last commit
- Sep 2026
Others that do the same job
Advanced
- Catalog kind
- skill
- Gateway key
benchmark-optimization-loop- Source
- github.com/affaan-m/ecc