Strategy Development Workflow

Strategies fail because developers skip straight to modeling. The correct process spends most time on hypothesis and data, with modeling as a small fraction.

The Problem

A researcher downloads data, fits a model, runs a backtest, and sees a 2.5 Sharpe ratio. They deploy, then lose money because there was no documented hypothesis, no feature validation, no cost model, and no holdout.

The Pattern

WRONG

# Jump straight to modeling - no hypothesis, no validation gates
import lightgbm as lgb

data = load_data()
features = data[["momentum", "volatility", "volume"]]
labels = data["next_day_return"]

model = lgb.LGBMRegressor().fit(features, labels)
predictions = model.predict(features)  # Predicting on training data!

# "Looks great, ship it"
print(f"R2: {r2_score(labels, predictions):.3f}")  # 0.95 - overfitting

CORRECT

# Seven stages with quality gates between each
hypothesis = {
    "mechanism": "Momentum persists due to slow institutional rebalancing",
    "signal": "12-month risk-adjusted return predicts 1-month forward return",
    "kill_criteria": "IC < 0.02 or non-monotonic quintiles",
    "capacity": "ETFs with >$50M daily volume",
}
# Commit term sheet to version control before proceeding

# Stage 2: Data  → fetch + validate
# Stage 3: Features → compute + factor research
# Stage 4: Model → CPCV + model validation
# Stage 5: Backtest → realistic costs
# Stage 6: Paper trade → live data, simulated fills
# Stage 7: Live → small size, slow ramp

Quality Gates

Gate	Condition	Fail Action
Hypothesis	Documented mechanism, kill criteria, capacity estimate	Do not start coding
Data	No gaps > 2 days, point-in-time correct, survivorship-free	Fix data pipeline
Features	IC > 0.02 (HAC-adjusted), stable across subperiods	Drop factor or redesign
Model	Loss rate < 50%, best config clears the selection bound	Simplify model or revisit features
Backtest	Sharpe > 0.5 net of costs, max DD < 20%	Revise sizing or cost assumptions
Paper trade	Fills within expected slippage, no execution anomalies	Fix execution logic

Guardrails

• If Sharpe > 2.0 on daily equity data, assume lookahead bias or selection bias until proven otherwise - inspect with ml4t-lookahead-bias and ml4t-deflated-sharpe
• If in-sample and out-of-sample performance match closely, suspect data leakage
• If the strategy requires > 20% annual turnover to work, verify cost assumptions with ml4t-transaction-costs
• If no documented hypothesis exists, stop and write one before any other work

Production Implementation

import asyncio

from ml4t.backtest import Strategy, run_backtest, BacktestConfig  # MyStrategy subclasses Strategy
from ml4t.live import LiveEngine, AlpacaBroker, AlpacaDataFeed

results = run_backtest(
    prices=prices, signals=signals, strategy=MyStrategy(), config=BacktestConfig()
)

async def trade_live():
    broker = AlpacaBroker(api_key, secret_key, paper=True)
    feed = AlpacaDataFeed(api_key, secret_key, symbols=["SPY"], experimental=True)
    engine = LiveEngine(MyStrategy(), broker, feed)
    await engine.connect()
    await engine.run()

asyncio.run(trade_live())

Checklist

• Hypothesis documented in term sheet before any code
• Data validated for gaps, survivorship bias, point-in-time correctness
• Features pass IC significance test with HAC standard errors
• Model validated via CPCV; loss rate < 50% (PBO: ml4t-backtest-overfitting)
• Backtest includes realistic transaction costs
• Deflated Sharpe computed across all trials (ml4t-deflated-sharpe)
• Paper trading completed for minimum 4 weeks
• Kill criteria defined and monitoring configured before going live