Model Health
Version v20261029-39712336 · shipped model linear · horizon 63 sessions (~3 months) · trained through 2026-07-28 · 328,810 rows / 118 tickers
Verdict: marginal edge on the locked holdout. Holdout rank-IC +0.048 with Newey-West t 2.08 — just over the pass bar of 2; the top-quintile backtest returned -6.8% a year vs -8.0% for the benchmark (it lost less, it did not make money). Real but weak ranking skill; the live-tracking table is the test that matters now.
Read t-stats with care: at a 63-session horizon the 504 holdout sessions contain only about 8.0 non-overlapping periods — the Newey-West correction helps, but a t near 2 here is fragile evidence.
Locked holdout rank-IC
+0.048
Newey-West t +2.08
Holdout Q1 − Q5 return
+1.3%
per 63-session horizon
Holdout backtest CAGR
-6.8%
benchmark -8.0% · 2024-10..2026-08
Development rank-IC
+0.104
t +4.96 · 6/40 folds negative
Dev backtest CAGR
+10.9%
benchmark +6.2% · 2014-05..2024-04
Rolling 12-month rank-IC
252-session rolling mean of the daily rank-IC. Shaded = locked holdout. Sustained dips below zero mean the model has stopped working. Drag the slider or use the buttons to zoom.
Development backtest · 2014-05..2024-04
Top quintile vs equal-weight universe, growth of 1 SAR. Configs and horizon were chosen on this period, so it flatters the model.
Locked holdout · 2024-10..2026-08
Same chart on data never used for any choice. Orange (right axis) = portfolio ÷ benchmark: above 1.00 = ahead, below = behind.
Model vs simple baselines (horizon 63 sessions)
| Approach | What it is | Dev IC | Dev t | Holdout IC | Holdout t | Holdout Q1−Q5 |
|---|---|---|---|---|---|---|
| lightgbm | Gradient-boosted ranker (LambdaRank/NDCG) | +0.085 | +4.56 | +0.005 | +0.26 | -0.4% |
| linear shipped | Ridge regression on all features | +0.104 | +4.96 | +0.048 | +2.08 | +1.3% |
| low_vol | Low volatility: prefer the calmest stocks | +0.041 | +1.67 | +0.086 | +2.39 | +2.0% |
| reversal_1m | 1-month reversal: prefer last month's losers | -0.021 | -1.22 | -0.051 | -1.91 | -1.2% |
| momentum_12_1 | 12-1 momentum: prefer last year's winners (skip last month) | +0.098 | +4.61 | +0.039 | +1.02 | +0.7% |
| blend | Equal-weight blend of low-vol, 1m reversal and 12-1 momentum | +0.060 | +2.99 | +0.045 | +1.70 | +0.6% |
The shipped approach is the simplest one within noise of the best on development data (order of preference: blend → linear → booster). Holdout columns were computed once, after every choice was frozen.
Horizon comparison
| Horizon | Shipped candidate | Dev IC | Dev t | Holdout IC | Holdout t |
|---|---|---|---|---|---|
| 63 (~3m) | linear | +0.104 | +4.96 | +0.048 | +2.08 |
| 126 (~6m) | linear | +0.122 | +3.93 | +0.040 | +1.87 |
| 252 (~12m) | linear | +0.098 | +2.30 | +0.072 | +4.38 |
Horizon picked by the highest development t-stat; holdout shown for information only.
Backtest detail
| Development | Holdout | |
| Window | 2014-05..2024-04 | 2024-10..2026-08 |
| CAGR net | +10.9% | -6.8% |
| Benchmark CAGR | +6.2% | -8.0% |
| Sharpe | 0.633 | -0.374 |
| Information ratio | 0.411 | 0.236 |
| Max drawdown | +37.3% | +26.1% |
| Months beating benchmark | 57% | 48% |
| Monthly turnover | +16.7% | +15.4% |
| Band-40 variant · CAGR net | +8.4% | -8.6% |
| Band-40 · turnover | +7.2% | +7.6% |
Band-40 = hysteresis variant: a holding is sold only when its score drops below the 40th percentile. Diagnostic only — it cuts turnover/costs but never feeds the ship or verdict rules.
Market exposure layer — how much to hold in stocks at all
Decision for 2026-11 (data through end of 2026-10): shipped rule always → invest 100%. Model P(3-month market return > cash) = 50%.
| Candidate | Index timing dev Sharpe | dev max DD | avg exposure | switches/yr | Ranking+exposure dev CAGR | dev max DD | holdout CAGR | holdout max DD | now |
|---|---|---|---|---|---|---|---|---|---|
| always ✔ | 0.482 | 32.0% | 100% | 0.0 | +10.9% | 37.3% | -6.8% | 26.1% | 100% |
| trend | 0.431 | 25.6% | 59% | 2.4 | +7.1% | 24.3% | -2.6% | 9.2% | 0% |
| logit | 0.276 | 31.4% | 63% | 2.94 | +4.5% | 41.1% | -8.7% | 26.2% | 50% |
| blend | 0.376 | 28.3% | 61% | 2.24 | +5.5% | 36.0% | -5.5% | 17.8% | 25% |
| blend_vt | 0.228 | 27.4% | 55% | 2.1 | +3.6% | 38.7% | -4.0% | 15.9% | 16% |
Ranking+exposure = the top-quintile portfolio with exposure applied monthly: cutting exposure sells holdings pro rata (costed at 15 bps/side), the rest earns the cash rate (US 3-month bill as a SAR proxy via the peg). Candidates and the selection rule were declared before any result was seen: dev months only: among candidates with lower max drawdown than buy-and-hold, highest Sharpe; must beat buy-and-hold Sharpe by >0.10 else 'always' (100%). Holdout columns are informational only (the holdout is already contaminated, REVIEW R6-4).
Live tracking — every recommendation date, measured as results come in
| As of | Model | Sessions elapsed | Status | Q1 top | Q5 bottom | Benchmark | Q1 vs bench |
|---|---|---|---|---|---|---|---|
| 2026-10-27 | v20261029-39712336 | 0 / 63 | running | +0.0% | +0.0% | +0.0% | +0.0% |
This is the only test free of backtest bias. A new row is added every time the model is retrained; give it a year before judging.
Prospective candidates — declared in advance, judged only on future results
Alt horizon 252 (~12m): shipped-candidate holdout IC +0.072 (t +4.38) — kept as a candidate because the frozen rules chose the shorter horizon on development evidence.
Factor candidate low_vol: holdout IC +0.086 (t +2.39, Q1−Q5 +2.0%) — picking it now from holdout results would be snooping, so it waits for forward evidence.
| Candidate | As of | Window | Status | Top quintile | Benchmark | vs bench |
|---|---|---|---|---|---|---|
| low-vol rule, 63-session | 2026-08-31 | → 2026-10-27 | running | -10.7% | -15.0% | +4.3% |
| low-vol rule, 63-session | 2026-09-01 | → 2026-10-27 | running | -10.8% | -15.0% | +4.2% |
| low-vol rule, 63-session | 2026-09-02 | → 2026-10-27 | running | -9.7% | -14.5% | +4.8% |
| low-vol rule, 63-session | 2026-09-03 | → 2026-10-27 | running | -18.5% | -14.8% | -3.7% |
| low-vol rule, 63-session | 2026-09-06 | → 2026-10-27 | running | -19.7% | -14.8% | -5.0% |
| low-vol rule, 63-session | 2026-09-07 | → 2026-10-27 | running | -18.6% | -14.4% | -4.2% |
| low-vol rule, 63-session | 2026-09-08 | → 2026-10-27 | running | -18.8% | -14.5% | -4.3% |
| low-vol rule, 63-session | 2026-09-09 | → 2026-10-27 | running | -18.3% | -14.3% | -3.9% |
| low-vol rule, 63-session | 2026-09-10 | → 2026-10-27 | running | -17.9% | -14.3% | -3.7% |
| low-vol rule, 63-session | 2026-09-13 | → 2026-10-27 | running | -16.9% | -13.2% | -3.7% |
| low-vol rule, 63-session | 2026-09-14 | → 2026-10-27 | running | -17.0% | -13.1% | -3.9% |
| low-vol rule, 63-session | 2026-09-15 | → 2026-10-27 | running | -15.8% | -11.7% | -4.1% |
| low-vol rule, 63-session | 2026-09-16 | → 2026-10-27 | running | -16.4% | -11.4% | -5.0% |
| low-vol rule, 63-session | 2026-09-17 | → 2026-10-27 | running | -16.6% | -11.9% | -4.7% |
| low-vol rule, 63-session | 2026-09-20 | → 2026-10-27 | running | -7.4% | -10.9% | +3.5% |
| low-vol rule, 63-session | 2026-09-21 | → 2026-10-27 | running | -14.9% | -10.3% | -4.6% |
| low-vol rule, 63-session | 2026-09-22 | → 2026-10-27 | running | -14.9% | -10.9% | -4.0% |
| low-vol rule, 63-session | 2026-09-24 | → 2026-10-27 | running | -13.8% | -10.2% | -3.6% |
| low-vol rule, 63-session | 2026-09-27 | → 2026-10-27 | running | -13.8% | -10.4% | -3.4% |
| low-vol rule, 63-session | 2026-09-28 | → 2026-10-27 | running | -12.7% | -9.4% | -3.3% |
| low-vol rule, 63-session | 2026-09-29 | → 2026-10-27 | running | -11.4% | -8.2% | -3.3% |
| low-vol rule, 63-session | 2026-09-30 | → 2026-10-27 | running | -11.1% | -7.5% | -3.6% |
| low-vol rule, 63-session | 2026-10-01 | → 2026-10-27 | running | -8.9% | -7.1% | -1.8% |
| low-vol rule, 63-session | 2026-10-04 | → 2026-10-27 | running | -10.4% | -8.2% | -2.3% |
| low-vol rule, 63-session | 2026-10-05 | → 2026-10-27 | running | -12.1% | -8.7% | -3.4% |
| low-vol rule, 63-session | 2026-10-06 | → 2026-10-27 | running | -14.3% | -9.7% | -4.6% |
| low-vol rule, 63-session | 2026-10-07 | → 2026-10-27 | running | -14.2% | -9.3% | -4.9% |
| low-vol rule, 63-session | 2026-10-08 | → 2026-10-27 | running | -12.1% | -7.5% | -4.7% |
| low-vol rule, 63-session | 2026-10-11 | → 2026-10-27 | running | -12.1% | -7.3% | -4.8% |
| low-vol rule, 63-session | 2026-10-12 | → 2026-10-27 | running | -11.4% | -6.5% | -4.9% |
| low-vol rule, 63-session | 2026-10-13 | → 2026-10-27 | running | -11.7% | -6.6% | -5.1% |
| low-vol rule, 63-session | 2026-10-14 | → 2026-10-27 | running | -9.2% | -4.6% | -4.6% |
| low-vol rule, 63-session | 2026-10-15 | → 2026-10-27 | running | -7.1% | -3.8% | -3.3% |
| low-vol rule, 63-session | 2026-10-18 | → 2026-10-27 | running | -7.7% | -4.5% | -3.2% |
| low-vol rule, 63-session | 2026-10-19 | → 2026-10-27 | running | -6.2% | -3.8% | -2.4% |
| low-vol rule, 63-session | 2026-10-20 | → 2026-10-27 | running | -3.8% | -1.8% | -2.0% |
| low-vol rule, 63-session | 2026-10-21 | → 2026-10-27 | running | -1.8% | -0.8% | -0.9% |
| low-vol rule, 63-session | 2026-10-22 | → 2026-10-27 | running | +2.7% | +2.8% | -0.1% |
| low-vol rule, 63-session | 2026-10-25 | → 2026-10-27 | running | +9.5% | +7.7% | +1.8% |
| low-vol rule, 63-session | 2026-10-26 | → 2026-10-27 | running | +4.2% | +3.0% | +1.2% |
| low-vol rule, 252-session | 2026-08-31 | → 2026-10-27 | running | -10.7% | -15.0% | +4.3% |
| low-vol rule, 252-session | 2026-09-01 | → 2026-10-27 | running | -10.8% | -15.0% | +4.2% |
| low-vol rule, 252-session | 2026-09-02 | → 2026-10-27 | running | -9.7% | -14.5% | +4.8% |
| low-vol rule, 252-session | 2026-09-03 | → 2026-10-27 | running | -18.5% | -14.8% | -3.7% |
| low-vol rule, 252-session | 2026-09-06 | → 2026-10-27 | running | -19.7% | -14.8% | -5.0% |
| low-vol rule, 252-session | 2026-09-07 | → 2026-10-27 | running | -18.6% | -14.4% | -4.2% |
| low-vol rule, 252-session | 2026-09-08 | → 2026-10-27 | running | -18.8% | -14.5% | -4.3% |
| low-vol rule, 252-session | 2026-09-09 | → 2026-10-27 | running | -18.3% | -14.3% | -3.9% |
| low-vol rule, 252-session | 2026-09-10 | → 2026-10-27 | running | -17.9% | -14.3% | -3.7% |
| low-vol rule, 252-session | 2026-09-13 | → 2026-10-27 | running | -16.9% | -13.2% | -3.7% |
| low-vol rule, 252-session | 2026-09-14 | → 2026-10-27 | running | -17.0% | -13.1% | -3.9% |
| low-vol rule, 252-session | 2026-09-15 | → 2026-10-27 | running | -15.8% | -11.7% | -4.1% |
| low-vol rule, 252-session | 2026-09-16 | → 2026-10-27 | running | -16.4% | -11.4% | -5.0% |
| low-vol rule, 252-session | 2026-09-17 | → 2026-10-27 | running | -16.6% | -11.9% | -4.7% |
| low-vol rule, 252-session | 2026-09-20 | → 2026-10-27 | running | -7.4% | -10.9% | +3.5% |
| low-vol rule, 252-session | 2026-09-21 | → 2026-10-27 | running | -14.9% | -10.3% | -4.6% |
| low-vol rule, 252-session | 2026-09-22 | → 2026-10-27 | running | -14.9% | -10.9% | -4.0% |
| low-vol rule, 252-session | 2026-09-24 | → 2026-10-27 | running | -13.8% | -10.2% | -3.6% |
| low-vol rule, 252-session | 2026-09-27 | → 2026-10-27 | running | -13.8% | -10.4% | -3.4% |
| low-vol rule, 252-session | 2026-09-28 | → 2026-10-27 | running | -12.7% | -9.4% | -3.3% |
| low-vol rule, 252-session | 2026-09-29 | → 2026-10-27 | running | -11.4% | -8.2% | -3.3% |
| low-vol rule, 252-session | 2026-09-30 | → 2026-10-27 | running | -11.1% | -7.5% | -3.6% |
| low-vol rule, 252-session | 2026-10-01 | → 2026-10-27 | running | -8.9% | -7.1% | -1.8% |
| low-vol rule, 252-session | 2026-10-04 | → 2026-10-27 | running | -10.4% | -8.2% | -2.3% |
| low-vol rule, 252-session | 2026-10-05 | → 2026-10-27 | running | -12.1% | -8.7% | -3.4% |
| low-vol rule, 252-session | 2026-10-06 | → 2026-10-27 | running | -14.3% | -9.7% | -4.6% |
| low-vol rule, 252-session | 2026-10-07 | → 2026-10-27 | running | -14.2% | -9.3% | -4.9% |
| low-vol rule, 252-session | 2026-10-08 | → 2026-10-27 | running | -12.1% | -7.5% | -4.7% |
| low-vol rule, 252-session | 2026-10-11 | → 2026-10-27 | running | -12.1% | -7.3% | -4.8% |
| low-vol rule, 252-session | 2026-10-12 | → 2026-10-27 | running | -11.4% | -6.5% | -4.9% |
| low-vol rule, 252-session | 2026-10-13 | → 2026-10-27 | running | -11.7% | -6.6% | -5.1% |
| low-vol rule, 252-session | 2026-10-14 | → 2026-10-27 | running | -9.2% | -4.6% | -4.6% |
| low-vol rule, 252-session | 2026-10-15 | → 2026-10-27 | running | -7.1% | -3.8% | -3.3% |
| low-vol rule, 252-session | 2026-10-18 | → 2026-10-27 | running | -7.7% | -4.5% | -3.2% |
| low-vol rule, 252-session | 2026-10-19 | → 2026-10-27 | running | -6.2% | -3.8% | -2.4% |
| low-vol rule, 252-session | 2026-10-20 | → 2026-10-27 | running | -3.8% | -1.8% | -2.0% |
| low-vol rule, 252-session | 2026-10-21 | → 2026-10-27 | running | -1.8% | -0.8% | -0.9% |
| low-vol rule, 252-session | 2026-10-22 | → 2026-10-27 | running | +2.7% | +2.8% | -0.1% |
| low-vol rule, 252-session | 2026-10-25 | → 2026-10-27 | running | +9.5% | +7.7% | +1.8% |
| low-vol rule, 252-session | 2026-10-26 | → 2026-10-27 | running | +4.2% | +3.0% | +1.2% |
Windows starting strictly after the locked-holdout end only (anything earlier would be backtest evidence). Judge after ~4 matured quarters. Declared in config/settings.yaml → prospective.
Factor panel — top-minus-bottom quintile spread
| Factor | Months | Premium / mo | NW t | Recent 3y / mo | Status |
|---|---|---|---|---|---|
| realized_vol | 184 | +3.02% | 4.37 | +2.28% | DECAYED |
| mom_12_1 | 184 | +1.11% | 3.03 | +1.09% | ok |
Market regime & volatility
| Regime state | down/turbulent |
| Regime method | vol-proxy |
| Conditional vol (ann.) | 20.51% |
| Vol percentile (1y) | 98.8 |
| Persistence (α+β) | 0.956 |
Context only — market-wide series are no longer model inputs.
Training runs (audit log)
| When | Data hash | Verdict | Headline IC | t | CAGR net | Bench CAGR | Duration |
|---|---|---|---|---|---|---|---|
| 2026-10-29 19:23:46 | 4fa1d74fdf | edge | +0.0479 | 2.08 | -6.8% | -8.0% | 1255s |
| 2026-10-29 18:49:22 | 4fa1d74fdf | edge | +0.0479 | 2.08 | -6.8% | -8.0% | 1246s |
| 2026-10-29 08:24:44 | 4fa1d74fdf | no reliable edge | +0.0367 | 2.02 | -8.7% | -8.0% | 1114s |
| 2026-10-29 06:57:17 | 4fa1d74fdf | no reliable edge | +0.0367 | 2.02 | -8.7% | -8.0% | 1161s |
| 2026-10-29 05:38:49 | 4fa1d74fdf | edge | +0.0499 | 2.09 | -6.5% | -8.0% | 1254s |
| 2026-10-29 04:59:01 | 5edb60fb15 | legacy | +0.0678 | — | +14.6% | +10.8% | 905s |
| 2026-10-29 04:15:44 | 6e0cf9a1f7 | legacy | +0.1655 | — | +16.8% | +11.4% | 418s |
| 2026-10-29 03:40:38 | 6e0cf9a1f7 | legacy | +0.1091 | — | +10.8% | +0.1% | 57s |
Rows marked legacy are from the earlier absolute-return design (and, before 04:59, from corrupted price data); their IC is not comparable — new rows report the locked-holdout IC.