Nifty 500 weekly ranking with a pooled LightGBM: build, optimisation and test, 2010 to September 2025
Update 31 Oct 2026 (version 2). Frozen v2 re-run on survivorship-free data: validation 2018 to September 2025 21.3% at 1x and 18.7% at 2x (earlier 25.6%), development 2010 to 2017 10.7% (earlier 16.0%) with a 50% drawdown in 2011; correlation with the two Longbow strategies about 0.45. Read the survivorship-free re-test, comparison and Kelly analysis. The figures below are the earlier version (Dhan price history without delisted stocks, 4%/yr allowance) and are kept for the record.
Inspired by the owner's weekly-ranking write-up · specification fixed 31 Oct 2026 before any result (research/rank500/STRATEGY_RANK500.md, with dated disclosures) · ₹50 lakh · out-of-sample walk-forward from 2010 (the earlier studies here started in 2018) · sealed hold-out from October 2025 never loaded
Short answer. The pooled model has real, persistent ranking skill: over 16 out-of-sample years the top predicted decile beat the bottom one by 0.76% a week, monotonically and in every year, and the strategy built on it beat random picks from the same stock universe by a wide margin in both periods. Two portfolio versions were frozen on 2010-2017 data and run once on 2018 to September 2025. Both pass every pre-registered gate; the lower-turnover v2 (N = 30, rebalance Friday every second week, buffer k = 2.0, liquidity floor ₹5 crore, 5-session model) returned 25.6% a year net against 11.8% for the Nifty 500 price index (still 15.9% after the 4% survivorship allowance at three times the assumed costs), with a similar drawdown (36.2% vs 34.4%). The development margin was smaller (16.0% against 10.1%; alpha t = 0.8 after the allowance), so the validation margin should not simply be extrapolated. Three things keep this a candidate, not a proven edge: most of the return comes from small and mid-sized, less liquid stocks in a period when they boomed (2020 to 2023); 6 to 26% of past index members have no price history in our data (survivorship); and the weekly index-direction call has no skill (accuracy 54% against a 59% always-up baseline). The sealed period (October 2025 to October 2026) was opened once, at your instruction: in a falling market (index -5.2%) v2 gained 4.5% a year net against random picks' -8.0%; a forward paper run of v2 starts on 2 November 2026.
1. What the write-up asks for, what we kept and what we changed
Purged expanding walk-forward, early stopping, regularised shallow trees
Kept, with 16 yearly folds (2010-2025) instead of 4
Embargo 6 sessions (11 for the 10-session model); three seeds averaged
Label from the rebalance close
Changed
Label is the return from the next open to the open five sessions later, matching how a close-based signal can actually be traded
Target: raw return
Changed
Target is the cross-sectional rank of that return (market component removed); market direction is a separate head
Holding buffer, N sweep, weekday sweep, empty slots in cash
Kept and extended
Buffer k up to 3, liquidity floor, 10-session model blend, weekly or fortnightly cadence (version 2)
Cost: square-root impact
Changed
Indian delivery charges plus slippage by liquidity tier (v1: 0.05/0.10/0.20%; v2: 0.05/0.10/0.25/0.50%), stressed at 2x and 3x
Selection by robustness and low turnover
Kept, with two safeguards
Selection among configurations positive in at least 60% of development years (and, in v2, alpha t above 1.5 at 2x costs and at least 4 turnovers a year)
Index-direction call from breadth of predictions
Changed
Separate small model on market features; evaluated honestly (section 8)
2. Data and test period
Period: predictions are out-of-sample for 2010-01 to 2025-09, 15.75 years; training starts in 2004. Earlier work on these data tested 2018 to 2025 only. Development = 2010-2017 (all 540 configurations run once; selection), validation = 2018 to September 2025 (one run per frozen version), sealed = October 2025 onward (not loaded).
Universe: Nifty 500 members on each date (point-in-time from 1998), 20-day median traded value at least ₹0.5 crore for the model, and the version's own floor for the portfolio. Daily prices from Dhan, split and bonus jumps removed by the project's rule (a one-day move above 35% is not counted, which also removes real crashes of that size).
Survivorship, measured: the share of Nifty 500 members that have price history in the panel was 60% in 2010, 71% in 2017, 79% in 2020 and 94% in 2025. Missing names are delisted or merged companies, which would have pulled returns down. A 4%/yr allowance is deducted in the gates; results with a larger allowance are in section 9.
Year-end
Index members
With price history
Coverage
2010
500
298
60%
2011
500
306
61%
2012
500
315
63%
2013
500
321
64%
2014
500
325
65%
2015
500
334
67%
2016
501
346
69%
2017
501
357
71%
2018
501
369
74%
2019
501
388
77%
2020
501
395
79%
2021
501
417
83%
2022
501
427
85%
2023
501
440
88%
2024
501
454
91%
2025
501
469
94%
3. The ranking model
Mean five-session return by predicted decile, overall and by liquidity: the ranking is monotone, but the long-side edge is concentrated in less liquid stocksRank IC by test year
Average daily rank IC across the 16 test years: 0.076 (range 0.041 to 0.114). Top-decile minus bottom-decile spread: 0.76% per five sessions overall. In stocks with more than ₹100 crore of daily traded value the top decile earns about the same as the average stock (0.45% vs 0.32%); the long-side edge sits in the ₹5 to ₹100 crore range and below. This is why the liquidity floor and the cost tiers matter.
4. Portfolio versions and results
v1 (as inspired, v1 costs): N = 30, rebalance Friday every week, buffer k = 1.0, liquidity floor ₹0.5 crore, 5-session model. v2 (improved, v2 costs): N = 30, rebalance Friday every second week, buffer k = 2.0, liquidity floor ₹5 crore, 5-session model. Both configurations were chosen on 2010-2017 only, by the pre-registered rules; validation was run once for each, together.
v2: the recommended version
Period
Costs
Net CAGR
After 4%/yr allowance
Nifty 500 price
Sharpe
Max drawdown
Index drawdown
Alpha after allowance
Years ahead of index
Turnover (× capital a year)
Costs a year
Development 2010-2017-12
1x
16.0%
12.0%
10.1%
1.00
25.8%
30.4%
1.8% (t 0.8)
75%
4.1
1.3%
Development 2010-2017-12
2x
14.3%
10.3%
10.1%
0.91
27.8%
30.4%
0.4% (t 0.2)
62%
4.2
2.7%
Development 2010-2017-12
3x
12.6%
8.6%
10.1%
0.82
29.7%
30.4%
-1.0% (t -0.5)
62%
4.2
4.0%
Validation 2018-2025-09
1x
25.6%
21.6%
11.8%
1.22
36.2%
34.4%
7.1% (t 2.2)
88%
7.4
2.0%
Validation 2018-2025-09
2x
22.7%
18.7%
11.8%
1.12
36.0%
34.4%
4.9% (t 1.5)
88%
7.5
4.0%
Validation 2018-2025-09
3x
19.9%
15.9%
11.8%
1.00
37.0%
34.4%
2.6% (t 0.8)
75%
7.5
6.0%
v1: close to the original design
Period
Costs
Net CAGR
After 4%/yr allowance
Nifty 500 price
Sharpe
Max drawdown
Index drawdown
Alpha after allowance
Years ahead of index
Turnover (× capital a year)
Costs a year
Development 2010-2017-12
1x
22.0%
18.0%
10.1%
1.37
21.9%
30.4%
7.6% (t 3.0)
88%
21.1
6.4%
Development 2010-2017-12
2x
13.8%
9.8%
10.1%
0.92
29.3%
30.4%
0.7% (t 0.3)
88%
21.5
13.1%
Development 2010-2017-12
3x
6.2%
2.2%
10.1%
0.46
42.0%
30.4%
-6.3% (t -2.5)
12%
21.8
20.1%
Validation 2018-2025-09
1x
29.0%
25.0%
11.8%
1.26
39.7%
34.4%
9.7% (t 2.3)
88%
28.1
7.0%
Validation 2018-2025-09
2x
19.2%
15.2%
11.8%
0.91
41.3%
34.4%
1.8% (t 0.4)
62%
28.5
14.5%
Validation 2018-2025-09
3x
10.1%
6.1%
11.8%
0.55
49.8%
34.4%
-6.1% (t -1.5)
50%
28.8
22.3%
Growth of ₹1 against the Nifty 500 price indexCost stress and yearly excess return in validation
Year
v2 net return
v1 net return
Nifty 500 price index
2010
17.7%
22.7%
13.1%
2011
-18.4%
-14.1%
-27.2%
2012
32.3%
24.0%
31.8%
2013
16.2%
20.7%
3.6%
2014
49.0%
70.3%
37.8%
2015
10.3%
9.3%
-0.7%
2016
0.5%
14.5%
3.8%
2017
33.5%
45.7%
35.9%
2018
0.7%
3.2%
-3.4%
2019
7.6%
3.5%
7.7%
2020
50.3%
49.7%
16.7%
2021
60.5%
76.5%
30.2%
2022
8.9%
24.6%
3.0%
2023
46.2%
53.3%
25.8%
2024
25.5%
30.2%
15.2%
2025
12.0%
2.2%
1.6%
2025 is January to September; 2018 to 2025 returns restart from cash on 1 January 2018.
5. Optimisation: what the grids showed
v1 grid on development data: net CAGR by N and weekday for three buffersv2 grid: mean net CAGR by buffer and liquidity floor at 1x and 2x costs
Turnover is the lever. Without a buffer the book turns over about 70 times a year and costs eat most of the edge; k = 1 cuts costs to about 7% a year, k = 2 with fortnightly rebalancing to about 1.3% (v2).
Liquidity floors trade return for robustness. Raising the floor from ₹0.5 to ₹5 to ₹20 crore lowers mean development CAGR from 21.9% to 17.3% to 13.3% (1x v2 costs) but makes the result less dependent on the slippage assumption.
Weekday and N matter little once the buffer is wide (spread across weekdays and between N = 20 and 30 is within a few points).
The 10-session blend and fortnightly cadence both help a little at 2x costs.
Degenerate setting caught: with k = 3 the book is almost never refreshed (buy-and-hold of the first picks); the selection rule was corrected before validation to require at least four turnovers a year (disclosure 9.1 in the specification).
Random selection (same universe, N, cadence, buffer and costs, 100 seeds) earns about the index level or less; the model is far above its 90th percentile in both periods. That is the strongest evidence of ranking skill.
Simple baselines do not do the same job consistently: 20-day momentum ranking earned 23.7% in validation (with a 58% drawdown) but only 7.6% in development; 5-day reversal lost money after costs.
Reality check across all 540 development configurations: best t = 5.29, p = 0.000. Plateau: every one-step neighbour of the frozen configurations was positive in both periods.
Market regimes (validation): the strategy gained over the index in low and medium-VIX weeks. In the nine weeks of the highest VIX regime (March 2020) it fell with the market and slightly more than the index.
7. Pre-registered pass rule
v2 condition
Result
Figures
1. Validation net CAGR after the 4%/yr allowance at least the index, alpha t above 1.5 after the allowance
pass
21.6% vs 11.8%; t = 2.16
2. Net excess over the index positive in at least 60% of validation years
pass
88% of years
3. Still ahead of the index (after the allowance) at 2x costs
pass
18.7% vs 11.8%
4. Development CAGR above the 90th percentile of random selection; reality check p below 0.10
pass
16.0% vs null p90 8.2%; reality-check p = 0.000
5. Maximum drawdown within the index's plus 10 points; worst year above −25%
pass
drawdown 36.2% vs index 34.4%; worst year 0.7%
6. Plateau: at least 60% of neighbouring configurations positive in both periods
pass
100% of 11 neighbours
7. Shifted sample (first two years dropped) keeps the sign
pass
alpha 3.1%, t = 1.31
v1 condition
Result
Figures
1. Validation net CAGR after the 4%/yr allowance at least the index, alpha t above 1.5 after the allowance
pass
25.0% vs 11.8%; t = 2.33
2. Net excess over the index positive in at least 60% of validation years
pass
88% of years
3. Still ahead of the index (after the allowance) at 2x costs
pass
15.2% vs 11.8%
4. Development CAGR above the 90th percentile of random selection; reality check p below 0.10
pass
22.0% vs null p90 -3.1%; reality-check p = 0.000
5. Maximum drawdown within the index's plus 10 points; worst year above −25%
pass
drawdown 39.7% vs index 34.4%; worst year 2.2%
6. Plateau: at least 60% of neighbouring configurations positive in both periods
pass
100% of 6 neighbours
7. Shifted sample (first two years dropped) keeps the sign
pass
alpha 8.5%, t = 3.13
All seven conditions pass for both versions at their assumed costs. v1 fails at 3x costs (10.1% against the 11.8% index) while v2 still earns 19.9% before and 15.9% after the allowance: another reason to prefer v2.
7b. Sealed hold-out, opened once (October 2025 to 28 October 2026)
On your instruction the sealed period was opened one time on the two frozen configurations (ledger: research/rank500/holdout_ledger.json). One model was trained on rows whose label ended before 1 October 2025 (same parameters and seeds) and the strategies started from cash. No gate was written for this period; it is reported as the last independent check. Window length is about 13 months, so statistical power is low.
Version
Net CAGR 1x costs
2x
3x
After 4%/yr allowance
Nifty 500 price index
Max drawdown vs index
Alpha after allowance
Random selection (median / 90th pct)
Turnover
v2
4.5%
3.1%
1.8%
0.5%
-5.3%
16.5% vs 13.1%
7.6% (t 1.2)
-8.0% / -2.0%
5.6
v1
10.9%
5.2%
-0.2%
6.9%
-5.3%
14.0% vs 13.1%
12.8% (t 2.0)
-19.0% / -11.5%
23.0
In a market that fell 5.2% over the window, v2 earned 4.5% a year net (the model beat the 90th percentile of random selection, -2.0%) but only 0.5% after the survivorship allowance, with alpha t = 1.2, below the 1.5 used in the validation gate. v1 earned 10.9% but fell to -0.2% at three times costs. The sealed period therefore supports the ranking skill (both versions are far above random picks and the index) without confirming a large absolute return; it also shows again that v2's low turnover is what keeps it cost-robust.
7c. Charts and trade details (v2, all three windows)
The same v2 book in each window: growth against the index, drawdown, how much of the book was invested, the shape of the trades, where the money was made, and the largest winners and losers. Per-trade, per-fill and per-day files are on the strategy page, which has the interactive equity chart, monthly table and the full trade log.
Per window: growth of ₹1 against the Nifty 500 price index, drawdown against the index, and the share of the book invested (the rest is cash).Distribution of round-trip returns (after slippage and charges are in the P&L; the return is price-to-price).Holding periods. The buffer keeps winners, so many positions last months.Realised P&L and costs by year (sealed window shown with its own exits).
Which names made and lost the money
Across all three windows the 10 best names contributed 20% of total net P&L of ₹30,831,434 (394 names traded, 1594 round trips).
Best 10 names
Trades
Net P&L
Avg trade
LUPIN
10
₹842,592
31.3%
CHOICEIN
2
₹833,453
74.0%
GLENMARK
16
₹670,608
8.0%
EICHERMOT
7
₹639,013
31.9%
TATAMOTORS
7
₹573,477
21.0%
TRENT
5
₹547,474
16.2%
MARUTI
7
₹504,330
24.9%
TATAELXSI
8
₹501,734
20.3%
BAJAJ-AUTO
9
₹496,212
13.4%
MCX
4
₹485,394
19.6%
Worst 10 names
Trades
Net P&L
Avg trade
HEROMOTOCO
12
₹-316,378
-3.1%
OFSS
11
₹-224,226
-6.3%
CHAMBLFERT
7
₹-219,914
-1.7%
KIMS
2
₹-214,896
-18.2%
INDIACEM
6
₹-212,880
-4.7%
SIEMENS
7
₹-210,552
-0.0%
UTIAMC
1
₹-198,435
-22.7%
NTPC
7
₹-196,922
-8.4%
BALRAMCHIN
7
₹-196,777
-2.0%
VTL
1
₹-195,192
-41.3%
Ten best and ten worst round trips
Name
Entry
Exit
Days
Entry price*
Exit price*
Return
Net P&L
Costs
Window
CHOICEIN
2023-10-16
2024-11-25
406
2.83
7.29
156.2%
₹876,566
₹5,621
validation
MARUTI
2014-01-20
2016-12-26
1071
4.73
13.81
191.2%
₹462,418
₹1,913
development
GLENMARK
2025-02-17
2025-09-08
203
36.21
56.57
55.7%
₹437,116
₹4,217
validation
TATAMOTORS
2023-11-13
2024-07-29
259
5.51
9.49
71.6%
₹428,624
₹3,094
validation
DIXON
2023-11-13
2024-05-21
190
9.12
15.50
69.4%
₹415,396
₹3,069
validation
ZYDUSLIFE
2024-01-23
2024-05-21
119
18.45
29.28
58.1%
₹410,755
₹3,829
validation
INDHOTEL
2022-10-17
2024-02-26
497
8.30
16.02
92.5%
₹408,599
₹2,458
validation
TORNTPHARM
2023-11-28
2024-12-23
391
79.91
131.29
63.7%
₹397,237
₹3,443
validation
MCX
2025-03-03
2025-06-09
98
3.86
5.84
50.8%
₹388,629
₹4,623
validation
SUNDARMFIN
2023-05-15
2024-08-26
469
40.53
73.07
79.5%
₹387,485
₹4,012
validation
Name
Entry
Exit
Days
Entry price*
Exit price*
Return
Net P&L
Costs
Window
HEROMOTOCO
2024-09-23
2025-02-17
147
13.18
8.38
-36.8%
₹-348,075
₹2,999
validation
BPCL
2024-09-09
2025-02-17
161
9.05
6.45
-29.0%
₹-266,022
₹3,032
validation
SIEMENS
2024-12-23
2025-04-07
105
65.07
46.75
-28.5%
₹-248,916
₹2,902
validation
CUMMINSIND
2024-06-03
2025-02-17
259
41.15
30.44
-26.4%
₹-215,179
₹2,744
validation
INDIACEM
2024-10-21
2025-02-17
119
7.59
5.93
-22.3%
₹-206,816
₹4,007
validation
CHAMBLFERT
2025-04-28
2025-08-04
98
28.36
21.98
-22.9%
₹-201,399
₹3,353
validation
COLPAL
2024-08-26
2024-12-23
119
40.92
32.07
-22.0%
₹-199,575
₹3,124
validation
UTIAMC
2024-12-23
2025-03-03
70
2.54
1.98
-22.7%
₹-198,435
₹4,791
validation
VTL
2022-04-11
2022-06-13
63
17.12
10.13
-41.3%
₹-195,192
₹2,262
validation
KIMS
2025-01-20
2025-03-03
42
3.20
2.49
-22.7%
₹-192,044
₹3,650
validation
* Prices are split-adjusted to today by the project's rule, so they can differ from the quoted price on that day. Every trade, fill and cost is in the downloadable files on the review page (trade log and audit tables).
8. The index-direction call
Weekly direction call versus the majority-direction baseline
A small LightGBM on market-context features (index returns and moving averages, volatility, VIX level and regime, breadth, drawdown from the 252-day high) predicts whether the Nifty 500 is higher five sessions later. Out of sample, Fridays 2010-2025: accuracy 53.7%, AUC 0.51 (0.50 is chance), against 56.8% for always predicting the majority direction. In validation: accuracy 53.8%, AUC 0.51. A rule that moves to cash on a bearish call reduced validation CAGR from 25.6% to 18.7% for v2. The weekly index call has no demonstrated skill and should not be used.
9. What could make this too good to be true, and the checks
Small and illiquid stocks. The edge sits where costs are most uncertain. v2 uses a ₹5 crore floor, 0.5% slippage per fill below ₹5 crore and 0.25% up to ₹20 crore; at three times those costs it still earns 19.9% in validation. Real fills in these names should be measured before capital is risked (position size is about ₹1.7 lakh, small against ₹5 crore of daily value).
Survivorship. With coverage of 74% to 94% in validation and 60% to 71% in development, results are overstated by an unknown amount. With an 8%/yr allowance instead of 4%, v2 validation would still be 17.6% against 11.8% and development 8.0% against 10.1%: validation holds, development does not.
Size and beta. Portfolio beta to the Nifty 500 is 1.10, and 2020 to 2023 were strong years for small and mid-sized stocks: the year table shows v2 returns of 50%, 61%, 9% and 46% in 2020 to 2023; the gains over the index were smallest in 2018, 2019 and 2022. Part of the "alpha" is a size tilt; random selection from the same universe is the fairer comparison and the model is far ahead of it.
Execution. Orders are assumed to fill at the next open plus slippage; names that open locked at a circuit limit (zero-range days) are skipped for entry and kept on exit. Partial circuits and auction-session effects are not modelled. Corporate-action adjustment and the 35% jump rule can hide real crashes.
Overfitting. 540 configurations on development; the reality check, the random null, the plateau table, the shifted sample and one-time validation are the controls. Version 2 and its rule corrections were decided on development data, but only after the v1 development results were seen, and validation was run after the freeze.
Bug history (disclosed). The first v1 development run misaligned prediction columns with price columns; it was fixed and rerun before any validation result.
10. Recommendation and next steps
Forward paper run of v2: approved and scheduled. Frozen specification research/rank500/paper_spec.json; paper start Monday 2 November 2026, ₹50 lakh notional, decisions every second Friday close (first: 6 November), orders recorded for the next open with the v2 cost schedule, a model trained at the start and refitted each January. No orders or broker calls. A nightly job (19:50 IST) replays the run and updates the forward section of the review page.
Sealed period: opened once (section 7b) and now spent for this strategy; it becomes training data for the forward models.
Measure real execution costs on a small test basket in the ₹5 to ₹20 crore names before sizing up: this needs real orders and therefore your explicit instruction; none have been placed.
Survivorship gap: work in progress (price histories of delisted and merged members from NSE archives); results are rerun if the data can be recovered.
Do not use the weekly index call.
11. Sources and files
Prices and membership: Dhan daily bars and NSE Nifty 500 constituent snapshots held in the project (data/research); India VIX and Nifty 500 index: Yahoo Finance series held in data/niftyscalp. Code, pre-registration, results and figures: research/rank500/. Education and research only; not investment advice.
Research and education only. Hypothetical and back-tested results have inherent limitations and do not guarantee future results. Not investment advice; the publisher is not presented as a SEBI-registered investment adviser or research analyst.