We Tested 864 Settings. Did the Winner Matter?

Run enough variations of a trading rule and one will usually look impressive. The difficult question is whether it found something repeatable or merely won the search.

Our September 23, 2026 futures research report offers a useful example. A parameter sweep produced a configuration with roughly twice the baseline's simulated net profit. Then the review counted how many chances we had given ourselves to find that winner.

This installment of StonkHub Research Diaries follows that change in perspective. The figures are historical research results, not a live futures track record.

The lead: a weak close might reverse

The rule used internal bar strength, or IBS: the closing price's position within the day's high-to-low range. A close near the low gives a low IBS value. The hypothesis was that some weak closes might be followed by a rebound.

The research combined that idea with other choices: a short-term momentum filter, a longer-term trend filter, holding periods, stop settings, and exit thresholds. Each choice made a slightly different strategy.

The initial long-window experiments used scaled ETF prices as proxies for index futures, with futures cost assumptions. The SPY-based series was scaled by ten for the ES-cost experiment; QQQ was scaled by forty for the NQ-cost experiment.

That helps construct a research screen, but it does not reconstruct actual futures trading. Financing, dividends, contract rolls, market sessions, margin, and execution can differ. The distinction matters before discussing the winner.

The search produced an attractive table

The full grid contained 864 configurations for each proxy. They were evaluated over January 3, 2022, through September 19, 2026. The SPY-based baseline recorded 31 simulated trades and approximately $11,708 net profit on a $25,000 starting-capital basis.

The selected configuration that cleared the research screening criteria recorded 35 trades, approximately $25,103 net profit, a profit factor of 2.52, and a maximum drawdown of 20.2%. Profit factor compares total winning-trade profits with total losing-trade losses.

Those figures look materially better. But the selected configuration was not a prediction written down before the experiment. It emerged from inspecting hundreds of alternatives on the same history.

The QQQ-based sweep was less encouraging: none of its 864 configurations cleared the report's screening bar. An idea that looks useful on one proxy cannot simply inherit that status on another.

Counting the attempts changed the interpretation

The report compared an unadjusted probabilistic Sharpe calculation with a deflated Sharpe calculation. These methods examine estimated risk-adjusted performance; the deflated version also penalizes the opportunity to select a good-looking result from many trials.

For the selected configuration, the reported unadjusted figure was 97.8%. The deflated figure was 10.4% under the report's assumptions. Those are outputs of a statistical model, not a literal probability that the strategy will make money.

The calculation used an independent-trial noise benchmark. The configurations share rules and history, so they are correlated. Its numerical adjustment should therefore be read with that assumption visible, rather than treated as an exact verdict.

There was another warning in the trade list: the three largest winners contributed 53% of the selected configuration's net profit. Thirty-five trades over more than four years provide limited evidence, especially when a few observations carry so much of the result.

The split tests preserved a positive result

The report did run split evaluations. Training on 2022–2024 and testing on 2025–2026 produced 55 test trades and approximately $9,124 net profit. Training on 2025–2026 and replaying 2022–2024 produced 39 test trades and approximately $2,201 net profit. Each split selected its own settings from training data.

That is positive evidence for further investigation. But the settings changed between splits, and the reverse split's average profit per trade fell from about $1,785 in training to $56 in testing, roughly a 96% decline. Training on later years and testing earlier years is a historical stress comparison, not a chronological test of future performance.

The futures check was not a clean confirmation

The report also replayed selected settings on an ES-specific daily series. That sounds closer to the instrument we wanted to study, but the indicator warmup reached into a period containing placeholder data.

It also compared regular-session and extended-session bars. Changing the session changes the high, low, and close used by the signal. A stronger extended-session result is therefore not simply the same strategy passing a second test.

These checks were useful for identifying questions. They were not untouched validation data, and the report's small trade counts and data limitations prevented them from establishing a clean independent confirmation.

What we would do differently

The next experiment would first establish a trustworthy futures series, including indicator warmup and contract-roll handling. It would freeze the selection process before evaluating reserved later data.

We would retain every attempted configuration, compare the chosen rule with the original baseline, and inspect performance without its largest winners. Prospective paper observations could then test signal timing and execution assumptions.

Those are proposed next steps. Reporting the stronger in-sample result again would not supply the missing evidence.

The conclusion

The search found an interesting candidate, not a demonstrated improvement ready for live capital. The number of settings we tried belongs beside the winner's returns.

Historical proxy simulations are not investment advice or live futures results. Trading can lose money. Read the financial disclaimer.

Explore the StonkHub blog or start this series with our Reddit attention experiment.