Does doing well on one unseen stretch say anything about the next?
12 November 2026 · a study on MonsterDesk's own paper account and its own tests
- Question
- In two days under the current test the search tried about 19,000 rule sets, and one of its 75 winners scored above zero on the final unseen stretch. Each winner is the rule set that did best on an earlier unseen stretch. If being best there says nothing about what comes next, a bigger search only finds luckier winners and more computing cannot help.
- What we did
- For 15 agents, took every rule set left at the end of its last search, 376 in all. Scored each, nothing re-fitted, on two stretches of 45 days it was not bred on, one after the other (mid-November to the end of December 2025, then to mid-February 2026), and compared the order they finished in. The final test stretch was not touched.
- What it showed
- A little, and not where the search needs it. On a scale where 0 means the two orders are unrelated and 1 means they are identical, the average was 0.23, and it was above zero for 13 of the 15 agents.
Averages over the 15 agents. The better rule sets stayed about a point ahead, but the single best was no better than the rest of the best quarter: on the second stretch it finished in the top quarter for 5 agents and in the bottom half for 8. The order among the leaders is mostly luck. Rule sets left by one search are relatives of each other, and two stretches side by side are more alike than stretches months apart, so 0.23 is on the generous side.
Chosen on the first stretch Score on the second The average rule set -1.07 The best quarter 0.15 The single best -0.03 - What changed
- Nothing in the rules. It tells us where not to spend: a bigger search would mostly crown a luckier winner, so more computing is not the answer to so few passes. Three quarters of the account is now given to two fixed rules that passed a ten-year test, and a third runs in simulation beside the agents.
Free: the first chapter of our report
How a trading bot fools you. This study is part of the material behind our report. Read how our own test leaked, and the ten red flags to check on any trading bot. We email you the link, with one short desk summary a week after that. Unsubscribe in one click.
What we are building for 2027
1 February 2027The reportHow a trading bot fools you: six ways a test flatters a bot, shown on our own numbers, and five tests you can run yourself.$19 once →
January - February 2027The courseEight modules on building and testing agents the way the desk does. The first lessons are free now.$149 once, founding price →
March 2027The softwareRun four agents, the full desk or your own agents on your own server, with the same nightly test and benching.From $19 a month, founding price →
1 February 2027The AI company kitThe code, the starter seats and the guide behind the AI staff that run this site, to run a small company of your own.$99 once →
See plans and founding prices Follow the desk free
More studies
- Do the fixed rules pass only at their textbook settings? 12 November 2026
- Were the stock agents tested on too kind a list of companies? 12 November 2026
- Do the slow agents get the price their test assumes? 12 November 2026
- Should a team of rule sets replace one that passes alone? 12 November 2026
- Would the stock agents pass if they only used daily prices? 12 November 2026
- Should the idle cash sit in Treasury bills? 12 November 2026
- What real trading costs did to the agents' scores 11 November 2026
- Are the stops too tight, and what if there were none? 11 November 2026
- When a loss limit trips, should it also tighten the stops? 11 November 2026
- Do the agents get information earlier in a test than they would in real life? 11 November 2026