Research and development
What we are testing, what each study found, and what we changed because of it. That includes the tests that went against us.
Everything on this page is a test. Experiments trade simulated money on our server, apart from the desk's paper account, and are not copied to members. A study replays rules on real past prices. Neither says what will happen next.
Running now
Faster decisions. Two versions decide every 15 minutes instead of every hour, on the same markets as mixed-medium, each with its own simulated $100,000. One is allowed to be very picky; the other must trade about three times a week in every test, and a set of rules that trades less than that fails. Once both have 20 closed trades, the one with more profit leads.
The same learning and tests as the hourly agents. It trades separate simulated money on our server: it never touches the desk's account, isn't copied to members and isn't part of the desk's return. Replayed over the last 30 days its current rules made trades, +1.60%.
Running since 10 Nov 2026. Win rate 50%.
The same learning and tests as the hourly agents. It trades separate simulated money on our server: it never touches the desk's account, isn't copied to members and isn't part of the desk's return. Replayed over the last 30 days its current rules made trades, +3.61%.
Running since 10 Nov 2026. Win rate –.
What we found
Each study starts with a question, runs on real past prices without changing anything live, and ends with what we did about it.
What real trading costs did to the agents' scores
- Question
- The test each agent sits charged a small flat cost per trade. Is that what trading really costs us?
- What we did
- Measured every fill the desk has made against the price the agent decided on, market by market, and charged each market its own figure in the test: about 0.7% a side for small coins and 0.025% for stocks. A stop that fires in a fast market is charged more, as it is in real life.
- What it showed
- Scores fell hard. One stock agent went from 3.70 to -3.27 and another from 3.43 to -1.14. After that learn, 16 of the 18 agents were on the bench. The rule sets that held up were the ones that trade less often, on longer bars.
- What changed
- The test now charges the costs we measure, and re-measures them at every learn. An agent that only looked good because trading was nearly free in its test stays on the bench until it finds rules that survive real costs.
Are the stops too tight, and what if there were none?
- Question
- Asked after the desk lost about $1,100 in one day: would the agents do better with their stops further away, or with no stop order at all?
- What we did
- Replayed 14 agents' current rules on three months of prices they had never learnt from, seven ways: as they trade now, with the stop 1.5, 2 and 3 times further away, with no stop, and with a wider stop on a smaller position.
- What it showed
- No stop was the worst choice: the lowest score, and a worst single loss of $2,342 against $921 with stops. Stops twice as far away did a little better than today's (+0.53% against +0.09% on average), but that is a small edge on three months. Six agents had settled on the tightest stop the learning allowed. And the day's loss was not about stop distance: one small-coin trade filled 2% above its decision price and its stop filled 2% below the stop.
- What changed
- The learning now chooses stops from a wider range and can no longer pick the tightest setting. A coin outside the three largest may be at most 2% of the account. Stops stay.
When a loss limit trips, should it also tighten the stops?
- Question
- When the account is down 3% in 24 hours or 6% in 7 days, the desk stops opening positions. The rule also moved every open position's stop closer. Nobody had tested that second part.
- What we did
- Replayed the whole desk on 360 days of real prices. At every hour a limit would have tripped, we copied the desk and played the same moment forward five days in three ways: no limit, no new positions, and no new positions plus tighter stops.
- What it showed
- Tighter stops did not protect the account.
Average change in a $100,000 account over the next five days. Five cases at the live level is a small sample, so the decision also leans on the stop study above, which pointed the same way.
Limit at Times it tripped No new positions Plus tighter stops 3.0% (the live rule) 5 -$744 -$1,365 2.0% 17 -$661 -$694 1.5% 25 -$32 -$214 1.0% 39 -$236 -$226 - What changed
- A loss limit now stops new positions and nothing else. Open positions keep the stops they were tested with. The limit itself stays: five cases is too few to remove a safety control.
Do the agents get information earlier in a test than they would in real life?
- Question
- Some agents use company-insider filings and how much attention a stock is getting. If a test shows those to an agent even a few hours early, its score is flattered by knowledge it could not have had.
- What we did
- Scored seven agents three ways: as the learning sees them, with those two feeds delayed by a full day, and with them removed. If the delayed score holds, the edge is real. If it collapses to the score with the feeds removed, the edge came from timing.
- What it showed
- No timing leak in any of the seven. The check itself had one fault: it raised a false alarm for an agent where the feeds added nothing either way.
- What changed
- The false alarm was fixed. Nothing changed in the agents.
Next on the bench
- Should a team of rule sets need a clear lead before it replaces one that passes alone? Two agents are benched on their team's score although their best single rule set passes.
- Loss limits per agent, sized to its share of the account. Today the limits watch the whole account.
- Do the slow agents enter at the same moment their test assumes? Agents that decide on daily prices are checked hourly when live.
- Fifteen-minute decisions. The experiment above is decided when both versions have 20 closed trades.
Members' ideas
Members can suggest something for the desk to test. We read every idea, the ones we take on join the list above, and we publish how each test came out. Most ideas fail the test, ours included, and taking an idea on is not a promise to trade it.
Ideas come from members. Join free or log in to send one.
See the agents that passed
The desk's 18 agents, their test scores and who is on the bench today.