The Research Record

Where Most of the Work Went, and Almost All of the Answers Were No

Section 5 of 5

Roughly a third of all commits in this project are research, not engineering. That ratio is the honest shape of the work: building a trading system is mostly the business of generating ideas and then trying very hard to destroy them, and the code that survives is the small residue of everything that did not.

The labs are structured so that outcome accumulates into a record rather than a feeling. Experiments are numbered, each finding is committed with its conclusion stated in the message, and a later experiment that overturns an earlier one is a new dated entry rather than an edit. The history is therefore auditable in both directions — what was believed, when, and what subsequently killed it.

What follows is a summary of that record rather than a reproduction of it, and deliberately weighted toward the failures. Rejected hypotheses are described freely because they cost nothing to share and show the method; the handful of things that survived are described only in the abstract.

239

research commits — about a third of all work

79

numbered experiments in one lab alone

2023–2026

span of walk-forward validation data

4

independent research labs

~90%

of candidates rejected before live code

The Funnel a Candidate Has to Survive

Each stage exists because something has died there. Most candidates do not reach the third box, and the ones that fail late are the expensive ones — they are the ideas that looked right for longest.

Candidate raisedFitted in-sampleWalk-forward on unseen dataCost and bias auditShadow — recorded, not tradingLivedies here:a plausible story, no measurable effectdies here:strong in-sample, nothing out of sampledies here:the result was selection noisedies here:no edge after fees, or it usedinformation not available at the timedies here:live behaviour diverges from the modelEvery rejection is committed with its reason, so a later idea cannot quietly resurrect a question already closed

The Pitfalls, and How Each Was Caught

The canonical failure is an idea that looks excellent on the data it was tuned against. The record contains it in its purest form: a morning compression setup reaching a profit factor above 2.2 in-sample and around 0.6 out of sample, committed with the conclusion that it was a textbook overfit and recorded as such rather than quietly dropped.

A separate experiment revisited the same figure from a different angle and reclassified it explicitly as selection noise — the strong number was not a measurement of an edge but an artifact of having looked at enough variants that one of them had to look good.

Another entry captures the same lesson structurally: re-learning the model on a rolling cadence left the proportion of matched-good outcomes at essentially the same figure, meaning the cadence reshuffled which cases failed without reducing how many did. That is a far more useful finding than a performance number, because it says the problem is not staleness and closes an entire direction of work.