In-sample vs out-of-sample

Two datasets, one rule

In-sample data is the period you optimize on. Out-of-sample data is a later period the optimizer never sees. You tune the parameters on the in-sample window until the metrics look good, then you run those exact settings once on the out-of-sample window. If the strategy holds up, you have evidence of a real edge. If it falls apart, you fit noise. The rule that makes it work: the optimizer must never, ever see the out-of-sample data.

Choosing the split

A 70/30 or 80/20 split is the usual starting point: optimize on the earlier 70 to 80 percent, hold out the most recent 20 to 30 percent. Keep the out-of-sample period in the future relative to in-sample, never the reverse, so the test mimics how you would actually deploy. Make sure both windows are long enough to produce a meaningful trade count; a holdout with eight trades tells you nothing.

• 70/30 or 80/20, in-sample earlier, out-of-sample later.
• Both windows need enough trades to be meaningful on their own.
• Never tune on the holdout, not even once, not even a little.

Holding out data in TradingView

The cleanest way to hold out data is to gate trading by date with inputs, optimize with the window set to the in-sample period, then move the window to the out-of-sample period and run once without changing any parameter. Because the gate is an input, you never edit the strategy between the two runs, which removes a whole class of accidental leakage.

//@version=5
strategy("IS / OOS Split", overlay=true)

splitDate = input.time(timestamp("2024-01-01 00:00"), "In-sample ends")
phase     = input.string("in_sample", "Phase", options=["in_sample", "out_of_sample"])

inSample = time < splitDate
active = phase == "in_sample" ? inSample : not inSample

longSignal = ta.crossover(ta.sma(close, 20), ta.sma(close, 100))
if longSignal and active
    strategy.entry("Long", strategy.long)
if not active
    strategy.close_all()

Leakage, the silent killer

Leakage is any way information from the out-of-sample period sneaks into your tuning. The obvious form is optimizing on the holdout by accident. The subtle forms are worse: peeking at the holdout result, then going back to tweak the in-sample settings; running the holdout dozens of times and picking the run that happened to work; or using indicators that repaint and quietly use future bars. Each one turns your honest test back into an in-sample fit.