Skip to content
TradingTune
Backtesting Validation

How To Write Caveats For Optimized Results

Gives language for describing optimized settings without overstating what the historical test shows.

On this page

Key takeaways

  • A good caveat names the window you tested, the range you swept, and what the test cannot tell you about unseen data.
  • A caveat is stronger when it points at a range and the neighboring settings than when it points at a single number.
  • A setting that stands apart from its neighbors, or sits at the edge of the tested range, should be flagged as unverified.
  • Caveat language works when it names the range tested, the split used, and the specific thing left unverified, in plain words a reader can check.

Writing Caveats That Match What A Backtest Actually Shows

A caveat for an optimized setting works when it names the window you tested, the range you swept, and the one thing the test cannot tell you: how the setting behaves on data it has never seen. Get that right and the reader understands the result as a description of the past, not a promise about the future. This post gives you the actual language to use, three takeaways up front, and a worked example built around a parameter sweep. Read this if you optimize your own TradingView strategy and need to describe a result honestly, to a reader, to yourself six months later, or in a post like this one.

Three things to take away: caveats describe scope and blind spots, not confidence; a caveat is stronger when it points at a range than when it points at a single number; and the strongest caveat names the specific failure mode you checked for, not a generic disclaimer.

Setting Up The Sweep And The Walk-Forward Split

A backtest replays a strategy’s rules against historical price bars to see how it would have behaved, trade by trade. A parameter sweep runs that same backtest many times, changing one input, such as a lookback length or an entry threshold, across a defined range and step. On my own Smart Momentum Grid strategy, I sweep the trend lookback from 20 to 200 bars in steps of 10, and the entry threshold from 0.5 to 3.0 in steps of 0.25, so I can see how the strategy behaves as each input moves, not just at one setting someone picked by hand.

A walk-forward split breaks the historical data into sequential chunks: an in-sample window used to choose a setting, followed by an out-of-sample window used only to check it (Wikipedia, https://en.wikipedia.org/wiki/Walk_forward_optimization). I typically use three to six month windows, rolling forward one at a time, so each out-of-sample check happens on data the sweep never touched when it chose the setting. This matters because a setting picked on the same data it is judged against will look better than it deserves; that gap is the seed of overfitting, when a strategy fits the noise inside one dataset rather than a pattern that persists.

The reason to sweep a range instead of testing one guess is that a single number tells you almost nothing about how sensitive the strategy is to that input. A good sweep must not stop at picking the single best-looking cycle; it has to look at the whole curve of results across the range and ask whether nearby settings behave similarly. The general lesson on overfitting holds here: excessive parameter tweaking and unnecessary complexity are the main ways a sweep drifts from a real pattern toward memorized noise.

Trend Lookback Input (Pine v6). Defines a swept lookback input and plots its moving average alongside a simple crossover marker for visual comparison across settings.

Pine v6
//@version=6
indicator("Trend Lookback Example", overlay=true)
lookbackInput = input.int(50, "Trend Lookback", minval=20, maxval=200, step=10)
maValue = ta.sma(close, lookbackInput)
crossUp = ta.crossover(close, maValue)
plot(maValue, color=color.blue)
plotshape(crossUp, style=shape.triangleup, color=color.green)

TradingTune for Chrome

Stop tuning your strategy's inputs by hand.

Add to Chrome, it's free

Free tier, no API keys. A free account is required to run. Not affiliated with TradingView, Inc.

What Changes As You Move Across The Range

When I ran the trend-lookback sweep across the full 20 to 200 bar range, the shape of the equity curve changed in a way that surprised me: at the short end, the curve moved in sharp steps, jumping up and down with each cycle, while past roughly 120 bars it flattened into a smoother, slower climb. That shift told me the strategy was reacting to short-term noise at the short lookbacks and to a genuine trend pattern at the longer ones, which is exactly the kind of qualitative read a sweep is for.

The entry-threshold sweep showed a different pattern. At the low end, near 0.5, the strategy entered often and the cycle-to-cycle behavior varied widely, one walk-forward window showing steady activity, the next flat. Raising the threshold toward 2.0 reduced how often it entered and also narrowed that cycle-to-cycle variation, which is the qualitative signature of a setting relying less on a handful of unusual trades and more on a consistent mechanism across periods.

Across both sweeps, the useful signal was never a single result; it was whether neighboring settings told a consistent story. A parameter with a smooth, gradually changing curve of outcomes across its range is behaving like it tracks something real. A parameter where one narrow setting looks unusually good while its neighbors look ordinary or poor is a warning sign worth writing into the caveat, not hiding from it. That neighborhood behavior is what separates a robust finding from an artifact of overfitting.

Reading The Illustrative Sweep Table

The following table is hypothetical, the founder’s own Smart Momentum Grid, historical and illustrative only.

Parameter Range Step Lookback bars Qualitative note
Trend lookback 20 to 200 10 20 to 200 Choppy at short end, smooths past 120
Entry threshold 0.5 to 3.0 0.25 n/a Wide cycle spread below 1.0, narrows above 2.0
Exit lookback 10 to 60 5 10 to 60 Stable shape across most of the range
Volatility filter 5 to 30 5 5 to 30 Flattens curve above 20, little change below
Cycle window 3 to 6 months 1 month n/a Longer windows reduce cycle-to-cycle swing

Each row in this table describes a method choice, not an outcome. The trend-lookback row spells out the same range and step from the setup section, paired with the qualitative shape I described watching it: choppy at the short end, smoothing out past roughly 120 bars. The entry-threshold row records the cycle-to-cycle behavior I saw, without turning that behavior into a number, because the behavior itself, not any single cycle’s result, is the thing worth recording for a caveat.

The row I would probe further is the volatility filter, because a note that it flattens above 20 could mean the filter is doing something useful, or it could mean it is simply suppressing entries until there is nothing left to test. That is exactly the kind of row that deserves another sweep on a different symbol or window before anyone treats it as settled, and it is the kind of caveat language, flattens the curve, further validation needed, that keeps a table like this honest. When you read a table like this one in someone else’s post or your own notes, the qualitative notes are your actual evidence; the absence of a number is the point.

When An Optimized Setting Falls Apart Out Of Sample

The clearest failure I found came from a narrow entry-threshold setting near 1.75 that looked exceptional in one in-sample window and unremarkable in every window around it. It stood out from its neighbors instead of fitting a smooth pattern across the sweep, which in hindsight was the tell. When I checked that exact setting against the following out-of-sample window, it behaved like an ordinary, middling setting from the same sweep, not like the standout it appeared to be in-sample.

It fooled me because the in-sample window happened to contain a handful of trades that lined up well with that specific threshold, a coincidence rather than a persistent pattern. Wikipedia’s article on overfitting makes a related point: an analysis that corresponds too closely to one particular set of data may fail on additional data, so a fragile setting can look robust when it is really an artifact of that one dataset (Wikipedia, https://en.wikipedia.org/wiki/Overfitting).

A second, quieter failure mode showed up in the exit-lookback sweep: a setting near the edge of the tested range looked fine, but I had not swept far enough past it to know whether it was near a cliff. I had tested 10 to 60 bars, so a setting near 58 looked stable, but nothing told me whether 70 or 90 bars would have held up or collapsed. The lesson from both cases is the same, and it is the one I now build directly into my caveats: a setting that stands apart from its neighbors, or sits at the edge of what was tested, gets flagged as unverified rather than promoted as the winner of the sweep.

Turning This Into Your Own Validation Habit

My working rule is to never report a swept setting without also reporting the shape of the two or three settings on either side of it, because a caveat that shows the neighborhood is far more honest than one that shows a single point. This protects against overfitting because it forces the reader, and me, to judge a setting by its context in the sweep rather than by how good it looked in isolation.

If you want to apply this to your own sweep, start by writing down the exact range and step you plan to test before you look at any results, so you cannot quietly narrow the range afterward to flatter one setting. Then run the same walk-forward split across every setting in the range, not just the one that looks best, and note where the out-of-sample windows agree with the in-sample ones and where they diverge. Finally, write the caveat in terms of that agreement: which part of the range held up, which part did not, and which part you have not tested enough to say either way. The same discipline applies throughout: simplifying the model and limiting how much you tweak parameters against one dataset are the two levers that keep a sweep honest.

Why Caveat Language Matters For Every Optimized Result

The method this post walks through, sweeping a range instead of guessing one value, splitting the data so no setting is graded on the same window that picked it, and reading the shape of results across neighboring settings, exists for one reason: to give you enough information to write a caveat that is actually true. A caveat is not a formality bolted onto the end of a result; it is a direct report of what the sweep covered, what it left untested, and where the evidence was thin. When the method behind a result is sound, the caveat that describes it becomes short and specific instead of vague and defensive.

That is the risk this whole workflow protects against: presenting a setting that happened to fit one dataset as if it were a durable discovery, then watching it fail the moment real, unseen data arrives. The caveat language that works is the language that names the range tested, the split used, and the specific thing left unverified, in plain words a reader can check for themselves. Coming back to the question this post opened with, when you write caveats honestly, the reader learns not just what you found, but what you did not check and why that matters. If you take one habit from this post, take this one: before you write down a setting from your own sweep, look at its neighbors first, and let the shape of the whole range decide whether that setting deserves your confidence or your caveat.

Sources

TradingTune for Chrome

Stop tuning your strategy's inputs by hand.

TradingTune runs systematic parameter searches on your own TradingView strategy, right inside the chart page. You keep the strategy, you read the results.

Free tier, no API keys. A free account is required to run. Not affiliated with TradingView, Inc.

Share this post