I saw an advertisement for a futures trading system the other day, and I'd thought I would discuss it here.
Now normally, I trade only futures, forex and commodity systems I create, since I know exactly what goes in
them, how they were tested, etc. That's not to say there aren't other vendors and systems out there that are good.
Some are very good, and worth checking into.
I usually scan ads to see what other system vendors in the
industry are up to, and alert my readers and subscribers of some of the tricks to be on the lookout for. I'm going to
share one of those sneaky tricks in a minute.
This ad caught my eye because it showed tremendous past performance,
such as the ability to make hundreds to thousands of dollars per day on 1 or 2 mini S&P contracts. Performance so good
that it makes you wonder why the vendor is selling it, but that's another story...
Enticed by the ad,
I clicked on through to show hypothetical performance data. In the vendor's mind, once I saw how good the system was at
picking trades - how well it picked tops and bottoms - I would undoubtedly be hooked. Unfortunately for them, it had the opposite
effect.
Why would hypothetical price charts that showed how well a system picked tops and bottoms work against
the vendor? Simple: incorrect backtesting. Please let me explain.
The system in question showed many trades
where the system bought at the absolute low of a bar, and sold at the absolute high of another bar. Since there is a bid/ask
spread in every futures market, it is normally impossible, or darn near impossible, to get filled on a buy order at the exact
lowest price (and the same for selling at the high). You might get filled once in a while at that price, but not consistently.
But when a vendor backtests incorrectly, the system can hypothetically get filled EVERY time at the low (for
buys) or at the highs (sells). Those are the results you'll see in the system performance report. In real life, you'd
only get filled on the losing trades, but only rarely on the winning trades. So, your results won't be nearly as good
as the hypothetical, vendor reported results. In fact, many times correcting for this trick turns a winning system into a
real life losing system - it is potentially that bad.
Confused? Well, it is a complicated topic, one most
people would not even recognize. But here is what to look for: If hypothetical results show the system can consistently buy
the absolute low and sell the absolute high (the extreme points) of a price bar, chances are the system will not work in real
time.
The lesson here: make sure you look at any performance history with a skeptical eye. Doing
so might save you a ton of money.