The Ultimate Guide To Buying An Algo Trading System
Buying an Algorithmic Trading System
"Ugh. I’ve done it again. I bought a 'can’t miss' trading system I saw on the Internet. What looked so good in the advertisement turned out to look just as bad when I traded it with real money. This seems to happen to me EVERY time I buy a system. Why?"
Sound familiar? It certainly should, if you are in the habit of buying trading systems. That is not to say there are not good ones out there – there certainly are. But by and large, purchasing trading systems is an exercise fraught with peril.
So, here is what you need to know BEFORE you go and buy an algo trading system.
First, let’s start with some terminology.
What Is A Trading System?
A trading system is simply a set of rules that tell you when to buy and sell. They can be simple (buy a 20 day breakout) or they can be complicated (thousands of lines of code).
Depending on the type of trading system you purchase, all the rules may be disclosed, some of the rules may be disclosed, or none of the rules may be revealed. More on that in a minute.
Finally, the system may be complete (all rules defined) or it may be a partial system (where you have to supply your own rules for exiting, for example).
Depending on the type of trading system you purchase, all the rules may be disclosed, some of the rules may be disclosed, or none of the rules may be revealed. More on that in a minute.
Finally, the system may be complete (all rules defined) or it may be a partial system (where you have to supply your own rules for exiting, for example).
What Is An Algo Trading System?
An algo trading system, also known as an algorithmic trading system, is a full set of rules, typically already coded, that provide all buy and sell decisions. An algo trading system is typically 100% rules based – no discretion involved.
What Are the Different Types Of Trading Systems?
First, there are “black box” systems, where you do not get to see the actual rules. You do get the signals, but not the logic behind them.
Second, there are “white box” systems, where all the rules and logic is fully disclosed.
Finally, there are “gray box” systems where most, but not all, of the logic is disclosed to you.
Vendors will typically sell one of three types of algo trading systems.
In my experience, “black box” strategies are REALLY hard to trade, since you have no idea what signals are based on. It is hard to put confidence in a system that could be giving you random buy and sell signals. You’ll realize how tough black box systems are to trade during a drawdown.
Second, there are “white box” systems, where all the rules and logic is fully disclosed.
Finally, there are “gray box” systems where most, but not all, of the logic is disclosed to you.
Vendors will typically sell one of three types of algo trading systems.
In my experience, “black box” strategies are REALLY hard to trade, since you have no idea what signals are based on. It is hard to put confidence in a system that could be giving you random buy and sell signals. You’ll realize how tough black box systems are to trade during a drawdown.
Is Buying A Trading System Better Than Other Approaches To Trading?
It can be, although for most people that it probably is not. Most trading systems for sale are based on unrealistic backtests, or are curve fitted. Be VERY careful when buying an algo trading system.
Dangers in Buying Algorithmic Trading Systems
When you buy a trading system based on an advertisement you see in an e-mail, magazine, or website, you typically will see some summary statistics, and a smooth looking equity curve. It probably will look too good to be true (your first warning sign!), but certainly it will look enticing, like the equity curve shown in Figure 1.
Although in the back of your mind you will wonder why a vendor would sell the “Holy Grail” so cheaply, you probably won’t dwell on his motivations for selling it (likely, he needs the money to make up for his trading losses!). Surely, you’ll reason, such great performance is possible, and you’d be happy with even 25% or 50% of the profits shown. At that point, the hook is in your mouth, and the vendor only needs to jerk the line and set the hook to bring you on his boat.
Although in the back of your mind you will wonder why a vendor would sell the “Holy Grail” so cheaply, you probably won’t dwell on his motivations for selling it (likely, he needs the money to make up for his trading losses!). Surely, you’ll reason, such great performance is possible, and you’d be happy with even 25% or 50% of the profits shown. At that point, the hook is in your mouth, and the vendor only needs to jerk the line and set the hook to bring you on his boat.
Can Future Trading Results Be Guaranteed Or Predicted?
If you've been trading futures, commodities or forex for any amount of time, chances are you've heard the phrase "past performance is not necessarily indicative of (or no guarantee of) future results." It is a US government mandated statement that is required for groups involved in certain trading activities. And as you'll see, it is required for very good reasons, and to answer the headline question - no, past performance does NOT predict future results.
This warning statement/disclaimer is mandatory in many situations because most people look at a historical performance record or equity curve, and extrapolate the results into the future. For example, if an algorithmic trading system made 40% average per year for the last 5 years, many people will assume it will make 40% for the next five years - or at least for the next 1-2 years. That is the absolute WORST thing to assume, for two reasons. First, the future is never a repeat of the past, and second, the historical record itself might be flawed. Both of these problems are discussed below.
Even though it is obvious that no one can predict the future, people always do. This happens in all walks of life, and shows up in such diverse activities as weather forecasting, fortune telling, sports championship predictions and stock market calls. Although it is fun to predict the future, no one really can with any accuracy (although many will claim they have that "gift" - if they really have this ability, how come they are not multi-billionaires?).
Nostradamus is a great case in point. If he was so good at predicting the future, why wasn't he the richest man alive?
So, a typical way to predict the future is to use the past as a guide. For example, the Tampa Bay Buccaneers won the Super Bowl recently, so based on past performance, they should have won it all the following year. Of course, they didn't. Or, maybe in late 2007 you thought that since the "buy and hold" method for stocks worked from 1910-2007, it should work in 2008 and beyond. See the problem? Knowing the past doesn't necessarily help us with the future. Many times it blinds us to other possibilities, which many people found out during the 2008 financial crisis. And that's one reason for the government warning.
A second reason for the "past performance is no guarantee" disclaimer is that, especially in the investment world, the performance curves and figures that are shown are many times hypothetical (meaning, no one actually traded that way with real money, and possibly no one even COULD have traded that way, even if they wanted to), or developed with the benefit of hindsight (improper "backtesting" is a good example). Therefore, the trading system performance you are looking at might have absolutely ZERO relevance going forward.
So, how can you protect yourself?
First, if you see someone making market predictions, realize that there is a good chance they will be wrong. No matter who the guru is, take any prediction with a grain of salt. No one truly knows the future.
Second, trading system track records are nice to look at, but never assume that the past performance shown will be repeated. It rarely is. However, a track record, especially if it is audited by an independent third party, may help give you confidence that the people running the investment know what they are doing. That is certainly better than putting your money with someone who has no track record. But again, it is no guarantee of performance going forward.
Third, for any track record you see, ask if it was produced in a real money account. If it wasn't, it still might be a viable trading strategy, but treat it with healthy skepticism. It might be a pie-in-the-sky trading method that is impossible to achieve in real life. Or, if it was produced with hindsight, it is almost a guaranteed loser going forward. There are good algo trading systems, and there are bad algo trading systems. Many times, it is hard to know which is which.
Finally, make sure you perform due diligence before buying any system. It is your money, and you worked extremely hard to accumulate it. Don't throw it away on a snap decision. Take the time and the effort to research and investigate every investment opportunity in detail. If you rush and fall for the best sales pitch, soon enough you'll learn why there is the required warning "past performance is no guarantee of future results."
Let’s say your initial look at a trading system is favorable. How do you know the trading system vendor is not pulling some tricks on you? Here are a few of the more common tricks deployed by many unscrupulous vendors.
This warning statement/disclaimer is mandatory in many situations because most people look at a historical performance record or equity curve, and extrapolate the results into the future. For example, if an algorithmic trading system made 40% average per year for the last 5 years, many people will assume it will make 40% for the next five years - or at least for the next 1-2 years. That is the absolute WORST thing to assume, for two reasons. First, the future is never a repeat of the past, and second, the historical record itself might be flawed. Both of these problems are discussed below.
Even though it is obvious that no one can predict the future, people always do. This happens in all walks of life, and shows up in such diverse activities as weather forecasting, fortune telling, sports championship predictions and stock market calls. Although it is fun to predict the future, no one really can with any accuracy (although many will claim they have that "gift" - if they really have this ability, how come they are not multi-billionaires?).
Nostradamus is a great case in point. If he was so good at predicting the future, why wasn't he the richest man alive?
So, a typical way to predict the future is to use the past as a guide. For example, the Tampa Bay Buccaneers won the Super Bowl recently, so based on past performance, they should have won it all the following year. Of course, they didn't. Or, maybe in late 2007 you thought that since the "buy and hold" method for stocks worked from 1910-2007, it should work in 2008 and beyond. See the problem? Knowing the past doesn't necessarily help us with the future. Many times it blinds us to other possibilities, which many people found out during the 2008 financial crisis. And that's one reason for the government warning.
A second reason for the "past performance is no guarantee" disclaimer is that, especially in the investment world, the performance curves and figures that are shown are many times hypothetical (meaning, no one actually traded that way with real money, and possibly no one even COULD have traded that way, even if they wanted to), or developed with the benefit of hindsight (improper "backtesting" is a good example). Therefore, the trading system performance you are looking at might have absolutely ZERO relevance going forward.
So, how can you protect yourself?
First, if you see someone making market predictions, realize that there is a good chance they will be wrong. No matter who the guru is, take any prediction with a grain of salt. No one truly knows the future.
Second, trading system track records are nice to look at, but never assume that the past performance shown will be repeated. It rarely is. However, a track record, especially if it is audited by an independent third party, may help give you confidence that the people running the investment know what they are doing. That is certainly better than putting your money with someone who has no track record. But again, it is no guarantee of performance going forward.
Third, for any track record you see, ask if it was produced in a real money account. If it wasn't, it still might be a viable trading strategy, but treat it with healthy skepticism. It might be a pie-in-the-sky trading method that is impossible to achieve in real life. Or, if it was produced with hindsight, it is almost a guaranteed loser going forward. There are good algo trading systems, and there are bad algo trading systems. Many times, it is hard to know which is which.
Finally, make sure you perform due diligence before buying any system. It is your money, and you worked extremely hard to accumulate it. Don't throw it away on a snap decision. Take the time and the effort to research and investigate every investment opportunity in detail. If you rush and fall for the best sales pitch, soon enough you'll learn why there is the required warning "past performance is no guarantee of future results."
Let’s say your initial look at a trading system is favorable. How do you know the trading system vendor is not pulling some tricks on you? Here are a few of the more common tricks deployed by many unscrupulous vendors.
Common Algo Trading System Vendor Tricks
Over-optimization
Take some market data, and subject it to enough tests, and eventually a great equity curve will result. This occurs all the time, through the abuse of a testing process known as optimization. If a vendor puts enough rules in the strategy, with enough parameters to create millions of different trading systems, by random luck a few systems will have extraordinary performance. In effect, the data has been tortured long and hard enough to give up its secrets. But those secrets almost never apply to future data.
The tough part about optimization is that many times you cannot tell if optimization was utilized properly during development. The key for me is that if it looks too good to be true, it probably was over-optimized.
Incorrect Development Process
Most trading system vendors have no clue how to properly develop a trading system. That is another peril of algo trading systems. That explains why they are primarily vendors, not traders! Developing a system the correct way is tough work, and many times extremely frustrating. For example, the trademarked Strategy Factory process I use is shown in Figure 2. It is long, tedious, and difficult. Most of the trading ideas I test turn out to be worthless. Although there are other legitimate ways to develop a system, this approach has worked time and time again for me.
As with over-optimization, you’ll almost never be able to know for sure if a trading system vendor developed the system with a flawed process. Because of this, I always assume the vendor did it incorrectly. That assumption can save you thousands.
Deceiving Backtest Software
Another common approach that “snake oil” vendors employ is to trick the backtest software into giving phony results. Every piece of software, including industry leaders such as Tradestation, Ninja Trader and Multicharts, has multiple ways to exploit the strategy backtest engine. Sneaky vendors know this, and take advantage whenever they can.
One popular exploit deals with limit orders. In the real money world, most limit orders are only filled when your limit price is exceeded. Most software, however, can be configured to provide limit fills as soon as the price is touched. This can make most scalping systems look terrific in backtest, when in real life they would fail miserably.
Here is an example, shown in Figure III. The equity curve on the left assumes that limit orders are filled as soon as they are touched (the incorrect way). The curve on the right shows trades that fill only when the limit order is penetrated. What a difference!
Although many times this exploit is hard to detect, one sure sign is to look at a chart of trades, as shown in Figure IV. If there are a number of buys at the low, or sells at the top, you can bet the limit order exploit was used and that results should be ignored.
Best Tips to Protect Yourself When Buying An Algo Trading System
Recognize Algo Trading System Vendor Tricks
Protect Yourself
Knowing three common ways trading system results are usually not what they seem, what can you do to protect yourself? Here are some recommended actions:
Be Skeptical - Realize the Holy Grail is Not for Sale
What kind of person sells a golden goose for a few dollars? Wouldn’t it be better to keep it for himself, and trade his way to financial riches from the proverbial tropical beach? Think about that before you buy a trading system. On the other hand, there are legitimate vendors who sell decent performing systems, and likely trade those themselves. But who wants to buy “decent” when you could instead buy the seemingly “Holy Grail” of systems? Good legitimate vendors with good systems get buried under a pile of shiny looking garbage. It is strange, but usually the better a trading system backtest looks, the worse it actually is!
Run in Real Time with No Money
Many times, an over-optimized system will fall apart with future data. It might take a week, or a month, but eventually it will fail. So, if you must buy a trading system, be patient before dedicating real money to it. Watch its performance for 3-6 months, and see what happens. You’ll definitely be glad you did.
Run Small Size, Check for Fill Matching
If you are worried about a system getting overly optimistic fills, which occurs many times with scalping systems, you can always try running a handful of trades with real money. If you then watch the real money performance and live strategy performance at the same time, you’ll frequently uncover the mystery behind the optimistic fills. Such a limited test is usually a small price to pay to reveal the truth.
Develop Your Own Systems
If you have the capability and desire, developing your own system is the way to go. It is the only way I do things now. Of course, you must utilize the correct process, otherwise you’ll make one of the hundreds of development mistakes that inexperienced traders make. But, do it correctly and you can create good trading strategy after good trading strategy.
Focus On The Downside
With almost every decision in life, people tend to focus on the positive, and minimize the negative. Think about buying a house - what entices you first? Usually, it is curb appeal, and if your first impression is favorable, you'll tend to minimize negatives, like maybe the amount of rehab the house needs on the inside.
It is the same thing with algorithmic trading systems or strategies. Most people tend to focus on the rate of return, or maybe even the winning percentage, and if that looks good, they'll downplay the negatives, like the drawdown. For those of you who do not know, "drawdown" is the amount of money lost from the highest equity point. It might be a temporary loss, but you'll have to endure it if you trade the system. Small drawdowns are best, but they are usually tied to low returns (think of interest on a bank account).
When looking at an automated trading system, your focus should be on downside risk. Drawdown is a great way to measure it.
So, what are some good ways to focus on the downside?
1. Make sure the trading history you see is actually representative of the system you are buting. There are a lot of knucklehead developers who will significantly change an algorithmic system midstream, yet claim it is the same system! So, when you look at an equity curve, you'll be seeing the combined results of 2 or more systems. That is ridiculous at best, misleading and unethical at worst.
2. Before you buy, find out what the historical maximum drawdown was. As a rule of thumb, multiply that by 1.5. Could you handle that amount of drawdown right off the bat? If not, then you shouldn't purchase it. Make sure to account for position size when you do this calculation.
3. If you have the ability, put all the trades in a Monte Carlo Simulation and see what shakes out. Monte Carlo simulation (available on my Calculator page) is a technique that introduces randomness into future trading results. Since the past is never repeated exactly, it is a good way to simulate possible future results. You will be able to see drawdowns in terms of probabilities. So, for example, you might find out that trading system X has a 50% chance of having a drawdown greater than 30% in the first year. It may have this severe of a drawdown, it may not, but the point is at least you'll have some better data to go off of.
4. BEFORE you purchase that algo trading system, determine your walk away point, and stick to it. Your thinking might be like this:
"I checked with the developer, and he has not changed his trading system since introduction. A check of his trades confirms that he is trading the same way he always has. His independently verified records show he had a 20% drawdown. Developer's hypothetical history shows a 25% drawdown, and Monte Carlo sim shows a 50% chance of a max drawdown of 30% in the first year. So, I'll take the worst case of them all, and multiply by 1.5. That gives me 1.5*30 = 45% drawdown I should be ready for. That is too much drawdown for me, so I either need to add capital to the account, or not trade this system. Looking at the results, if I double my initial capital, I still get a very good rate of return, and I can withstand a 22.5% drawdown. If that drawdown gets hit, I am out - NO QUESTIONS ASKED. Otherwise, I stick to the system!"
The major point is: don't be sucked in by the fancy curb appeal of net profits or rate of return. Dig deeper, and look at the potential risk, especially drawdown. It takes more effort, but in trading there is no easy way out. Those who try the easy way usually end up losing all their money.
Conclusion
Buying a trading system is more often than not a bad idea, especially if you only know the vendor from the equity curves in his ads. Most vendors are either failed traders, or wannabe traders. They make their money from vending, not trading. They employ many sneaky tricks!
After 30+ years in the retail trading world, I have concluded that building your own trading systems is the only way to go. While you have to watch out for many of the same tricks that vendors use to sell garbage, once you have a solid development process, comprising of sound practices, you’ll be a lot closer to being a successful trader than you would by relying on a hack trading system vendor. In other words, your trading success will be tied to your abilities, not the phantom success of a trading system vendor.
Knowing three common ways trading system results are usually not what they seem, what can you do to protect yourself? Here are some recommended actions:
Be Skeptical - Realize the Holy Grail is Not for Sale
What kind of person sells a golden goose for a few dollars? Wouldn’t it be better to keep it for himself, and trade his way to financial riches from the proverbial tropical beach? Think about that before you buy a trading system. On the other hand, there are legitimate vendors who sell decent performing systems, and likely trade those themselves. But who wants to buy “decent” when you could instead buy the seemingly “Holy Grail” of systems? Good legitimate vendors with good systems get buried under a pile of shiny looking garbage. It is strange, but usually the better a trading system backtest looks, the worse it actually is!
Run in Real Time with No Money
Many times, an over-optimized system will fall apart with future data. It might take a week, or a month, but eventually it will fail. So, if you must buy a trading system, be patient before dedicating real money to it. Watch its performance for 3-6 months, and see what happens. You’ll definitely be glad you did.
Run Small Size, Check for Fill Matching
If you are worried about a system getting overly optimistic fills, which occurs many times with scalping systems, you can always try running a handful of trades with real money. If you then watch the real money performance and live strategy performance at the same time, you’ll frequently uncover the mystery behind the optimistic fills. Such a limited test is usually a small price to pay to reveal the truth.
Develop Your Own Systems
If you have the capability and desire, developing your own system is the way to go. It is the only way I do things now. Of course, you must utilize the correct process, otherwise you’ll make one of the hundreds of development mistakes that inexperienced traders make. But, do it correctly and you can create good trading strategy after good trading strategy.
Focus On The Downside
With almost every decision in life, people tend to focus on the positive, and minimize the negative. Think about buying a house - what entices you first? Usually, it is curb appeal, and if your first impression is favorable, you'll tend to minimize negatives, like maybe the amount of rehab the house needs on the inside.
It is the same thing with algorithmic trading systems or strategies. Most people tend to focus on the rate of return, or maybe even the winning percentage, and if that looks good, they'll downplay the negatives, like the drawdown. For those of you who do not know, "drawdown" is the amount of money lost from the highest equity point. It might be a temporary loss, but you'll have to endure it if you trade the system. Small drawdowns are best, but they are usually tied to low returns (think of interest on a bank account).
When looking at an automated trading system, your focus should be on downside risk. Drawdown is a great way to measure it.
So, what are some good ways to focus on the downside?
1. Make sure the trading history you see is actually representative of the system you are buting. There are a lot of knucklehead developers who will significantly change an algorithmic system midstream, yet claim it is the same system! So, when you look at an equity curve, you'll be seeing the combined results of 2 or more systems. That is ridiculous at best, misleading and unethical at worst.
2. Before you buy, find out what the historical maximum drawdown was. As a rule of thumb, multiply that by 1.5. Could you handle that amount of drawdown right off the bat? If not, then you shouldn't purchase it. Make sure to account for position size when you do this calculation.
3. If you have the ability, put all the trades in a Monte Carlo Simulation and see what shakes out. Monte Carlo simulation (available on my Calculator page) is a technique that introduces randomness into future trading results. Since the past is never repeated exactly, it is a good way to simulate possible future results. You will be able to see drawdowns in terms of probabilities. So, for example, you might find out that trading system X has a 50% chance of having a drawdown greater than 30% in the first year. It may have this severe of a drawdown, it may not, but the point is at least you'll have some better data to go off of.
4. BEFORE you purchase that algo trading system, determine your walk away point, and stick to it. Your thinking might be like this:
"I checked with the developer, and he has not changed his trading system since introduction. A check of his trades confirms that he is trading the same way he always has. His independently verified records show he had a 20% drawdown. Developer's hypothetical history shows a 25% drawdown, and Monte Carlo sim shows a 50% chance of a max drawdown of 30% in the first year. So, I'll take the worst case of them all, and multiply by 1.5. That gives me 1.5*30 = 45% drawdown I should be ready for. That is too much drawdown for me, so I either need to add capital to the account, or not trade this system. Looking at the results, if I double my initial capital, I still get a very good rate of return, and I can withstand a 22.5% drawdown. If that drawdown gets hit, I am out - NO QUESTIONS ASKED. Otherwise, I stick to the system!"
The major point is: don't be sucked in by the fancy curb appeal of net profits or rate of return. Dig deeper, and look at the potential risk, especially drawdown. It takes more effort, but in trading there is no easy way out. Those who try the easy way usually end up losing all their money.
Conclusion
Buying a trading system is more often than not a bad idea, especially if you only know the vendor from the equity curves in his ads. Most vendors are either failed traders, or wannabe traders. They make their money from vending, not trading. They employ many sneaky tricks!
After 30+ years in the retail trading world, I have concluded that building your own trading systems is the only way to go. While you have to watch out for many of the same tricks that vendors use to sell garbage, once you have a solid development process, comprising of sound practices, you’ll be a lot closer to being a successful trader than you would by relying on a hack trading system vendor. In other words, your trading success will be tied to your abilities, not the phantom success of a trading system vendor.
HTML Comment Box is loading comments...
About The Author: Kevin Davey is an award winning private futures, forex and commodities trader. He has been trading for over 25 years.Three consecutive years, Kevin achieved over 100% annual returns in a real time, real money, year long trading contest, finishing in first or second place each of those years.
Kevin is the author of 5 highly acclaimed books, including "Building Algorithmic Trading Systems: A Trader's Journey From Data Mining to Monte Carlo Simulation to Live Trading" (Wiley 2014). Kevin provides a wealth of trading information at his website: https://www.kjtradingsystems.com
Copyright, Kevin Davey and KJ Trading Systems. All Rights Reserved. Reprint of above article is permitted, as long as the About The Author information is included.
Kevin is the author of 5 highly acclaimed books, including "Building Algorithmic Trading Systems: A Trader's Journey From Data Mining to Monte Carlo Simulation to Live Trading" (Wiley 2014). Kevin provides a wealth of trading information at his website: https://www.kjtradingsystems.com
Copyright, Kevin Davey and KJ Trading Systems. All Rights Reserved. Reprint of above article is permitted, as long as the About The Author information is included.