Backtesting

For backtesting software, the historical data is usually in the form of daily opens, closes, highs and lows. That leaves a lot of data off the table. Two daily chart candles may look identical, opening at a low of 1 and closing at a high of 2. But what if the first chart candle saw price gyrate from 1 to 2 many times during the day, while the second one saw price shoot straight up to 2 and stay there for the day? Depending on intraday trading signals one uses, the two candlesticks might generate vastly different intraday trading activities, though they may appear alike to the backtesting software.

Suppose one trades a simple trading system that is either all-in a single stock, or all cash. Let us assume one buys when prices hit the one dollar level and sells when prices hit the two dollar level. Let us further assume one is currently all-in a certain stock. Let us call it Stock A. Suppose on the next trading day, Stock A hit 2 dollars, triggering a sell signal. During the same day, a certain Stock B hits 1 dollar, triggering a buy signal. What does your backtesting software do? If it shows a sale of Stock A and purchase of Stock B, that implies the former trading signal triggered and was taken, and sometime after, the second trading signal triggered, and was also taken. If the backtest shows a sale of Stock A but no transaction in Stock B, then it implies the buy signal could not be taken. Perhaps the software assumes the buy signal came first, and the sell signal much later, at which time Stock B may have already dropped and was no longer at the buy signal price.

The more info is considered by the backtest, the better. This may be the only way for backtesting to avoid making assumptions and really convey realistic historical trading results of a hypothetical trading system. One cannot assume, for instance, that one would get a $100-million order filled on a day the targeted stock only trades $10-million. Does the backtesting software account for these differences between order size and daily trading volumes? Does it adjust the trade price to be an accurate average over time or not? What about the impact one’s posting would have on short-term market behavior itself? As we all know, the sight of large postings often send traders scurrying in the short-term in the opposite direction.

The issue of execution price when one moves from all-in to partial, or zero investment, and back to all-in is best resolved by having a mammoth amount of historical price data. If the price levels and postings of every single second is known, then the backtesting software can more realistically depict what trades would have been possible, and at what prices, average fill or otherwise. Of course, the backtester may want to further specify the validity of intraday signals. For instance, if Stock B’s intraday buy signal comes 15 minutes before Stock A’s intraday sell signal, and thereafter dropped and stayed at non-buy signal levels, would the trading system still take the buy signal or not? It would depend on the detailed trading rules.

There will always be gray areas in backtesting. It’s best to be aware of their existence and try to account for them as best one can if one’s backtests are to be sufficiently reliable.

Advertisements
This entry was posted in Financial Markets, Trading and tagged , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s