Backtesting pitfalls
#1
Posted 30 August 2004 - 07:30 PM
#2
Posted 30 August 2004 - 07:38 PM
"Bottoms come fast...Tops take forever"
#3
Posted 30 August 2004 - 07:42 PM
I've been backtesting a few systems mostly on QQQ and SMH (mostly for their high liquidity and high volatility -- the same reasons everyone trades them) but I'm not sure how to interpret the results.
Over the historical period for both the buy and hold return is very poor (obviously) since most of it was during a pretty brutal bear market. However I'm not sure how to interpret system backtesting results for this period.
On one hand a system that can make money at all during that large a decline (especially a long only system!) must be decent. However, it partially benefits from lower exposure, only being in the market 25% of the time. This low exposure then becomes a slight underperformance during a rally like the one in 2003.
What are people's thoughts on a system like this and on backtesting during the bear market in general? I'm thinking maybe if you filter for bull/bear (using 200 day SMA?) then maybe you can decide when to buy/hold vs. trading a system that doesn't do as well in bull markets.
Thoughts?
SSB
A trend following system will need a filter to keep you in trends. You can use a 14 period ADX to determine trendiness of market. Other methods include Vertical Horizontal filter, R2 when using linear regression, Aroon etc.
No amount of backtesting will get you prepared to trade the markets. It will give you a historical perspective and not much else. It will give you the confidence when you put money on the line.
Systems can be complex. My view is take a simple system. One that determines trend and then keeps you in it. you can then apply the system to a broader spectrum of stocks.
#4
Posted 30 August 2004 - 08:29 PM
S&P
COMP
Dow
"Bottoms come fast...Tops take forever"
#5
Posted 31 August 2004 - 07:11 AM
#6
Posted 31 August 2004 - 07:55 AM
"Bottoms come fast...Tops take forever"
#7
Posted 31 August 2004 - 08:09 AM
Below is a very simple long only back test using Welles Wilder's Directional Movement with one caveat. If the +DM crosses the -DM up then you use a buy stop 1/8 over the highest value of the previous 12 days. This simple caveat reduced whipsaws quite a bit. You exit the trade when the +DM crosses the -DM to the downside on the opening the next day. And the results over the 14 year period shows a sharpe ratio of 1.38 which is pretty good.
I guess what I am really trying to say is the time period used means alot. Try adding filters to your buy and sell signals and see what turns up. You should look for a sharpe ratio greater 1.
Good Luck
Bill
#8
Posted 31 August 2004 - 09:04 AM
Here are three [edited: they are weekly, not monthly] charts of S&P 500, Naz comp, and Dow. All have an ADX trading system plotted. 99% percent of the time they are in a buy (green) or sell (red). There are two or three bars neutral (dark blue). The big thing would be to stay commited to the system.
an ADX system, is this the conventional ADX system, out there in the
public domain? or, is this a "proprietary" system?
#9
Posted 31 August 2004 - 09:29 AM
Edited by mini-trader, 31 August 2004 - 09:34 AM.
"Bottoms come fast...Tops take forever"
#10
Posted 01 September 2004 - 01:16 PM
Signal Date Price MA1 MA2 (Points : Percent : Percent) BUY SIGNAL : 19830817 165.29 0.00 152.73 (-0.00 : -0.00 : -0%) SELL SIGNAL : 19840323 156.86 164.26 164.25 (-8.43 : -0.05 : -5%) BUY SIGNAL : 19840328 159.89 164.16 164.16 (-3.03 : -0.02 : -2%) SELL SIGNAL : 19840411 155.00 163.85 163.85 (-4.89 : -0.03 : -3%) BUY SIGNAL : 19841214 162.69 159.70 159.74 (-7.69 : -0.05 : -5%) SELL SIGNAL : 19880115 252.07 288.00 287.96 (89.38 : 0.55 : +55%) BUY SIGNAL : 19880831 261.52 258.34 258.39 (-9.45 : -0.04 : -4%) SELL SIGNAL : 19900613 364.90 343.73 343.70 (103.38 : 0.40 : +40%) BUY SIGNAL : 19900830 318.71 343.80 343.81 (46.19 : 0.14 : +14%) SELL SIGNAL : 19901001 314.94 341.36 341.36 (-3.77 : -0.01 : -1%) BUY SIGNAL : 19901128 317.95 334.73 334.74 (-3.01 : -0.01 : -1%) SELL SIGNAL : 19910111 315.23 333.70 333.69 (-2.72 : -0.01 : -1%) BUY SIGNAL : 19910305 376.72 335.74 335.76 (-61.49 : -0.16 : -16%) SELL SIGNAL : 19910312 370.03 336.48 336.47 (-6.69 : -0.02 : -2%) BUY SIGNAL : 19910617 380.13 343.47 343.49 (-10.10 : -0.03 : -3%) SELL SIGNAL : 19921110 418.62 413.03 413.02 (38.49 : 0.10 : +10%) BUY SIGNAL : 19921204 432.06 413.96 413.97 (-13.44 : -0.03 : -3%) SELL SIGNAL : 19940815 461.23 460.58 460.56 (29.17 : 0.07 : +7%) BUY SIGNAL : 19950206 481.14 458.73 458.82 (-19.91 : -0.04 : -4%) SELL SIGNAL : 20000516 1466.04 1387.77 1387.68 (984.90 : 2.05 : +205%) BUY SIGNAL : 20000523 1373.86 1388.83 1388.84 (92.18 : 0.07 : +7%) SELL SIGNAL : 20001108 1409.28 1441.58 1441.51 (35.42 : 0.03 : +3%) BUY SIGNAL : 20001120 1342.62 1438.61 1438.78 (66.66 : 0.05 : +5%) SELL SIGNAL : 20010124 1364.30 1424.14 1423.61 (21.68 : 0.02 : +2%) BUY SIGNAL : 20020205 1090.02 1164.13 1164.28 (274.28 : 0.25 : +25%) SELL SIGNAL : 20020226 1109.38 1160.16 1159.88 (19.36 : 0.02 : +2%) BUY SIGNAL : 20020726 852.84 1084.84 1085.36 (256.54 : 0.30 : +30%) SELL SIGNAL : 20020822 962.70 1071.38 1071.25 (109.86 : 0.13 : +13%) BUY SIGNAL : 20030530 963.59 885.41 885.66 (-0.89 : -0.00 : -0%) SELL SIGNAL : 20030619 994.70 892.28 892.14 (31.11 : 0.03 : +3%) BUY SIGNAL : 20030721 978.80 899.77 899.78 ( 15.90 : 0.02 : +2%) Time: 5517 trading days Points: +2058.99 Trades: 31 Return: +372%