I've been backtesting a few systems mostly on QQQ and SMH (mostly for their high liquidity and high volatility -- the same reasons everyone trades them) but I'm not sure how to interpret the results.
Over the historical period for both the buy and hold return is very poor (obviously) since most of it was during a pretty brutal bear market. However I'm not sure how to interpret system backtesting results for this period.
On one hand a system that can make money at all during that large a decline (especially a long only system!) must be decent. However, it partially benefits from lower exposure, only being in the market 25% of the time. This low exposure then becomes a slight underperformance during a rally like the one in 2003.
What are people's thoughts on a system like this and on backtesting during the bear market in general? I'm thinking maybe if you filter for bull/bear (using 200 day SMA?) then maybe you can decide when to buy/hold vs. trading a system that doesn't do as well in bull markets.
Thoughts?
SSB
Full Version: Backtesting pitfalls
If you can tolerate a big initial draw down, a system that stays in long or short 100% of the time and trades off long term charts will do wonderful.
QUOTE (SideShowBob @ Aug 30 2004, 07:30 PM)
I've been backtesting a few systems mostly on QQQ and SMH (mostly for their high liquidity and high volatility -- the same reasons everyone trades them) but I'm not sure how to interpret the results.
Over the historical period for both the buy and hold return is very poor (obviously) since most of it was during a pretty brutal bear market. However I'm not sure how to interpret system backtesting results for this period.
On one hand a system that can make money at all during that large a decline (especially a long only system!) must be decent. However, it partially benefits from lower exposure, only being in the market 25% of the time. This low exposure then becomes a slight underperformance during a rally like the one in 2003.
What are people's thoughts on a system like this and on backtesting during the bear market in general? I'm thinking maybe if you filter for bull/bear (using 200 day SMA?) then maybe you can decide when to buy/hold vs. trading a system that doesn't do as well in bull markets.
Thoughts?
SSB
Over the historical period for both the buy and hold return is very poor (obviously) since most of it was during a pretty brutal bear market. However I'm not sure how to interpret system backtesting results for this period.
On one hand a system that can make money at all during that large a decline (especially a long only system!) must be decent. However, it partially benefits from lower exposure, only being in the market 25% of the time. This low exposure then becomes a slight underperformance during a rally like the one in 2003.
What are people's thoughts on a system like this and on backtesting during the bear market in general? I'm thinking maybe if you filter for bull/bear (using 200 day SMA?) then maybe you can decide when to buy/hold vs. trading a system that doesn't do as well in bull markets.
Thoughts?
SSB
A trend following system will need a filter to keep you in trends. You can use a 14 period ADX to determine trendiness of market. Other methods include Vertical Horizontal filter, R2 when using linear regression, Aroon etc.
No amount of backtesting will get you prepared to trade the markets. It will give you a historical perspective and not much else. It will give you the confidence when you put money on the line.
Systems can be complex. My view is take a simple system. One that determines trend and then keeps you in it. you can then apply the system to a broader spectrum of stocks.
Here are three [edited: they are weekly, not monthly] charts of S&P 500, Naz comp, and Dow. All have an ADX trading system plotted. 99% percent of the time they are in a buy (green) or sell (red). There are two or three bars neutral (dark blue). The big thing would be to stay commited to the system.
S&P
COMP
Dow
S&P
COMP
Dow
Monthly charts are great...but I think most people are looking for swing trading. Using monthy charts to trade IMO is for folks with deeeeeeeeeeeep pockets. Something I dont have..
You could use weekly charts to determine direction and then take daily trades in the direction of the trend.
You could use weekly charts to determine direction and then take daily trades in the direction of the trend.
They are weekly charts. That was a typo and I can no longer edit. I rarely even look at monthly charts and have no idea why I wrote that.
Alot depends upon the time frame you are using for your back test period. For instance the QQQ began trading in 1990, does your back test go back that far or is it from another time. Buy and hold since then shows a 6 times your money return, 10,000 invested at 5.2 (in 1990) is worth roughly 64,000 today (8/30/04). Naturally you sat through alot in 2000-2002 if you could stand it.
Below is a very simple long only back test using Welles Wilder's Directional Movement with one caveat. If the +DM crosses the -DM up then you use a buy stop 1/8 over the highest value of the previous 12 days. This simple caveat reduced whipsaws quite a bit. You exit the trade when the +DM crosses the -DM to the downside on the opening the next day. And the results over the 14 year period shows a sharpe ratio of 1.38 which is pretty good.
I guess what I am really trying to say is the time period used means alot. Try adding filters to your buy and sell signals and see what turns up. You should look for a sharpe ratio greater 1.
Good Luck
Bill
Below is a very simple long only back test using Welles Wilder's Directional Movement with one caveat. If the +DM crosses the -DM up then you use a buy stop 1/8 over the highest value of the previous 12 days. This simple caveat reduced whipsaws quite a bit. You exit the trade when the +DM crosses the -DM to the downside on the opening the next day. And the results over the 14 year period shows a sharpe ratio of 1.38 which is pretty good.
I guess what I am really trying to say is the time period used means alot. Try adding filters to your buy and sell signals and see what turns up. You should look for a sharpe ratio greater 1.
Good Luck
Bill
QUOTE (mini-trader @ Aug 31 2004, 01:29 AM)
Here are three [edited: they are weekly, not monthly] charts of S&P 500, Naz comp, and Dow. All have an ADX trading system plotted. 99% percent of the time they are in a buy (green) or sell (red). There are two or three bars neutral (dark blue). The big thing would be to stay commited to the system.
an ADX system, is this the conventional ADX system, out there in the
public domain? or, is this a "proprietary" system?
It is convention except that I took away all the filters. All it does is plot a buy when +DI (14) is higher than -DI (14) and a sell when -DI is above +DI.
I guess ADX could just be used for your "level of commitment" or amount of funds dedicated to the trade.
I guess ADX could just be used for your "level of commitment" or amount of funds dedicated to the trade.
Did some testing of the performance of a simple SMA or EMA system on daily data. Best MA combination for S&P 500 i found was SMA(225) SMA(210) which did yield a non-leveraged annual return of 16%. I tested total 1 milion uniqe SMA/EMA combinations, ie (500x500)x4.
CODE
Signal Date Price MA1 MA2 (Points : Percent : Percent)
BUY SIGNAL : 19830817 165.29 0.00 152.73 (-0.00 : -0.00 : -0%)
SELL SIGNAL : 19840323 156.86 164.26 164.25 (-8.43 : -0.05 : -5%)
BUY SIGNAL : 19840328 159.89 164.16 164.16 (-3.03 : -0.02 : -2%)
SELL SIGNAL : 19840411 155.00 163.85 163.85 (-4.89 : -0.03 : -3%)
BUY SIGNAL : 19841214 162.69 159.70 159.74 (-7.69 : -0.05 : -5%)
SELL SIGNAL : 19880115 252.07 288.00 287.96 (89.38 : 0.55 : +55%)
BUY SIGNAL : 19880831 261.52 258.34 258.39 (-9.45 : -0.04 : -4%)
SELL SIGNAL : 19900613 364.90 343.73 343.70 (103.38 : 0.40 : +40%)
BUY SIGNAL : 19900830 318.71 343.80 343.81 (46.19 : 0.14 : +14%)
SELL SIGNAL : 19901001 314.94 341.36 341.36 (-3.77 : -0.01 : -1%)
BUY SIGNAL : 19901128 317.95 334.73 334.74 (-3.01 : -0.01 : -1%)
SELL SIGNAL : 19910111 315.23 333.70 333.69 (-2.72 : -0.01 : -1%)
BUY SIGNAL : 19910305 376.72 335.74 335.76 (-61.49 : -0.16 : -16%)
SELL SIGNAL : 19910312 370.03 336.48 336.47 (-6.69 : -0.02 : -2%)
BUY SIGNAL : 19910617 380.13 343.47 343.49 (-10.10 : -0.03 : -3%)
SELL SIGNAL : 19921110 418.62 413.03 413.02 (38.49 : 0.10 : +10%)
BUY SIGNAL : 19921204 432.06 413.96 413.97 (-13.44 : -0.03 : -3%)
SELL SIGNAL : 19940815 461.23 460.58 460.56 (29.17 : 0.07 : +7%)
BUY SIGNAL : 19950206 481.14 458.73 458.82 (-19.91 : -0.04 : -4%)
SELL SIGNAL : 20000516 1466.04 1387.77 1387.68 (984.90 : 2.05 : +205%)
BUY SIGNAL : 20000523 1373.86 1388.83 1388.84 (92.18 : 0.07 : +7%)
SELL SIGNAL : 20001108 1409.28 1441.58 1441.51 (35.42 : 0.03 : +3%)
BUY SIGNAL : 20001120 1342.62 1438.61 1438.78 (66.66 : 0.05 : +5%)
SELL SIGNAL : 20010124 1364.30 1424.14 1423.61 (21.68 : 0.02 : +2%)
BUY SIGNAL : 20020205 1090.02 1164.13 1164.28 (274.28 : 0.25 : +25%)
SELL SIGNAL : 20020226 1109.38 1160.16 1159.88 (19.36 : 0.02 : +2%)
BUY SIGNAL : 20020726 852.84 1084.84 1085.36 (256.54 : 0.30 : +30%)
SELL SIGNAL : 20020822 962.70 1071.38 1071.25 (109.86 : 0.13 : +13%)
BUY SIGNAL : 20030530 963.59 885.41 885.66 (-0.89 : -0.00 : -0%)
SELL SIGNAL : 20030619 994.70 892.28 892.14 (31.11 : 0.03 : +3%)
BUY SIGNAL : 20030721 978.80 899.77 899.78 ( 15.90 : 0.02 : +2%)
Time: 5517 trading days
Points: +2058.99
Trades: 31
Return: +372%
BUY SIGNAL : 19830817 165.29 0.00 152.73 (-0.00 : -0.00 : -0%)
SELL SIGNAL : 19840323 156.86 164.26 164.25 (-8.43 : -0.05 : -5%)
BUY SIGNAL : 19840328 159.89 164.16 164.16 (-3.03 : -0.02 : -2%)
SELL SIGNAL : 19840411 155.00 163.85 163.85 (-4.89 : -0.03 : -3%)
BUY SIGNAL : 19841214 162.69 159.70 159.74 (-7.69 : -0.05 : -5%)
SELL SIGNAL : 19880115 252.07 288.00 287.96 (89.38 : 0.55 : +55%)
BUY SIGNAL : 19880831 261.52 258.34 258.39 (-9.45 : -0.04 : -4%)
SELL SIGNAL : 19900613 364.90 343.73 343.70 (103.38 : 0.40 : +40%)
BUY SIGNAL : 19900830 318.71 343.80 343.81 (46.19 : 0.14 : +14%)
SELL SIGNAL : 19901001 314.94 341.36 341.36 (-3.77 : -0.01 : -1%)
BUY SIGNAL : 19901128 317.95 334.73 334.74 (-3.01 : -0.01 : -1%)
SELL SIGNAL : 19910111 315.23 333.70 333.69 (-2.72 : -0.01 : -1%)
BUY SIGNAL : 19910305 376.72 335.74 335.76 (-61.49 : -0.16 : -16%)
SELL SIGNAL : 19910312 370.03 336.48 336.47 (-6.69 : -0.02 : -2%)
BUY SIGNAL : 19910617 380.13 343.47 343.49 (-10.10 : -0.03 : -3%)
SELL SIGNAL : 19921110 418.62 413.03 413.02 (38.49 : 0.10 : +10%)
BUY SIGNAL : 19921204 432.06 413.96 413.97 (-13.44 : -0.03 : -3%)
SELL SIGNAL : 19940815 461.23 460.58 460.56 (29.17 : 0.07 : +7%)
BUY SIGNAL : 19950206 481.14 458.73 458.82 (-19.91 : -0.04 : -4%)
SELL SIGNAL : 20000516 1466.04 1387.77 1387.68 (984.90 : 2.05 : +205%)
BUY SIGNAL : 20000523 1373.86 1388.83 1388.84 (92.18 : 0.07 : +7%)
SELL SIGNAL : 20001108 1409.28 1441.58 1441.51 (35.42 : 0.03 : +3%)
BUY SIGNAL : 20001120 1342.62 1438.61 1438.78 (66.66 : 0.05 : +5%)
SELL SIGNAL : 20010124 1364.30 1424.14 1423.61 (21.68 : 0.02 : +2%)
BUY SIGNAL : 20020205 1090.02 1164.13 1164.28 (274.28 : 0.25 : +25%)
SELL SIGNAL : 20020226 1109.38 1160.16 1159.88 (19.36 : 0.02 : +2%)
BUY SIGNAL : 20020726 852.84 1084.84 1085.36 (256.54 : 0.30 : +30%)
SELL SIGNAL : 20020822 962.70 1071.38 1071.25 (109.86 : 0.13 : +13%)
BUY SIGNAL : 20030530 963.59 885.41 885.66 (-0.89 : -0.00 : -0%)
SELL SIGNAL : 20030619 994.70 892.28 892.14 (31.11 : 0.03 : +3%)
BUY SIGNAL : 20030721 978.80 899.77 899.78 ( 15.90 : 0.02 : +2%)
Time: 5517 trading days
Points: +2058.99
Trades: 31
Return: +372%
calculation doesn't seem to be right
SELL SIGNAL : 20000516 1466.04 1387.77 1387.68 (984.90 : 2.05 : +205%)
SELL SIGNAL : 20000516 1466.04 1387.77 1387.68 (984.90 : 2.05 : +205%)
QUOTE (vitaminm @ Sep 1 2004, 11:07 PM)
calculation doesn't seem to be right
SELL SIGNAL : 20000516 1466.04 1387.77 1387.68 (984.90 : 2.05 : +205%)
SELL SIGNAL : 20000516 1466.04 1387.77 1387.68 (984.90 : 2.05 : +205%)
Enter long position: 481.14
Exit long position: 1466.04
(1466.04 - 481.14) / 481.14 = 2.047 = 205% increase.
Seems correct to me? Do you get another result for this trade vitamin?
you are right.
show trades for this year.
show trades for this year.
Lurk,
I wonder if you do the same curve fit from the same start date but end it in 1996. I doubt you would get the same values for the moving averages. Do the values you get still give a decent profit in the range from 1996 to present? If not it's just a curve fit.....if they do work then it's a viable system, at least it should be....
SSB
I wonder if you do the same curve fit from the same start date but end it in 1996. I doubt you would get the same values for the moving averages. Do the values you get still give a decent profit in the range from 1996 to present? If not it's just a curve fit.....if they do work then it's a viable system, at least it should be....
SSB
Yes right conclusion, I dont think it's a viable or usefull system or anything like that, I belive it's just randomness that this particular MA pair comes out as the one yielding the best result. More of a experiment to see which SMA/ EMA pairs that gives the best profit yield for SP.
A good viable system should perform well on whatever you throw at it, be it a index, stock, commodity, currency, etc.
Here's top ranking for each category.
First colum is points profit and last one is number of trades.
A good viable system should perform well on whatever you throw at it, be it a index, stock, commodity, currency, etc.
Here's top ranking for each category.
First colum is points profit and last one is number of trades.
CODE
SMA(x) SMA(x)
1949.26 SMA(6) SMA(8) 842
1973.98 SMA(223) SMA(216) 47
1980.33 SMA(225) SMA(211) 31
1990.13 SMA(5) SMA(11) 586
1990.48 SMA(7) SMA(8) 1046
2009.42 SMA(227) SMA(203) 25
2009.59 SMA(225) SMA(209) 31
2027.73 SMA(224) SMA(209) 31
2045.69 SMA(224) SMA(210) 31
2060.73 SMA(225) SMA(210) 31
SMA(x) EMA(x)
1717.66 SMA(253) EMA(194) 14
1718.29 SMA(254) EMA(197) 14
1720.48 SMA(256) EMA(200) 12
1720.55 SMA(257) EMA(202) 12
1721.27 SMA(256) EMA(201) 12
1724.88 SMA(257) EMA(203) 12
1730.63 SMA(254) EMA(196) 12
1731.26 SMA(255) EMA(198) 12
1732.39 SMA(255) EMA(199) 12
2491.55 SMA(3) EMA(4) 1476
EMA(x) EMA(x)
1473.16 EMA(204) EMA(66) 15
1473.74 EMA(222) EMA(14) 31
1474.52 EMA(198) EMA(20) 24
1475.68 EMA(271) EMA(1) 99
1475.68 EMA(272) EMA(1) 99
1478.81 EMA(202) EMA(15) 29
1480.76 EMA(228) EMA(55) 15
1483.69 EMA(199) EMA(16) 27
1488.93 EMA(209) EMA(18) 25
1488.93 EMA(210) EMA(18) 25
EMA(x) SMA(x)
1738.45 EMA(84) SMA(50) 58
1752.52 EMA(10) SMA(12) 618
1771.39 EMA(9) SMA(12) 582
1782.40 EMA(9) SMA(10) 744
1788.18 EMA(8) SMA(10) 688
1789.53 EMA(12) SMA(13) 580
1804.10 EMA(63) SMA(49) 115
1816.78 EMA(52) SMA(45) 149
1881.40 EMA(10) SMA(11) 688
1938.22 EMA(9) SMA(11) 652
1949.26 SMA(6) SMA(8) 842
1973.98 SMA(223) SMA(216) 47
1980.33 SMA(225) SMA(211) 31
1990.13 SMA(5) SMA(11) 586
1990.48 SMA(7) SMA(8) 1046
2009.42 SMA(227) SMA(203) 25
2009.59 SMA(225) SMA(209) 31
2027.73 SMA(224) SMA(209) 31
2045.69 SMA(224) SMA(210) 31
2060.73 SMA(225) SMA(210) 31
SMA(x) EMA(x)
1717.66 SMA(253) EMA(194) 14
1718.29 SMA(254) EMA(197) 14
1720.48 SMA(256) EMA(200) 12
1720.55 SMA(257) EMA(202) 12
1721.27 SMA(256) EMA(201) 12
1724.88 SMA(257) EMA(203) 12
1730.63 SMA(254) EMA(196) 12
1731.26 SMA(255) EMA(198) 12
1732.39 SMA(255) EMA(199) 12
2491.55 SMA(3) EMA(4) 1476
EMA(x) EMA(x)
1473.16 EMA(204) EMA(66) 15
1473.74 EMA(222) EMA(14) 31
1474.52 EMA(198) EMA(20) 24
1475.68 EMA(271) EMA(1) 99
1475.68 EMA(272) EMA(1) 99
1478.81 EMA(202) EMA(15) 29
1480.76 EMA(228) EMA(55) 15
1483.69 EMA(199) EMA(16) 27
1488.93 EMA(209) EMA(18) 25
1488.93 EMA(210) EMA(18) 25
EMA(x) SMA(x)
1738.45 EMA(84) SMA(50) 58
1752.52 EMA(10) SMA(12) 618
1771.39 EMA(9) SMA(12) 582
1782.40 EMA(9) SMA(10) 744
1788.18 EMA(8) SMA(10) 688
1789.53 EMA(12) SMA(13) 580
1804.10 EMA(63) SMA(49) 115
1816.78 EMA(52) SMA(45) 149
1881.40 EMA(10) SMA(11) 688
1938.22 EMA(9) SMA(11) 652
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.