Jump to content



Photo

Indicator illusion


  • Please log in to reply
21 replies to this topic

#11 NAV

NAV

    Member

  • Traders-Talk User
  • 16,087 posts

Posted 22 July 2006 - 05:34 PM

NAV,

I think I'm still not totally with you. Let me try an example.

Let's say you have a series of 4 indicators that were developed strictly from backtesting, or curve fitting. You backtested them over the last 8 years, and over that period of time they were accurate in picking short-term bottoms 115 times, and on 25 occassions they were wrong. That would be an 82% win ratio spread out over 8 years with a "lot" of signals. Assume also that they pretty much consistently achieved that same 82% win ratio in each of the 8 years even though those 8 years included both bull and bear markets. This is still curve fitting, right, which if I am understanding you correctly you don't look favorably upon? But is it not also technical analysis, i.e., the recognition of a pattern that repeats and can be expected to continue repeating? Does the size of the sample have a bearing on whether you would consider it a legitimate noteworthy pattern versus just curve fitting?

Bullshort


Yes, what you are describing is curve fitting. Now unless one knows the distribution of the signals over the bull and bear market periods, it's hard to know what your odds of success are. To give you a simple example:

Lets say, i have a simple bottom picking system that uses RSI and has been backtested over 10 years. Let's say the signal is when the RSI goes below 30 and comes back above it, it's a buy signal. Here are 6 signals out of it. 10 days after the signal was generated, the gains from the signals were

Sig #1 : +4.0%
Sig #2 : -1.0%
Sig #3 : +2.0%
Sig #4 : +3.0%
Sig #5 : +1.0%
Sig #6 : -2.0%

Now, let's assume that signal Sig #1, Sig #2, Sig #3, Sig#4 were generated in a bull context and Sig #5 and Sig #6 was generated in a bear context.

Now looking at all the 6 signals (bull and bear context), it would give a 66% win/loss ratio.

Whereas considering Sig#1 - Sig#4 only (bull context only), would give a 75% win/loss ratio.

Whereas considering Sig#5 and Sig#6 only (bear context only), would give a 50% win/loss ratio i.e a toss of coin.

Now let's say, if we were to enter a protracted bear market and i get two more signals.

Sig #7 : -3.0%
Sig #8 : -6.0%

Now the bear context win/loss ratio becomes 25%, making it a worthless system.


So if the character of the market changes to completely different than from the backtested years, the results could be very different from what you have observed from your backtested years. That's why every mechanical system which works like a charm during a certain period, fails miserably during another period.

It's not to say that backtested systems are useless, but one needs to have statistics for the signals from both bull/bear market contexts and keep those statistics seperate. One should also adapt/modify their systems, when the context changes. That's the reason i am not in favor of pure mechanical systems.

If anyone has a mechanical system with 80% win/loss ratio in both bull and bear contexts, i would really love to see one, cuz i haven't seen one to date. I would strongly encourage them to post their signals in real-time so that we all can verify that.

I met a sales guy, a few years back, who showed me a mechanical system with 85% accuracy. When i tested the system with data from 73-75, it gave me a win/loss ratio of 35%. When i presented my results to the sales guy, his response was "Hey, we are not living in the 70s dude :lol:".

"It's not the knowing that is difficult, but the doing"

 

https://twitter.com/Trader_NAV

 

 


#12 rightside

rightside

    Member

  • Traders-Talk User
  • 237 posts

Posted 22 July 2006 - 09:21 PM

A couple observations, from my own experience with trading systems.

First of all. Price is king. Moving averages are above everything else in determining the trend. Price can keep erroding bumping into moving averages overhead, while oscillators will be getting overbought, oversold and what not.

Second, the hardest part for me is not entrering the trade, but where to exit ? Its pretty easy to generate good entry signals. But exit signals is an issue. You want to exit before the trend starts rolling over, at the same time you want to maximize the profit, and not to leave a big chunk on the table.

There are a lot of situations when a buy signal for example is followed by a strong rally, but then price falls apart so fast that by the time indicators turn the price is already below the entry point. And some other time the same entry signal is followed by a rally that just keeps going and going. And you really want to maximize the profit from winning trades, as every system gets whipsawed once in a while and has to take a loss.

Anyone good ideas on it ?


Position sizing! I currently have several programs that I run based on the "locks" being changed from time to time. I optimize and adapt my programs on a regular basis based on what is working over the last 3-6 months. I have fully automated trading systems, yet I prefer to enter myself.

I currently have a program that is performing at %100 percent recently (3 months) on the short side %60 on the long. The reason I mention this is not to brag but to tell you that one of the keys to this program is it takes profits quicker than any of my others. Spend %50 of the time on the sideline waiting to pounce!

Long + Short Long Only Short Only Buy & Hold
Net Profit $11,376.03 $4,772.02 $6,604.02 $-3,250.00
Profit per Bar $3.79 $2.38 $6.67 $-0.16

Number of Trades 17 8 9 1
Avg Profit/Loss $669.18 $596.50 $733.78 $-3,250.00
Avg Profit/Loss % 0.96% 0.86% 1.04% -4.59%
Avg Bars Held 176.35 251.00 110.00 20,524.00

Winning Trades 14 5 9 0
Winning % 82.35% 62.50% 100.00% 0.00%
Gross Profit $12,356.02 $5,752.01 $6,604.02 $0.00
Avg Profit $882.57 $1,150.40 $733.78 $0.00
Avg Profit % 1.26% 1.65% 1.04% 0.00%
Avg Bars Held 179.21 303.80 110.00 0.00
Max Consecutive 7 3 9 N/A

Losing Trades 3 3 0 1
Losing % 17.65% 37.50% 0.00% 100.00%
Gross Loss $-979.99 $-979.99 $0.00 $-3,250.00
Avg Loss $-326.66 $-326.66 $0.00 $-3,250.00
Avg Loss % -0.46% -0.46% 0.00% -4.59%
Avg Bars Held 163.00 163.00 0.00 20,524.00
Max Consecutive 1 2 0 N/A

Max Drawdown $-1,110.00 $-1,090.00 $-790.00 $-7,250.00
Max Drawdown Date 6/13/2006 6/15/2006 7/13/2006 7/21/2006

Profit Factor 12.61 5.87 INF 0.00
Recovery Factor 10.25 4.38 8.36 0.45
Payoff Ratio 2.77 3.62 INF 0.00
Ulcer Index 7.79 20.14 3.57 111.58
Wealth-Lab Error Term 288.96 344.18 288.65 1,082.26
Luck Coefficient 1.57 1.20 1.09 0.00
Pessimistic Rate of Return 6.00 2.11 0.00 0.00

#13 n83

n83

    Member

  • Traders-Talk User
  • 1,086 posts

Posted 22 July 2006 - 09:25 PM

price-and-volume's bottom picking indicator, piqued my long standing interest in presenting some "Trading system" development pitfalls, that i have learn't over the past 6 years. This is not a critique of price-and-volume's indicator nor is it directed towards him, as i respect anyone's work which is freely shared on this board. This is about "Trading Systems" development in general,


Can i get an indicator which catches all the swing turning points ?

Sure, welcome to the world of curve fitted systems. We have an army of snake oil salesman waiting to take you on a tour of "Bottom picking system", "Top picking system", "Top/Bottom picking hybrid system". Seriously there's nothing that a beaten down trader wants other than a magical indicator, which can catch the swing turns with accuracy. Especially when the indicator is nicely displayed with the price chart, catching all the turns, you have a gauranteed customer.

Now here's the problem. By the virtue of the underlying mathematics of most Momentum based indicators, it virtually impossible to have the indicator display the same behaviour under different market contexts. If one defines for simplicity sake that there are three types of markets BULL (rising EMA), BEAR (falling EMA) and SIDEWAYS (flat EMA), you bet the indicator would show different types of behaviour in different contexts. So having a holy grail indicator which would work in all three contexts would require tweaking or forcing or tuning the indicator to catch all the turns in apparently different market contexts. Well by playing with the different parameters in the indicator, it's possible to acheive that, but it's a backasswards way of developing a system. If one develops a system with the means i described above, it would be a nice mechanical system that works like a charm in a particular market context and fails miserably when the market characteristic changes. Now i constantly hear the phrase, "Indicators are useless. They change the locks on the market every few years". That's true of mechanical systems, because they are designed with a backassward approach. If mechanical systems really worked, i would be trading from Tahiti, with all those polynesian beauties around, showing how cool a guy i am, pressing a button on every signal beep and counting millions.

Here's an example of a forced indicator.Now it took me a total of 15-minutes to whip up these two indicators. I could the sell the first indicator as one which catches all the bottoms (major and minor ones) and the second one which catches only the major bottoms. Now how naive could someone be, who beleives that a indicator can know whether a bottom is a major or a minor one. Now do you think this indicator would work when tested over a period of 100 years. You bet not !!. Now if you ask a "Trading Systems" salesman, why it failed over a certain period, he would respond by saying that it was designed for the current market conditions. Now how do i know, when the market conditions change ?

There are even more clever curve fitted systems which looks like a cyclical indicator which catches all the swing turns, which are purely mathematical curve fitting algorithms, which operate on a data series. Let's not even get there. This discussion is about momentum indicators.

Posted Image

Posted Image


What are those magical settings you use on your indicator ?

The answer is, "It's irrelevant". Indicator settings are either fast or slow, depending on the style of the trader. Aggressive traders who try to play all the wrinkes on the chart keep their settings fast. Slow traders who care less about the minor wrinkes keep the settings slow. But there' nothing like a magical setting that catches all the turns. It's the obverver of the indicator who catches turns, while the indicator just reports what's happening in the market. So it's more important to understand how an indicator works rather trying to come up with a magical setting.



If one does not tweak the indicators to catch all the turns, how does one catch the turns or why use the indicator at all?

Here's the meat of "Trading Systems" development. An indicator whether tuned or not, can catch turns. It's only a question as to whether the turn was genuine or not. Even a well tuned indicator, under different market context can give a false signal. That's what a system developer realizes once he starts using the system to make real trades. So forcing the indicator to give what you want and what you would like to see is not the solution. Instead start with an indicator, tune it either fast or slow, depending on your style. Then the system will start churning out signals. Now as we all know, the signals spit out by the system is a random distribution of good and bad signals. Now as long as the system generates more than 40% good signals, it's still possible to make money with good money management. But heck, if a system has such a crappy probability like 40% sucess rate, why bother to develop one ? Instead flip a coin, which has 50% probability of success. So the goal of the system developer is to increase that success rate by filtering bad signals.

How does one filter bad signals ?

This is a very tough question to answer and in all honesty, it's beyond the scope of this thread. But i can throw in some filtering techniques.

1) Let's say you are a 30-min trader and take all the 30-min signals from your system, but would like to filter out the bad ones. This requires a multi-timeframe analysis. One needs to have another chart from a higher timeframe such as 120-min or 240-min. Let's assume that the 120-min flips into a sell.

Filter #1 : Since 120-min is now on a sell, only take the 30-min sell signals and ignore the 30-min buy signals.
Filter #2 : 120-min is not oversold. So continue to take all the continuation 30-min sell signals.
Filter #3 : 120-min is oversold. Do not take the 30-min continuation sell signals anymore, because the higher context is oversold and the trend here could turn on a dime. But if you have a position hold it with trailing stops at prior swing points. Do not take the 30-min buy signals as well, since 120-min is still on a sell.


Now the 120-min turns to a buy. Start taking all the 30-min buys with the above filters.

Not only will this keep one on the right side of the trend and let him/her ride the trend to it's exhaustion, but will also filter out bad signals. If one is a 120-min trader, the same rules apply, but the higher context would be daily in this case.

2) Second level of filtering can be done based on the EMA configuration. For instance one could ingore signals when a EMA is flat or take signals only in the direction of EMA slope i.e shorts only when a particular EMA is sloping down and longs only when the EMA is sloping up. Again it doesn't matter whether one uses 34 EMA or 21 EMA or 55 EMA - whatever works best with your system.

3) Another powerful filtering technique is based on historical backtesting. Go thru different market contexts such as the 80s bull, 70s bear , 1929 bear , 1999-2000 bubble etc and find out where your signals failed. They give some unique insights into how the indicator functions in those scenarios and one could codify those failures into objective rules.

4) Oversold is not a buy signal and overbought is not a sell signal. Divergences are not buy/sell signals. They are just warning signs of an impending turn. They could remain is that zone for a long time in trending markets.

5) Good buy/sell signals come out of oversold/overbought conditions. Let's say one uses a MACD crossover for his/her buy signals. Seriously, can a simple thing like MACD crossover make you money ?? Not unless, it's filtered. Unfiltered signals are the primary cause of frustration with momentum indicators and which is why most people throw in the towel and go for fancy stuff like Bradley, Astro, Gann etc. Let's say i am using a 30-min chart and taking MACD crossover signals. Out of 4 signals i have two good signals and two whipsaws. I could probably have done the same with a coin toss. However try this. Use another indicator which acts as a pure overbought/oversold indicator. I personally like to use the Full Stoch as overbought/oversold indicator. Now take only those MACD buy crossovers when the Full stoch is oversold and MACD sell crossovers when the Full Stoch is overbought. This will not only ensure that you avoid taking those false signals, but will also let you ride the trend longer by ignoring those false whipsaws.

Again, the set of rules for filtering should be minimal so that one can quicky make decisions when the time comes to pull the tirgger.

So in essence, the goal of a system is not make it confirm to price turns but instead using it as a probe to gather information and determine price turns. This requires watching the market and making the decisions, which is a lot of work, as opposed to the lazy approach of the mechanical system making decisions for you. Well, in the end, one gets what one deserves.

Which momentum indicators work the best ?

They are all good. RSI, MACD, CCI, Stochastics, etc as long as one understands how the indicator works and understanding all it's nuances in various market contexts.

I use Full Stochastics as a overbought/oversold oscillator. I use MACD and CCI for determining momentum turns. Of course, with a bunch of EMAs for determining dynamic supports and resistances.


What i have tried to describe is what a general trading system should comprise of, without giving the details of my system, which i have no intentions of giving out. My point is that there are no magical indicators, no holy grail. Don't force your system to see what you want. It's called indicator illusion !


all you mentioned is known (to me)..except for the settings which you do not seem to want to reveal? (know about curve fitting etc)

#14 *JB*

*JB*

    Member

  • Traders-Talk User
  • 915 posts

Posted 22 July 2006 - 09:47 PM

There are even more clever curve fitted systems which looks like a cyclical indicator which catches all the swing turns, which are purely mathematical curve fitting algorithms, which operate on a data series. Let's not even get there. This discussion is about momentum indicators.

Nav --

a GREAT post on indicators/mechanical systems -- momentum ones are the worst when one tries to make a mechanical system out of them.

A few points.

1 -- We have all noticed, I'm sure :rolleyes: , that MACD (for example) will crossover and fall as price rises (visa versa for down trend). IF one uses a confirming signal (I'm NOT discussing a mechanical system here) -- such as breaking a (correctly drawn) trendline -- one can do much better in picking turns if you are willing to leave some money on the table (my strong belief is that signals generated AFTER the turn at the most SAFE and can get you in soon enough to make a profit with clear exits being obvious)

2 -- I have found the above "combo" -- using weekly charts -- is a very solid way to manage a long term -- UNLEVERAGED -- account, like an index fund or ETF or the like. (I traded my wife's retirement account after we transferred it to a self directed IRA (going only long, of course) and NEVER had a losing year from 1992 to present) using just this simple system. (had some GREAT years when trend ran, got out for all significant down moves, including 2000.

3 -- Double and triple divergences (with a trend break) often give great signals for this kind of account.

4 -- Curve fitting an unprofitable system is SO easy to do -- especially if you have some kind of optimizer with you software (like Tradestation)...OPTOMIZERS, the most dangerous trading software component there is.

There are a ton of bums out there who who optomize a mechanical system and sell them (some that trade off of 1 and 5 minute charts!!). I am still seeing that same ads from the same system sellers I first saw in the early 90s...(David Wright from Canada is one)
"Don't think...LOOK!"
Carl Swenlin, founder of Decision Point and original Fearless Forecasters board.

#15 Rogerdodger

Rogerdodger

    Member

  • TT Member*
  • 26,863 posts

Posted 22 July 2006 - 10:29 PM

John, I agree the idea of using trend lines and MACD is very simple and effective.
No better example than Jack Chan at Stockcharts public lists.
http://stockcharts.c...84904&r=959.png

#16 NAV

NAV

    Member

  • Traders-Talk User
  • 16,087 posts

Posted 22 July 2006 - 10:48 PM

There are a ton of bums out there who who optomize a mechanical system and sell them (some that trade off of 1 and 5 minute charts!!). I am still seeing that same ads from the same system sellers I first saw in the early 90s...(David Wright from Canada is one)


JB,

Yes, not many novices understand this fact about optimizers and waste a lot of money on not just the systems, but also by trading them. That's one of the reasons for my post.

"It's not the knowing that is difficult, but the doing"

 

https://twitter.com/Trader_NAV

 

 


#17 arbman

arbman

    Quant

  • Traders-Talk User
  • 19,504 posts

Posted 22 July 2006 - 10:58 PM

Rodger, I understand the support lines on that chart, but how he draws the resistance line ® is a mistery to me, it seems like you would rather draw over the previous highs and think of it more of a b/o and a retest at first...

Posted Image

- kisa

#18 ogm

ogm

    Member

  • Traders-Talk User
  • 13,780 posts

Posted 22 July 2006 - 11:20 PM

1 -- We have all noticed, I'm sure :rolleyes: , that MACD (for example) will crossover and fall as price rises (visa versa for down trend).



MACD should be traded only keeping in mind the with the position of the averages themselves. The most profitable trades actually come from reverse crossovers. For example if EMA 12 is above the EMA 26, take only the trades when MACD 12,26 crosses up. Reverse crossover is a very powerful momentum signal. And the closer the reverse crossover is to the zero line the better.

As I said, in my post.. the position of the moving averages is above everything else.

Just an example. Look how the averages themselves never crossed down, but how many profitable trades you could make on the reverse crossovers.

Posted Image

Edited by ogm, 22 July 2006 - 11:25 PM.


#19 Sentient Being

Sentient Being

    Member

  • Traders-Talk User
  • 4,262 posts

Posted 23 July 2006 - 08:30 AM

Nav, thanks for sharing your thoughts on trading in such detail.
In the end we retain from our studies only that which we practically apply.

~ Johann Wolfgang Von Goethe ~

#20 vitaminm

vitaminm

    Member

  • Traders-Talk User
  • 6,701 posts

Posted 23 July 2006 - 10:23 AM

http://finance.yahoo...P...9,r14,ss&c=
vitaminm