Mechanical System Stability and Market Evolution - Towards the Holy Grail

[Tadragh1]

Hello, Everyone!

I have been around for some time, though did not post much before. I've been involved in automated trading systems development for a few years now, still looking for a system I can trust enough to use it for live trading. In this thread I would like to discuss an issue, which is very important from my point view and seems not to be discussed often - mechanical trading system stability in the face of evolving financial markets.
For the sake of this thread let us concentrate on "technical analysis"-based systems, understood as either classic chart analysis or some kind of statistical modeling.
I am happy to get into quite a technical discussion here, though let us start in a simple and (hopefully) clear way. I'll try to convey the subject in the form of questions and answers.
Alright, enough of the introduction.

1Q. What is a Holy Grail?
1A. It is a trading strategy bringing a) adequate profits (level determined personally) and b) exhibiting a certain risk profile (again determined personally), which is c) stable over time. Some might add that d) it should also be effective on different markets. (This definition is clearly my own, but probably general enough for many of you to accept.)

2Q. Why (many) traders do not find their Holy Grail, what is missing from their strategies or what is wrong with them?
2A. If only mechanical systems are taken into account, subjectivity and inconsistent performance is eliminated, so are psychological issues (to an extent). Let us ignore potential technical / IT / broker / slippage issues. It seems that although it is not that difficult to find systems with a) and b) characteristics, it is hard to combine these with c), not to mention d). So I consider c) (stability of a system's results) to be the key point in the search for the Grail.

3Q. What are the risks of running profitable and not overly risky systems which are not stable over time?
3A. In case the trader is aware that his system's profitability might end, he is watching for signs of it drying out. However determining these signs might be tricky and waiting for the ultimate confirmation might cost him a lot. Also, while looking for a stable, profitable system, he might have overoptimized in the backtesting phase to fit a long period of data and actually his system would not work from the beginning or would begin to fall apart quickly.

4Q. Why do systems work over a certain period and then gradually or suddenly fail?
4A. I can see three reasons: they were overoptimized; they worked by pure luck; the market behaviour changed. I would like to concentrate in this thread on market behaviour change.

5Q. What do you mean by the "market behaviour change"?
5A. I mean that certain market characteristics changed so trading rules which used to be profitable stopped bringing profit. For example traders might say that the X market used to trend nicely, but now is much less directional – or it used to be very volatile, but now trades in a significantly narrower range.

6Q. Why do market behaviour changes occur?
6A. As the financial markets evolve in general, also the market participants’ behaviour changes. For example if more market participants trade in one market and liquidity increases, it is quite probable that some of the market properties change. Whether it is an evolutionary process rather than a cyclical one is tough to judge. Possibly a mixture of both, especially if you consider it on different time scales – as one might say that market behaviour is different during different times of the day (cyclical) and in the same time it changes from year to year (evolutionary), still going through periods of recessions and dynamic growth (cyclical).

HERE COMES THE CENTRAL QUESTION OF THIS THREAD

7Q. Is there a way to cope with this market change process in order to get one step closer to the Grail?
7A. I can see two possibilities. One, a mild version, is to try to link the system parameters with certain market characteristics. Some people for example link their stop loss to market volatility. This is what I have in mind in here. Such a system would be a self-adjusting one, constantly reparametrizing along with the market, although the main principle would remain constant. As an example you could take an EMA crossover system with EMA periods depending on long-term market volatility and the stop loss / take profit parameters tied to short-term volatility. The full-blown solution would be to actually develop a few of these self-adjusting systems basing on different trading principles and choose which one to operate basing on market characteristics. For example one might trade an EMA-based system when he considers the market to be trending and some kind of an oscillator-based system when the market is ranging. Of course, the number of such systems might be quite high, actually enabling such a Grail (being a strategy consisting of many systems) to operate in different markets.

8Q. What market characteristics could be used to differentiate between the market evolution stages?
8Q. A couple of ideas: profitability of simple trading rules, descriptive statistics, various volatility measures, Hurst exponent, autocorrelation structure.

I am looking forward to your answers to the questions 7 and 8. I hope this topic proves as interesting to you as it is to me :cheesy:

Cheers!

 

[Glenn]

Quite simple really. Markets only behave 'Technically' some of the time, not all of the time.
Events (News etc) will throw TA out of the window until things settle down again.
Even Gaps (which are arguably small events, or represent responses to them) can cause a shift in the TIme aspects of Indicators.
Markets are Chaotic, meaning that they respond to Initial Conditions (events) and events are not technical in nature.
You cannot rely solely on TA all the time and any attempt to adjust a technical method, the apparent need for which adjustment was caused by 'changing market conditions', will result in failure because event effects fade and TA works again in the same old way.
So don't tinker, just sit on the sidelines until 'normality' prevails.
As for automation of sitting on the sidelines - good luck
imho.
Glenn

 

[Tadragh1]

Glenn, point taken
However I rather would concentrate on a large time scale and general changes. For example - I am backtesting a system which works fine between 2000-2005 and then starts losing money. I argue that the system is good for specific market conditions which occured between 2000-2005, but then a "market behaviour change" occured.

 

[Market Wizard]

5Q. What do you mean by the "market behaviour change"?
5A. I mean that certain market characteristics changed so trading rules which used to be profitable stopped bringing profit. For example traders might say that the X market used to trend nicely, but now is much less directional – or it used to be very volatile, but now trades in a significantly narrower range.

Hello Tadreagh,

Great post .

I have also heard about markets changing their behaviour, though have not experienced it fully first hand yet. Also your explanation to the reason why is the same conclusion that i came to . I found that as the OVERALL volatility changes, though the entry/exit parameters are still met , in some instances the market is just giving away limited points after taking into account slippage/spreads etc.

 

[Glenn]

Glenn, point taken
However I rather would concentrate on a large time scale and general changes. For example - I am backtesting a system which works fine between 2000-2005 and then starts losing money. I argue that the system is good for specific market conditions which occured between 2000-2005, but then a "market behaviour change" occured.

Backtesting - Hmmm. - another touchy subject.
Optimised ?
Only one 5-year Short trade ? (sorry, couldn't resist that - lol).
History is no guide to the future ?

A sample list of other things that can make a different to a market.
1. Quants.
2. Hedge funds (some of whom have recently blown up after using Quants who should have received 'adult supervision'). Ftse option premiums fell quite a lot when the Hedge funds started up.
3. New major players, skilled and unskilled.
4. Old major players leaving.
5. New/old Marlet makers.
6. New Exchange rulings.
7. Electronic trading and elimination of Pit trading
8. Rogue traders
9. Bubbles
etc. etc.
what next ?- 24 hour markets ?

There is a fellow whose name I forget who runs a mech system and alonsgide it runs several clones of he same system with different parameters. When the main system starts to go awry he looks to see which of the parallel systems is performing and adapts to that. Whether this works or not I have no idea, but it's an idea which may interest you.

Glenn

 

[Tadragh1]

Backtesting - Hmmm. - another touchy subject.
Optimised ?
Only one 5-year Short trade ? (sorry, couldn't resist that - lol).
History is no guide to the future ?

One short trade is a good one :cheesy: Anyway, it was a hypothetical suggestion saying that I often happen to find systems which "stop working", and other people also mention this phenomenon quite often, too. The dates were just given as an example.

A sample list of other things that can make a different to a market.
1. Quants.
2. Hedge funds (some of whom have recently blown up after using Quants who should have received 'adult supervision'). Ftse option premiums fell quite a lot when the Hedge funds started up.
3. New major players, skilled and unskilled.
4. Old major players leaving.
5. New/old Marlet makers.
6. New Exchange rulings.
7. Electronic trading and elimination of Pit trading
8. Rogue traders
9. Bubbles
etc. etc.
what next ?- 24 hour markets ?

Agreed, definitely - the question is how to take all of these into account while trading mechanically?

There is a fellow whose name I forget who runs a mech system and alonsgide it runs several clones of he same system with different parameters. When the main system starts to go awry he looks to see which of the parallel systems is performing and adapts to that. Whether this works or not I have no idea, but it's an idea which may interest you.

Definitely interesting. I know that you forgot the name but any idea how could I look for him, a website possibly?

Actually, this "one 5-year trade idea" gave me a thought - I argue that traders use my kind of thinking (adapting to changing market conditions) all the time, but in a very simple way. All I am saying is that one should adapt the parameters of the strategy he uses or the strategy itself to changing market conditions, which may change either cyclically or evolutionary. Let's start from the first principles, dumb strategies. I will give an example of a very simple strategy and then adjust it for the market changes. Let's say I am trading a stock:

1. Strategy 1 is - buy and hold.
I earn some money, but after a few days/weeks the stock starts going down, although it never did before (market changed), so I modify my strategy.
2. Strategy 2 is - buy and hold only when the stock is trending up. To determine trend draw a line on the chart.
3. I notice that my lines sometimes mislead me and do not show trends properly. I am starting to use moving averages to gauge the market trend (strategy 3).
4. Working fine, but I notice, that actually I could make money also when the stock is in a downtrend - so I sell short when I encounter a downtrend (strategy 4).
5. I notice, that the market actually sometimes trades in a range, so I add oscillators (strategy 5).
And so on and so forth...

So actually this trader chooses his "strategy" (simply either "buy" or "sell") depending on the market characteristics gauged by moving averages, oscillators etc. I argue we can go one more level up in this - choose the gauges we use to make "buy" and "sell" signals also depending on market characteristics and assume that they actually change: and that such a change actually does not prove that a system is wrong. Because "buy and hold" is a fantastic system - when certain market characteristics apply :cheesy:

Ugh, I hope what I am writing does not appear too convoluted

 

[Glenn]

One short trade is a good one :cheesy: Anyway, it was a hypothetical suggestion saying that I often happen to find systems which "stop working", and other people also mention this phenomenon quite often, too. The dates were just given as an example.



Agreed, definitely - the question is how to take all of these into account while trading mechanically?



Definitely interesting. I know that you forgot the name but any idea how could I look for him, a website possibly?

Actually, this "one 5-year trade idea" gave me a thought - I argue that traders use my kind of thinking (adapting to changing market conditions) all the time, but in a very simple way. All I am saying is that one should adapt the parameters of the strategy he uses or the strategy itself to changing market conditions, which may change either cyclically or evolutionary. Let's start from the first principles, dumb strategies. I will give an example of a very simple strategy and then adjust it for the market changes. Let's say I am trading a stock:

1. Strategy 1 is - buy and hold.
I earn some money, but after a few days/weeks the stock starts going down, although it never did before (market changed), so I modify my strategy.
2. Strategy 2 is - buy and hold only when the stock is trending up. To determine trend draw a line on the chart.
3. I notice that my lines sometimes mislead me and do not show trends properly. I am starting to use moving averages to gauge the market trend (strategy 3).
4. Working fine, but I notice, that actually I could make money also when the stock is in a downtrend - so I sell short when I encounter a downtrend (strategy 4).
5. I notice, that the market actually sometimes trades in a range, so I add oscillators (strategy 5).
And so on and so forth...

So actually this trader chooses his "strategy" (simply either "buy" or "sell") depending on the market characteristics gauged by moving averages, oscillators etc. I argue we can go one more level up in this - choose the gauges we use to make "buy" and "sell" signals also depending on market characteristics and assume that they actually change: and that such a change actually does not prove that a system is wrong. Because "buy and hold" is a fantastic system - when certain market characteristics apply :cheesy:

Ugh, I hope what I am writing does not appear too convoluted

The chaps name is Dr John F Clayburg. I don't know if he has a web site, I just happened to read about him.

Your ideas may be all very well in theory, but are they based on wishful thinking or detailed research ? If highly qualified Quants can get it wrong, do you suppose that you can do better ? It's not clear how much experience you have.
One thing you must include is Risk Management first and foremost (above money management and all other considerations).

"the question is how to take all of these into account while trading mechanically?"
Are you assuming that that is possible, or do you know that it is ?
In my view you have to choose those times when you should stay out of the market altogether.
"Markets can stay irrational longer than you can stay solvent."

Glenn

 

[zentrader22]

@Tadragh1,

concerning your questions there are imho two main activity areas:

1. during system development you have to think about the real nature of market. Not that i think, that conventional TA or higher stat maths are always bad, but markets behaviour isn't linear. So I think you should use some non linear aspects and power laws to build a trading system which can "dance" the market rythm... (Mandelbrot, Sloman and others give a good input to this)

2. during the system test phase you can improve your back test quality with monte carlo simulations. Not only the usual stress tests (although this is important, because risk also isn't linear) but more important data simulation, which enables you to simulate future market conditions (with random influence). Ok, also with this method there is no 100% reliability (think at Taleb's "Black swan"), but it's a closer step to validate your system setup. I've described this in an Traders' article some time ago:
http://www.zentrader.de/mcs_article_traders2007.pdf

bye,
zentrader

 

[Tadragh1]

The chaps name is Dr John F Clayburg.

Thanks, Glenn, I found his website: Clayburg.Com - Your Source on the Web for Self-Adaptive Systems and Indicators
It actually looks quite interesting.

Your ideas may be all very well in theory, but are they based on wishful thinking or detailed research ? If highly qualified Quants can get it wrong, do you suppose that you can do better ? It's not clear how much experience you have.
One thing you must include is Risk Management first and foremost (above money management and all other considerations).

I am an econometrician by profession, hence my approach. Another guy who does something similar is John Ehlers, he is a signal processing guy. In one of his books, "Rocket Science for Traders: Digital Signal Processing Applications" (cheesy title, but the book is interesting), he describes a system differentiating between trend and cycle trading basing on market characteristics. The backtest results seem good, but they are not very recent. His website is here: Home There are a couple of interesting articles there.

In my view you have to choose those times when you should stay out of the market altogether.

Sure, Glenn - I agree. So can you define such a time, do you use this approach in your systems?

1. during system development you have to think about the real nature of market. Not that i think, that conventional TA or higher stat maths are always bad, but markets behaviour isn't linear. So I think you should use some non linear aspects and power laws to build a trading system which can "dance" the market rythm... (Mandelbrot, Sloman and others give a good input to this)

Sure, Zentrader22, but aren't the scaling laws derived exactly through "higher stat maths"? Olsen and Associates give a lot of information about this, Olsen - Advancing the Frontiers of Finance The book they wrote "Introduction to High-Frequency Finance" is highly recommended. Do you use some of these concepts in practice while designing your systems?
By the way, I checked out Jim Sloman, but it looks a bit "hype" to me:
Jim Sloman's Ocean Book
Does he actually add something to Hurst/Mandelbrot/Olsen?

2. during the system test phase you can improve your back test quality with monte carlo simulations.

Of course, this is quite a common approach. My question is how to gauge the changes in market behaviour, which cause a specific system to stop working? Or in other words, how to determine, which system to use in which market?

Thank you, Gentlemen, for interesting points!

 

[mr_cassandra]

mechanical systems

I think that for any system to try and fit the meaning of robust, it needs to survive a backtest thu a bull mkt, bear mkt and trading range mkts.

In addition, it should avoid over-dependence on any one indicator or genre of indicators.

By that I mean vix and pcr would be a better combo than normal pcr and equity pcr. Along the same track if you could get volume indicators working in tandem with price all the better.

You need (imo) indicators looking at various things, rather than 5 indicators all looking at price.

Hello, Everyone!

I have been around for some time, though did not post much before. I've been involved in automated trading systems development for a few years now, still looking for a system I can trust enough to use it for live trading. In this thread I would like to discuss an issue, which is very important from my point view and seems not to be discussed often - mechanical trading system stability in the face of evolving financial markets.
For the sake of this thread let us concentrate on "technical analysis"-based systems, understood as either classic chart analysis or some kind of statistical modeling.
I am happy to get into quite a technical discussion here, though let us start in a simple and (hopefully) clear way. I'll try to convey the subject in the form of questions and answers.
Alright, enough of the introduction.

1Q. What is a Holy Grail?
1A. It is a trading strategy bringing a) adequate profits (level determined personally) and b) exhibiting a certain risk profile (again determined personally), which is c) stable over time. Some might add that d) it should also be effective on different markets. (This definition is clearly my own, but probably general enough for many of you to accept.)

2Q. Why (many) traders do not find their Holy Grail, what is missing from their strategies or what is wrong with them?
2A. If only mechanical systems are taken into account, subjectivity and inconsistent performance is eliminated, so are psychological issues (to an extent). Let us ignore potential technical / IT / broker / slippage issues. It seems that although it is not that difficult to find systems with a) and b) characteristics, it is hard to combine these with c), not to mention d). So I consider c) (stability of a system's results) to be the key point in the search for the Grail.

3Q. What are the risks of running profitable and not overly risky systems which are not stable over time?
3A. In case the trader is aware that his system's profitability might end, he is watching for signs of it drying out. However determining these signs might be tricky and waiting for the ultimate confirmation might cost him a lot. Also, while looking for a stable, profitable system, he might have overoptimized in the backtesting phase to fit a long period of data and actually his system would not work from the beginning or would begin to fall apart quickly.

4Q. Why do systems work over a certain period and then gradually or suddenly fail?
4A. I can see three reasons: they were overoptimized; they worked by pure luck; the market behaviour changed. I would like to concentrate in this thread on market behaviour change.

5Q. What do you mean by the "market behaviour change"?
5A. I mean that certain market characteristics changed so trading rules which used to be profitable stopped bringing profit. For example traders might say that the X market used to trend nicely, but now is much less directional – or it used to be very volatile, but now trades in a significantly narrower range.

6Q. Why do market behaviour changes occur?
6A. As the financial markets evolve in general, also the market participants’ behaviour changes. For example if more market participants trade in one market and liquidity increases, it is quite probable that some of the market properties change. Whether it is an evolutionary process rather than a cyclical one is tough to judge. Possibly a mixture of both, especially if you consider it on different time scales – as one might say that market behaviour is different during different times of the day (cyclical) and in the same time it changes from year to year (evolutionary), still going through periods of recessions and dynamic growth (cyclical).

HERE COMES THE CENTRAL QUESTION OF THIS THREAD

7Q. Is there a way to cope with this market change process in order to get one step closer to the Grail?
7A. I can see two possibilities. One, a mild version, is to try to link the system parameters with certain market characteristics. Some people for example link their stop loss to market volatility. This is what I have in mind in here. Such a system would be a self-adjusting one, constantly reparametrizing along with the market, although the main principle would remain constant. As an example you could take an EMA crossover system with EMA periods depending on long-term market volatility and the stop loss / take profit parameters tied to short-term volatility. The full-blown solution would be to actually develop a few of these self-adjusting systems basing on different trading principles and choose which one to operate basing on market characteristics. For example one might trade an EMA-based system when he considers the market to be trending and some kind of an oscillator-based system when the market is ranging. Of course, the number of such systems might be quite high, actually enabling such a Grail (being a strategy consisting of many systems) to operate in different markets.

8Q. What market characteristics could be used to differentiate between the market evolution stages?
8Q. A couple of ideas: profitability of simple trading rules, descriptive statistics, various volatility measures, Hurst exponent, autocorrelation structure.

I am looking forward to your answers to the questions 7 and 8. I hope this topic proves as interesting to you as it is to me :cheesy:

Cheers!

 

[rnicoll]

Bit of background: I'm a full time software developer/researcher as a day job, doing part time PhD in my free time, and trading when I should be sleeping. I've been backtesting and coding algorithmic systems for about a month now, and have a system almost ready for live testing.


The biggest obstacle I see to "the holy grail" is that the market inherently reacts to trading. My trading system is based on spotting when human traders get carried away and the market swings significantly out of where it probably should be, and trading their errors. I fully expect that while my initial live tests will be too small for the market to notice, once I start trading any significant amounts the market will adapt to my trades and optimise out the flaws I'm trading. Traders who cannot adapt like this will lose money, and will drop out.

Genetic algorithms or neural nets could theoretically allow for self-developing/modifying systems that can evolve themselves, but the processing requirements to evolve these are... significant... and it then requires trusting large sums of money to a system that is likely beyond the comprehension of it's creators.

So far, I'm seeing algorithmic trading as a way of:

1. Trading 24/7; particularly useful on Forex.
2. Smoothing out workload over time.
3. Working more efficiently.

So, I fully expect to continually have to develop new algorithms to stay ahead of the market, but if I can do that better than I can trade by hand, then that makes sense for me. I don't think there ever will be a single solution to trading the market.

 

[Victor90]

I fully expect that while my initial live tests will be too small for the market to notice, once I start trading any significant amounts the market will adapt to my trades and optimise out the flaws I'm trading. Traders who cannot adapt like this will lose money, and will drop out.

If you are going to trade a big market, like the forex, I belive that you will never reach that volume alone. I read about a guy who had gone in with a 100 000 000 position at EUR/USD. It only moved the market 1 pip.

 

[rnicoll]

If you are going to trade a big market, like the forex, I belive that you will never reach that volume alone. I read about a guy who had gone in with a 100 000 000 position at EUR/USD. It only moved the market 1 pip.

I was hoping that might be the case. Awkwardly, I get the stablest results on freaky crosspairs like EUR/AUD, which is more likely to show these up. Still, keeping trades small but frequent may well be a perfect good approach to staying under the radar as it were...

 

[Victor90]

I get the stablest results on freaky crosspairs like EUR/AUD, which is more likely to show these up

Well, im no expert. But since your system is about profiting from misspricing there should be more misspricing on "freaky" crosspairs since my guess is that 99% of the volume in EUR/USD is either done by Institutional traders or big companies that are exchanging money. On the EUR/AUD i think that more volume (in %) is made by non professionals (my defenition of those who do not work for a big fund or a bank). Then the chances are greater that a missprice will exist. But this is just my opinion.

 

[rnicoll]

Well, im no expert. But since your system is about profiting from misspricing there should be more misspricing on "freaky" crosspairs since my guess is that 99% of the volume in EUR/USD is either done by Institutional traders or big companies that are exchanging money. On the EUR/AUD i think that more volume (in %) is made by non professionals (my defenition of those who do not work for a big fund or a bank). Then the chances are greater that a missprice will exist. But this is just my opinion.

Also occurs that there's also a MUCH better chance that errors aren't snaffled up by someone else in the relatively long time it takes my stuff to respond...

Now if only EUR/NZD had enough liquidity for the spread to be vaguely sane...

 

[Victor90]

aha, yea! The big spread! Thats the downside of strange crossings.

However, when you get BIG volume you will have much lower spreads

 

[rnicoll]

aha, yea! The big spread! Thats the downside of strange crossings.

However, when you get BIG volume you will have much lower spreads

I think I've got my idea for version 2 of this algorithm :-D

 

[Victor90]

I think I've got my idea for version 2 of this algorithm :-D

Thats great
I know you cant (and shouldnt) give the system here, but can you tell what kind indicators you are using in your system, becuse it sound so intresting.

 

[rnicoll]

Thats great
I know you cant (and shouldnt) give the system here, but can you tell what kind indicators you are using in your system, becuse it sound so intresting.

It's not actually anything that impressive, my first stab at an algorithm seemed to work fairly well, and I've just been polishing it for a while now. It's based off Bollinger bands and simple moving average; the trickyness is in how it reads them.

Edit: SMA over a different timescale than the one in the Bollinger bands, I mean.

 

[Victor90]

aha, very intresting. Like an automated scalper?