The definitive test

[stevet]

the foundation for all systems must come from 'back testing' to the extent that you must have arrived at the parameters for the system from what you have seen occur in the market previously

but it is impossible to back test an intra-day system since the number of variables are too great as well as their integrity

and to a large degree this would even effect day period or longer systems which are concerned with any intraday data, as opposed to purely end of day closes

and then you plug it in on low size, iron out the bugs, and refine until you know it is a winner or a pile of ****!

 

[JTrader]

Thanks.

I have recently acquired ts2000i, and plan to do all my system writing, backtesting and optimising with this software.

say you were using a day period mechanical trading system for example, which you had back-tested over 1,2 or maybe even 3 years. How often would you review the parameters (and optimise the parameters if necessary)? does on a monthly basis sound sensible?

And if paying attention to the most recent past is important. If performing monthly review backtests/optimisations, over how far backs worth of data would you perform the monthly review backtests/optimisations?

Would any of these considerations be affected by the length of the intraday timeframes that you traded within?

Many thanks

jtrader.

 

[stevet]

once you trade a system - you will find that the second it loses money for you - you will become uniquly focused on reviewing its parameters at every possible opportunity and in an extremly intense way - and should it just make money for you - you will be uniquly focused on not even breathing near it - in case you upset it in any way

 

[JJL3142]

stevet

What do you trade ?

Your profile seems rather bare for someone who has put 950 opinions on every subject under the sun

 

[JTrader]

Thanks

do you think it is possible to trade basically the same mechanical day or intraday system, using the same indicators, for years on end, with the only alterations that you do make being periodical reviews/optimisations of the included indicators parameter settings?

Cheers

jtrader.

 

[mr_cassandra]

Historicaal vs real time

I think this concern needs further consideration. If you are dealing with a short term ta based trading system with 20 years of backtest that encompass bull-bear-flat-and bubble mkts, how much more can you expect it to do? In 20 years, it got thru all those mkt conditions and placed hundreds of trades. I suspect what then happens is the people who say they need to see real time results will then shift the time needed forward each time the system does what they originally asked for.

I have done some work in this area and it seems to me the only test is on realtime data as historicl simulations may well be curve fitted and the curve fitting can be very well done so that it continues to look very stable over all variables.
This may not be exactly what you are asking but I think that as far as construction is concerned there is a fairly well defined criteria for testing systems and it is based on the splitting of data into 3 parts. the first is used for the system construction to test the idea, the second to refine the system and the third to walk forward test the system as if it is real-time. If the system performs well on the third set then it may be assumed to be reasonably stable. The actual process through these 3 steps is too complex to go into in one mail. Just the data integrity issue is bad enough at least when it comes to Futures. I am sure there is a good amount written about it however and maybe somebody could suggest a good site/books on the subject.

 

[mr_cassandra]

time frame

I saw someone say a long time ago that the shorter the time frame you try to work with, the better your chances of success are. As the years have gone by, this looks more and more true.

A good discussion topic.

"..the rules need to be defined before anyone can say they have a winning system:"

Are you assuming that there is such a thing as a winning automatic trading system ?
Imho this alone is a very big assumption which must be proven before trying to look any further into rules etc.

There are plenty of people who can produce a backtested system which has given good results (although they may not disclose the method ). But that proves nothing.
I too have such a system which has produced 27,000 points allowing for commissions and slippage since June 1998, including walk forward testing. At the moment it is long in a down market.
I have learnt that applying some simple TA (Supp/Res) has prevented me entering losing positions generated by the system, so currently I am flat instead of 120 points adrift.
I remain to be convinced about the merits of fully automated systems and expect that a lot of time needs to pass in order to 'prove' any system. Until that happens, rightly or wrongly I will continue to override the system as I see necessary.

At a practical/psychological level, long term systems generate another quandary. It is easy to look at a 5 year equity chart and see the big rise over time. But when you look at the detail you can see that there a long flat periods and losing periods, some of which can last for months.
It is very different when you are actually trading them in realtime.
And if you happen to start trading a system during one of the losing periods, it can be rather disconcerting

Perhaps the acid test is whether a system will work on a randomly generated dataset.

Glenn

 

[Hoggums]

I saw someone say a long time ago that the shorter the time frame you try to work with, the better your chances of success are. As the years have gone by, this looks more and more true.

Going to the original question - you shold divide your back data into two - optimise your algorithm using half the data and then run a forward test on the other half. If your system makes a profit on both then you have a good starting point - but it doesn't mean to say that your system will continue to make a profit!

There has been no mention here of adding in the cost of rolling transactions, commissions and spreads and of course money management.

It's not realistic to expect you wont scale up the trade size as you make money - so I've added a rudimentary money management to the spreadsheet we were supplied with, I've chosen a fairly agressive system which allocated 1/500th of your pot to each point risked in your trades. I've also assumed that you only do 1 trade per day and the cost is just 1 pip either side of the transaction. Even though your system has made a 266 point profit by 8 June, when you add in money management you have only just gone past break even! After 3 months - you've made no money - do you think you would continue it?

Also your maximum drawdown on this system is a whopping 70% - so if you drop down the aggressiveness of the money management to 1/2000th of the pot per trade point leaving a max of 22% (which is still high if you're playing with say £100k) then it takes over 2 years to turn your £500 into £10000.

Food for thought.....

 

[zentrader22]

Going to the original question - you shold divide your back data into two - optimise your algorithm using half the data and then run a forward test on the other half. If your system makes a profit on both then you have a good starting point - but it doesn't mean to say that your system will continue to make a profit!

...and if you use also synthetic data (generated from your test data) to simulate changing market conditions during additional system tests you will get even more experiences about the behaviour of your system!

bye,
zentrader
www.zentrader.de