Sample Size

[milesy]

Well I've been testing systems and looking for a trading plan for a couple of years now without any real success.
I usually look at 5 minute timeframes on the US and UK futures and am looking for something that gives a 60% + win rate with stop equalling target (around 10 ticks /pips). Alternatively I have looked at stuff with 15 tick stop and 10 tick target and looking for 70% plus win rates.
Now and then I've come up with signals that look interesting and started gathering some stats on them as they occur as well as going through historical charts and seeing how the signal fares over different market conditions.
Recently I have been testing a simple macd crossover system that showed some potential on the ftse. At a sample size of 150 trades using 10target 15 stop I was showing a consistent 70 to 70% win rate. Over a sample size that large I thought that I had something.... I carried on testing and once up to 250 trades my equity curve was break even.
Just a reminder that when you test something make sure the sample size is big enough.... how big? I 'm not sure yet and that's something I'll be giving a lot of thought and study to before I go live.
Something that may hold some truth : trading on larger hourly or daily timefames may need smaller sample sizes than trading on 3 or 5 min simply because larger timeframes are taking more of the noise out of the Market , thus signals being more reliable.
I am very interested in other Traders thoughts on signal sample sizes before going live,
Cheers,
Milesy

 

[Rossini]

You seem to be looking for the holy grail. I would have thought any sample size large enough would give the results that you show. Non of my business but 2 years of just looking and not trading (or demo trading?) seems a very long time.

 

[Rossini]

P.S. If the method is suited to particular market conditions for example the 30min breakout. Then it is up to you to identify when the method is breaking down and adjust the position size accordingly. Back tested every day this method looses money, but if you add in some discretion e.g. not taking the signal when there is a huge gap and instead playing the gap! It makes a huge difference. I would just take a simple method like this one and start trading it using very small stakes. I think you will learn more this way.

 

[ronblack]

Michael Harris of Tradingpatterns.com makes a few suggestions regarding data series and trade sample sizes here:

Data FAQ

I agree that shorter time period require a larger sample for basic statistical significance.

Ron

 

[milesy]

Thanks Rossini but I don't think that I am looking for the holy grail, an Edge yes. I appreciate your advice to keep it simple and and add some discretion to my trading but surely adding some discretion should come after finding some sort of mechanical edge?
I certainly agree that knowing in what market conditions my system or edge works can make a huge difference and that is where I will be concentrating in the future.
However my post really is to get traders thinking more about sample size, some think that if they have 40 winners out of 50 and a positive profit ratio backtesting then they are on to a winner and then go live.
Mark Douglas recommends a minimum of 20 trades sample size, if that were sufficient then the systems I have tested that looked good at 150 trades should have been clear winners, not coming back to break even after 250.
So far all the indicators and systems I have looked at have not been able to statistically provide me with an edge. Maybe at the end of the day they never will and the only edge that I, You or every other Trader in this arena is going to have is going to come from our discretionary trades and the sooner we admit it and trade purely from support, resistance, volume and price action and forget about indicators, the better.
Trade To Trade Well,
Milesy

 

[Rossini]

I think one of the biggest edges that a retail trader can have, is the ability to stand aside when he/she feels that the conditions are not optimum or do not feel right. Imagine you are a market maker, you can see the market, you have a very strong idea that the market will continue downwards but you still have to make the prices and offer your customers the chance to sell it! I think not having to trade is a good edge and certainly one that I try to make full use of.

 

[milesy]

I think one of the biggest edges that a retail trader can have, is the ability to stand aside when he/she feels that the conditions are not optimum or do not feel right. Imagine you are a market maker, you can see the market, you have a very strong idea that the market will continue downwards but you still have to make the prices and offer your customers the chance to sell it! I think not having to trade is a good edge and certainly one that I try to make full use of.

Sounds like a fine way to approach and assess the markets and get an edge, thanks Rossini I'll definitly look into standing aside under certain conditions,
Milesy.

 

[forex99]

I would suggest you stay away from such small timeframes and start watching 1/4hr charts for trade setups and only then go to a 5/15 min chart for entry.
If you're not seeing positive returns after 30 trades I would look for something else.
Don't get stuck up on what percentage of your trades actually make money. The key concept in trading is to have your winners out run your losers. Simple maths.
My system is right maybe one in five but my winners are 6-10 times my losers. I trade off an hourly chart so there are plenty of set ups. Mechanical systems I personally don't feel will work forever. The market changes over time and the system just dies.