Metatrader backtests flawed

Most of data used for backtesting is gotten from bucketshop's non executable quotes/prices as opposed to live executable prices supplied by the banks.

If trader's try to execute trades at bucketshop non executable quotes , they will not be filled.The bucketshop is merely quoting the interbank rate without a spread.The bucketshop has to add its spread to the non executables prices.

Thats a problem if you need to get filled in fast trending markets.

The other major problem is the bucketshops use dealing desks ,and in the absence of a dealing desk ,automated dealer plugins are employed to carry out dirty tricks.

Dealing desks execution results in execution delay , because order has to be verified by dealing desks .This verification time delays execution and fills in fast trending markets.

If you are a serious trader ,there are big disadvantages in dealing with most metatrader brokers.The serious traders deal directly with the banks.

There are a few honest and reliable brokers using metatrader.

The data supplied on Metatrader demo accounts is flawed.Here are comparisons between live and demo accounts .A 500 pip more loss on real accounts ,the demo had 500 pip less loss.

In the attachment s demo account lost 337 pips @0.10 per pip*£1.63 , and live account lost,and live accounts lost 881 pips .These tests used same expert advisors and set ups on both live and demo accounts

The Forex Non-Dealing Desk Trader

As a result of the above information , We can deduce most of the backtesting using metatrader demo data feeds is flawed.One can not rely on it.

OILDAYTRADER

This is a valid point and another nail in the coffin for the farce of back testing.

Even if it wasn't flawed as above, back testing is based on the incorrect assumption that the market will behave tomorrow like it did yesterday. In a totally random set of figures, there will be patterns (or it wouldn't be totally random) and embryo traders seek out these patterns in the naive belief that they will repeat in the future. Obviously, markets change even if these past patterns had meaning.

There have been literally millions and millions of back tests and yet not one has produced a 100% reliable system for making a profit in real life (over any length of time long enough not to be luck). Yet people still insist on doing it!

People still waste countless hours searching for a system that works by persistent back testing. What they are really doing is adjusting the system until it fits the curve. I know at least one person will reply by saying that you have to back test a system. Argue all you like - back testing is mostly a waste of time and has never produced anything worthwhile. There will be people who think they have found a system that works as a result of back testing but come back in six months time and admit that your account has been blown...

There are benefits of backtesting on many years of data.The backtests will show how the market behaved in different types of market conditions, and backtests will also show how system performs in different market conditions.

To use any backtests to trade with metatrader brokers is certain to result in losses.In live trading these bucketshops deliberately hit traders with slippage, slippage is not shown in backtester.

These bucketshops have virtually zero liquidity,the liquidity they claim of $ billions would be of no use ,liquidity is provided when not required and required liquidity is never available.Backtests and demo accounts show unlimited liquidity.

There are other problems like platform failure,power failure,internet connection failure and computer failure etc

£10kLoser said:

This is a valid point and another nail in the coffin for the farce of back testing.

Even if it wasn't flawed as above, back testing is based on the incorrect assumption that the market will behave tomorrow like it did yesterday. In a totally random set of figures, there will be patterns (or it wouldn't be totally random) and embryo traders seek out these patterns in the naive belief that they will repeat in the future. Obviously, markets change even if these past patterns had meaning.

There have been literally millions and millions of back tests and yet not one has produced a 100% reliable system for making a profit in real life (over any length of time long enough not to be luck). Yet people still insist on doing it!

People still waste countless hours searching for a system that works by persistent back testing. What they are really doing is adjusting the system until it fits the curve. I know at least one person will reply by saying that you have to back test a system. Argue all you like - back testing is mostly a waste of time and has never produced anything worthwhile. There will be people who think they have found a system that works as a result of back testing but come back in six months time and admit that your account has been blown...

Click to expand...

Quite simply, that is wrong.

Rossini said:

Quite simply, that is wrong.

Click to expand...

Too simply, I would say...

Why is it wrong?

£10kLoser said:

This is a valid point and another nail in the coffin for the farce of back testing.

No its a nail in the coffin for back testing with incorrect data, using a live accounts data from someone like FXCM is going to be much more accurate.

Even if it wasn't flawed as above, back testing is based on the incorrect assumption that the market will behave tomorrow like it did yesterday. In a totally random set of figures, there will be patterns (or it wouldn't be totally random) and embryo traders seek out these patterns in the naive belief that they will repeat in the future. Obviously, markets change even if these past patterns had meaning.

How can people make this statement? Markets do not change; they go up, down and sideways. They cannot do anything else; the only thinks that can change is volatility and liquidity, both I would argue are more important factors that contribute to system profitability than whether the market is going up down or sideways. After all there is not point having a system that alerts you to potential profit making opportunities if you cannot get filled.

There have been literally millions and millions of back tests and yet not one has produced a 100% reliable system for making a profit in real life (over any length of time long enough not to be luck). Yet people still insist on doing it!

Well that’s not true either.

People still waste countless hours searching for a system that works by persistent back testing. What they are really doing is adjusting the system until it fits the curve. I know at least one person will reply by saying that you have to back test a system. Argue all you like - back testing is mostly a waste of time and has never produced anything worthwhile. There will be people who think they have found a system that works as a result of back testing but come back in six months time and admit that your account has been blown...

Click to expand...

What you are referring to here is curve fitting and over optimization. We both agree that doing this is very likely to result in system losses. But to say that any system back testing is done to over optimize the parameters to ovoid and minimise losses is obviously a gross generalisation.

Of course markets change. You can create a set of indicators that work well and then fail when the market changes - that can be seen frequently.

Markets change because of sentiment and so they then behave differently. What previously triggered a rise, no longer does so, for example. Chart patterns that usuually meant one thing will then mean either nothing or perhaps something else. This was very obvious after October last year and the 'credit crunch'. Most markets behaved entirely differently from when people believed in unchecked growth.

Systems and indicators that worked well in one set of conditions fail when the market changes.

If you believe that back testing has succeeding in producing a truly profitable system, then kindly reveal it to the rest of us! There have been hundreds if not thousands come and go and every one has failed to live up to its claims. Just flick through ClickBank as a starting point. You are too quick to just say 'that's not true' with no evidence...

Systems have to be continually updated to allow for market changes if they are to remain profitable. Not only do markets change but markets are different from each other. A system may work well on Forex but not gold, for example.

£10kLoser said:

Of course markets change. You can create a set of indicators that work well and then fail when the market changes - that can be seen frequently.

You can, but that is probably because you are curve fitting the 'system' to certain conditions that do not last forever. Take a broader view and this becomes harder to do.

Markets change because of sentiment and so they then behave differently. What previously triggered a rise, no longer does so, for example. Chart patterns that usuually meant one thing will then mean either nothing or perhaps something else. This was very obvious after October last year and the 'credit crunch'. Most markets behaved entirely differently from when people believed in unchecked growth.

I’m sorry I see no evidence of that, to me the markets remain the same post credit crunch, just more volatile.

Systems and indicators that worked well in one set of conditions fail when the market changes.

If you believe that back testing has succeeding in producing a truly profitable system, then kindly reveal it to the rest of us! There have been hundreds if not thousands come and go and every one has failed to live up to its claims. Just flick through ClickBank as a starting point. You are too quick to just say 'that's not true' with no evidence...

You say ''thousands come and go and everyone has failed to live up to it claims’’ So what?? Thousands of mechanical systems have failed, just like thousands of traders have and will fail. Does that mean it is not possible for traders to succeed? No. Does that mean is not possible for mechanical systems to succeed? No. All it means is that you have no heard of any mechanical system succeeding, and why would you? What would be the need for someone who has such a system to show you?

Systems have to be continually updated to allow for market changes if they are to remain profitable.

Click to expand...

Again this is just a generalisation, how can you say this if you do not know the system. Surly it is totally dependent on the system??

Rossini said:

Quite simply, that is wrong.

Click to expand...

I lost interest when I saw the vendor tag; selling trader software programmes...

Black Swan said:

I lost interest when I saw the vendor tag; selling trader software programmes...

Click to expand...

I didnt see that...

Rossini is correct, invest some time in backtesting and you'll realise why.

OK, now I know why I gave up posting here a while ago.

If I was arguing from the point of view of a vendor of software, I would be arguing the case FOR not AGAINST systems, wouldn't I?

As for Rossini's arguments, that's like saying that men can't fly is a generalisation because I've not yet seen one who can. This is the point where I shall give up posting again and leave the 95% who fail at trading yet are convinced they are one of the other 5%.

You too will drop out soon...

I've spent thousands and thousands of hours back testing and developed several systems. I have also been a successful trader living well on the proceed for many years.

Good bye and good luck!

£10kLoser said:

OK, now I know why I gave up posting here a while ago.

If I was arguing from the point of view of a vendor of software, I would be arguing the case FOR not AGAINST systems, wouldn't I?

As for Rossini's arguments, that's like saying that men can't fly is a generalisation because I've not yet seen one who can. This is the point where I shall give up posting again and leave the 95% who fail at trading yet are convinced they are one of the other 5%.

But the one time you do see a man flying, it will invalidate all the other times where you haven’t. Black Swan and all that.

You too will drop out soon...

I won’t.

I've spent thousands and thousands of hours back testing and developed several systems. I have also been a successful trader living well on the proceed for many years.

Good bye and good luck!

Click to expand...

Same to you.

Backtesting is ESSENTIAL for any strategy.

However a successful backtest will only tell you that a strategy would have worked in the past. A backtest is important for that reason alone. If your backtest fails then you know you are dealing with a strategy that is likely not to work. This is why you should backtest - not to see how much money you think you'll make - but to verify that the strategy would have worked.

A successful backtest merely indicates that a strategy might be successful in the future.

It is always important never to over optimise - and bring up a graph of the returns - very often with over optimised systems you find 80-90% of the time the system loses gradually and a highly curve fitted portion at the end that gives the mega profits.

£10kLoser said:

If you believe that back testing has succeeding in producing a truly profitable system, then kindly reveal it to the rest of us! There have been hundreds if not thousands come and go and every one has failed to live up to its claims.

.

Click to expand...

£10k

Yes I can make that claim , I have found many profitable systems from backtesting .I normally run backtests on 8 years of 1 min data ,I also run months of forward tests.My systems are devised on the basis of historical performance in all kinds of market conditions.In designing systems I do not look for the most profitable system , but the most consistently reliable in all market conditions.

If everybody knew my system and the whole world started trading it, my system would fail .Simply every broker would get millions of bids at the same time leaving very little liquidity for myself to get a fill.By giving my better systems out I will be stabbing my own back.This is the main reason why all my main systems remain private.

Unfortunately this system will only work with directly with the banks on FX

OILDAYTRADER

Rossini said:

Originally Posted by £10kLoser
This is a valid point and another nail in the coffin for the farce of back testing.

No its a nail in the coffin for back testing with incorrect data, using a live accounts data from someone like FXCM is going to be much more accurate.

Click to expand...

The data you will get is the prices traded by some of these outfits.It is not the actual interbank rate or market rate.The outfits have certain leeway as to how many pips they can reach out for stops and fills, even though the interbank does not reach those prices.It is not going to be any more reliable.

The only reliable data I have heard of is on FXDD.

OILDAYTRADER

There is no free lunch when these metatrader bucketshops give you a free backtester.

The spreads on the backtester aren't real , because actual market spreads are much wider ,because of their own spreads added to prices.

http://www.trade2win.com/boards/metatrader/74696-metatrader-bucket-shops.html

They lie to you low spreads .fixed spreads and BS NO DEALING DESK.

You can't backtest execution delays,dirty tricks and non fills,effectively these backtests are worthless if trading live with a bucketshop

http://www.trade2win.com/boards/forex-brokers/75944-dealing-desk-brokers-avoid-them.html

oildaytrader said:

There is no free lunch when these metatrader bucketshops give you a free backtester.

The spreads on the backtester aren't real , because actual market spreads are much wider ,because of their own spreads added to prices.

http://www.trade2win.com/boards/metatrader/74696-metatrader-bucket-shops.html

They lie to you low spreads .fixed spreads and BS NO DEALING DESK.

You can't backtest execution delays,dirty tricks and non fills,effectively these backtests are worthless if trading live with a bucketshop

http://www.trade2win.com/boards/forex-brokers/75944-dealing-desk-brokers-avoid-them.html

Click to expand...

Whenever MT4 backetests a system it uses the current spread. The way the backtest works in terms of the spread cost is designed by Metaquotes and not set by the broker.

So you're saying backtests are worthless, yet you mention the benefit of back testing to find your profitable systems?

Jason Rogers said:

Whenever MT4 backetests a system it uses the current spread. The way the backtest works in terms of the spread cost is designed by Metaquotes and not set by the broker.

So you're saying backtests are worthless, yet you mention the benefit of back testing to find your profitable systems?

Click to expand...

The broker has no liquidity ,in other words he does not hold any currency to sell or buy.If broker is quoting a spread of 3 pips on cable ,he is quoting liquidity provider's spread.The broker goes to liquidity providers to fill orders.if customer places an order the broker has to add his own profit of 2 pips ,to the 3 pips quoted by liquidity provider,effectively the spread is 5 pips.The 3 spread used by tester does not include the spreads added by broker, it includes only the liquidity provider's quotes/spreads.

If a backtested system makes 2 pips per trade , is backtested without the broker's spread of 2 pips,it will produce zero profits i.e 2 pips profit less 2 pip hidden spread = zero profits in real accounts.

The backtests are worthless if trader is going to trade with the broker .They are not worthless if trader goes directly to the liquidity provider, because trader will avoid the hidden spread of broker.

The only advantage of backtests is to see how well the system performed in ideal execution conditions.My systems would be much more profitable if traded directly in interbank market.They would perform as well as they did in backtests,provided they are traded in interbank markets.

O D T

Even the optimisation sucks

http://www.trade2win.com/boards/mec...s-vs-forwardtesting-results-3.html#post357434