Most back testing is useless

[piphoe]

I found my attempts at backtesting were useful at showing all my implicit assumptions about my trade. Discovered I couldn't define all my "rulz" or reasons explicitly. That told me I didn't really know why I took that tarde. Some of my strategy remains vague and undefined to this very day. I gave up on backtesting, for now.:innocent:

The best computer is that neural net right between your ears.

 

[foroom lluzers]

“Insanity is doing the same thing, over and over again, but expecting different results.” “The reason I talk to myself is because I'm the only one whose answers I accept.”

market timing does not work , Warren buffet does not try to time the markets , why are these lunatics promoting back testing software and vendors?“Insanity is doing the same thing, over and over again, but expecting different results.”

http://www.trade2win.com/boards/edu...s-when-you-try-time-market-6.html#post2897216

 

[foroom lluzers]

Price action doesn’t replicate itself over time

First of all, market prices are chaotic, and. The patterns observed on any day’s trading do not replicate themselves at other times. As a result, the attempt at testing today’s market action on the basis of yesterday’s patterns will never lead to results that can be duplicated over the long term.

A trader lost his entire account , after market volatility increased 200 % and back tested patterns changed.

 

[seekingTruth16]

Price action doesn’t replicate itself over time

First of all, market prices are chaotic, and. The patterns observed on any day’s trading do not replicate themselves at other times. As a result, the attempt at testing today’s market action on the basis of yesterday’s patterns will never lead to results that can be duplicated over the long term.

A trader lost his entire account , after market volatility increased 200 % and back tested patterns changed.

Consider the open of a futures market, the lunch time period, the close and consider the time around a major news release. Do you think that the price behaves basically the same for each of these, and that if you did some statistical analysis, you'd find no difference?

If there's a difference, then that's a replicating pattern. Do interest rates go down for a long period, and then they go up for long period, or is it usually up down up down. Are they mean-reverting? Are there any other things that might have some pattern...time of year, central bank intervention?

 

[foroom lluzers]

Consider the open of a futures market, the lunch time period, the close and consider the time around a major news release. Do you think that the price behaves basically the same for each of these, and that if you did some statistical analysis, you'd find no difference?

If there's a difference, then that's a replicating pattern. Do interest rates go down for a long period, and then they go up for long period, or is it usually up down up down. Are they mean-reverting? Are there any other things that might have some pattern...time of year, central bank intervention?

This sort of information is for getting gambler bias , leading to trading losses.
When we gamble, we constantly process information (often unconsciously) in a consistently biased way. Humans tend to exhibit consistent biases when cognitively processing information in gambling situations.


http://www.trade2win.com/boards/gen...fet-suceeded-95-traders-fail.html#post2902728

 

[foroom lluzers]

Most back testing would not have predicted Dax is going to this level.Only hindsight results trades can tell this.

 

[Solas0077]

Must read. Explains the backtesting conundrum well. https://medium.com/towards-data-science/the-paradox-of-the-elusive-market-edge-5e6b57b04398

 

[foroom lluzers]

Must read. Explains the backtesting conundrum well. https://medium.com/towards-data-science/the-paradox-of-the-elusive-market-edge-5e6b57b04398

markets , volatility and fundamentals change.These unique back testing edge opportunities of profitability are all good for hindsight trading , if unrepeatable market edge opportunities repeat themselves.

Forward testing is much better.

 

[NVP]

I'm surprised you'd say something like this. Back-testing is followed by forward-testing followed by the evaluation of results and necessary modification of protocols.

Does no one understand the scientific method anymore?

no they dont .......

 

[NVP]

markets , volatility and fundamentals change.These unique back testing edge opportunities of profitability are all good for hindsight trading , if unrepeatable market edge opportunities repeat themselves.

Forward testing is much better.

throw your proposed systems into a historic chop period......nothing beats it :smart:

 

[foroom lluzers]

throw your proposed systems into a historic chop period......nothing beats it :smart:

Exactly !

Put your system method with fundamental drivers of price , and every system will work.!

 

[tomorton]

Most back testing would not have predicted Dax is going to this level.Only hindsight results trades can tell this.

There is no need for a system to know how high something going up will go over 5 years. Or even a month. The fact that any chosen TA-based system won't identify the final all-time high doesn't mean it can't be used to repeatedly make a profit going long.

 

[tomorton]

markets , volatility and fundamentals change.These unique back testing edge opportunities of profitability are all good for hindsight trading , if unrepeatable market edge opportunities repeat themselves.

Forward testing is much better.

If fundamentals change it shows up in price. If fundamentals change but price doesn't, they were mostly irrelevant in the first place. I say "mostly" rather than completely as it can be useful for some traders to know fundamentals have changed, in case price does indeed "catch up", though my opinion is this simply reflects a poorly devised trade plan.

On a broader note, I hear lots of old hands saying markets change, or markets have changed since then and that's why this or that system doesn't work any more. Or markets are going to change, and that will stop this or that system working. WTF are they talking about?

 

[Solas0077]

Uh that absolutely is not the scientific method. Not unless you first have a hypothesis which says this particular set of protocols should work, and then predict how well they should work, and then try to falsify your prediction using back testing.

Starting out with 1000 variations of a protocol. Finding 100 that "work". Then doing a forward test and reducing it down to 10 that "work", is more accurately known as the stupidific method.

Time wasting and expensive too. Many got fired in hedge funds trying to do exactly this. This relies on luck and exactly what should not be doing. Excellent post.

 

[Solas0077]

If fundamentals change it shows up in price. If fundamentals change but price doesn't, they were mostly irrelevant in the first place. I say "mostly" rather than completely as it can be useful for some traders to know fundamentals have changed, in case price does indeed "catch up", though my opinion is this simply reflects a poorly devised trade plan.

On a broader note, I hear lots of old hands saying markets change, or markets have changed since then and that's why this or that system doesn't work any more. Or markets are going to change, and that will stop this or that system working. WTF are they talking about?

Not only fundamentals but technicals change in an important way. In a sense the subset of TA that works at any given time changes due to changing markets.

BTW if someone needs an operation - God forbid - should he chose a young or an old hand? When I see young guys on financial TV I change channel. They sound smart but they have no experience. Soros is abouts 85, would you listen to him or to a 25 year old from Yale?

 

[Windlesham1]

Markets change of course they do-the many varied outside influences are in a state of flux-governments change, banksters run the world. There's the old chestnut- the good news is your stock's fundamentals are still as good,the bad news, the stock price just dropped 90%. Markets rise on lies and sell off on the truth. Liars and psychopaths gravitate to the world of money, it's a whole lot easier than bashing rocks for a living.

 

[tomorton]

Not only fundamentals but technicals change in an important way. In a sense the subset of TA that works at any given time changes due to changing markets.

BTW if someone needs an operation - God forbid - should he chose a young or an old hand? When I see young guys on financial TV I change channel. They sound smart but they have no experience. Soros is abouts 85, would you listen to him or to a 25 year old from Yale?

Yes but what is a changing market?

 

[foroom lluzers]

Yes but what is a changing market?

In the past markets , markets used to fear higher interest rates , nowadays prices keep going up with Fed puts on stocks and the trump rally .They are not the same anymore.

https://www.bloomberg.com/news/vide...oss-buy-what-the-central-banks-haven-t-bought

http://time.com/money/4437099/bill-gross-stocks-bonds/

It seems like the governments never want to raise rates.

 

[tomorton]

In the past markets , markets used to fear higher interest rates , nowadays prices keep going up with Fed puts on stocks and the trump rally .They are not the same anymore.

https://www.bloomberg.com/news/vide...oss-buy-what-the-central-banks-haven-t-bought

http://time.com/money/4437099/bill-gross-stocks-bonds/

It seems like the governments never want to raise rates.

So you don't say the TA has changed but FA is disconnected from price? So the TA-ers are right and the FA-ers are wrong.

 

[foroom lluzers]

So you don't say the TA has changed but FA is disconnected from price? So the TA-ers are right and the FA-ers are wrong.

TA is disconnected from previous behaviour , because for the same entry of several years ago , the markets are now behaving differently to the same t/a set up.

The thread argument is backtesting is useless , due to non testing of psyche.This makes use historic t/a more complex.