Most back testing is useless

foroom lluzers · Apr 17, 2017

If back testing on excel ,manual back testing or using automated software , then applying it on manual trading ,it will be totally inefficient back testing.This is because backtesting has no emotions , stress or any phsychological handicaps being back tested .Backtesting in such an inappropriate environment without emotions , stress or any phsychological handicaps , for which it will not be used , is useless and the dumb trading industry does it .

Results are much worse on actual trading .Most people can not replicate demo results or back testing results on live accounts.On live accounts ,trading psychology , execution liquidity and human commands element is involved

Traders have zero edge , unless you have a mental edge .All your illusionary edges are worth zero , if you can't execute your edge.

https://www.youtube.com/watch?v=GhKJ9P3agRc&feature=youtu.be

tomorton · Apr 17, 2017

Agreed. Back-testing is like drawing a racing car, then asking for the F1 title without even building and driving it.

edmond360 · Apr 17, 2017

This concept of back testing and probability is barely beyond school-level statistics. The other concept of psychology is even worse, how the hell can your psychology make you money? It obviously can't, unless you believe in magic. The reason brokers and salesman are continually looking to drum these concepts into people's heads is because it makes them a profit over a long period of time in which the trader can't tell if it's working or not working... because it's random, it's sequences, stick at it, don't give up, your edge takes time to play out, keep trading, keep pushing that button, you don't need to predict the price, don't worry about anything just stick to the process, follow the rules, follow follow follow, and on and on you go. It's utter nonsense.

Only those traders who do lots of other analysis and understand the markets and understand economics, they can get away with school-level statistics because it's just the cherry on the cake for them. For everyone else, it's not enough to just use a systematic pattern or process. How can it be? Common sense alone should tell you it's not that easy. Unless you do all the hard work which banks pay their analysts to do, then you're not going to get very far with nothing but blind faith in some statistical edge you think you have uncovered.

Just look at the quantitative firms who also rely upon nothing but statistics, they only recruit people with PhD's. Why? They would only do it if financial data was a little more difficult to analyse than just counting up the outcomes of a coin-flip.

Solas0077 · Apr 17, 2017

Backtesting is extremely useful. There is no quant trading without backtesting. But you have to pay attention to experts apart from your own conclusions. Backtesting will never tell you anything beyond which strategy failed or won in the past. Making inferences about the future requires advance level of statistics for establishing whether results are due to data-mining bias. I have learned a lot from some experts. Below are two must read articles IMO for understanding this complex subject

Inferences From Backtest Results Are False Until Proven True

The Probability of Backtest Overfitting

Instead of complaining about backtesting maybe it is better to try to master it. There is plenty of material around for someone who is willing to learn and be successful.

foroom lluzers · Apr 17, 2017

I never said that "all back testing is useless ".You can find edges with back testing .

Most of it is useless , if you can not execute it , if trader does not have mental skills.Executing is where psychology comes in , without mental skills to execute the edges , the back testing exercise is useless .

I am sure this will upset many vendors associated with selling software and services related to backtesting.

dbphoenix · Apr 17, 2017

tomorton said:
Agreed. Back-testing is like drawing a racing car, then asking for the F1 title without even building and driving it.

I'm surprised you'd say something like this. Back-testing is followed by forward-testing followed by the evaluation of results and necessary modification of protocols.

Does no one understand the scientific method anymore?

edmond360 · Apr 17, 2017

dbphoenix said:
I'm surprised you'd say something like this. Back-testing is followed by forward-testing followed by the evaluation of results and necessary modification of protocols.

Does no one understand the scientific method anymore?

Uh that absolutely is not the scientific method. Not unless you first have a hypothesis which says this particular set of protocols should work, and then predict how well they should work, and then try to falsify your prediction using back testing.

Starting out with 1000 variations of a protocol. Finding 100 that "work". Then doing a forward test and reducing it down to 10 that "work", is more accurately known as the stupidific method.

foroom lluzers · Apr 17, 2017

Here is bexcel back testing showing an edge ,so I am not against back testing.

dbphoenix · Apr 17, 2017

edmond360 said:
Uh that absolutely is not the scientific method. Not unless you first have a hypothesis which says this particular set of protocols should work, and then predict how well they should work, and then try to falsify your prediction using back testing.

Starting out with 1000 variations of a protocol. Finding 100 that "work". Then doing a forward test and reducing it down to 10 that "work", is more accurately known as the stupidific method.

The hypothesis is formulated through observation, then back-testing to determine if it "works" in hindsight. Then it's tested forward to determine whether or not it works in foresight. This is of course not the same protocol as would be used in testing a drug, but it works for testing trading protocols.

edmond360 · Apr 17, 2017

dbphoenix said:
The hypothesis is formulated through observation, then back-testing to determine if it "works" in hindsight. Then it's tested forward to determine whether or not it works in foresight. This is of course not the same protocol as would be used in testing a drug, but it works for testing trading protocols.

The big players who buy and sell financial assets are doing so for ever-changing reasons, but they're obviously not doing it randomly. It makes little sense to me to look through the charts or price data to find some behaviour or weird pattern which correlates with those ever-changing reasons, and then have to do more work to make sure it's not just a spurious correlation. It is like trying to study animal behaviour from the tiny markings left in its bones, I get it when you have no choice, but in trading you have a living specimen out in the real world right there for you to go look at and ask questions to.

Unless you really really get how to accurately model financial data using mathematics, then you're unlikely to make any money from a back-tested / forward-tested protocol. The problem today is that quants are more likely to be computer scientists rather than mathematicians or engineers.

seekingTruth16 · Apr 17, 2017

dbphoenix said:
The hypothesis is formulated through observation, then back-testing to determine if it "works" in hindsight. Then it's tested forward to determine whether or not it works in foresight. This is of course not the same protocol as would be used in testing a drug, but it works for testing trading protocols.

It's strange that anyone could argue against what you have said. Obviously any form of testing is better than none, and back-testing followed by forward testing is likely to be of value and very unlikely to cause harm. What do you lose from doing it?

Does it get resistance because it sounds like too much work? Unless the point is that a lot of back-testing time will end up with no usable strategy. But that's better than losing money on an untested system.

Those I know in the industry wouldn't dream of putting money on something that wasn't back-tested (out of sample) and forward tested, and that has nothing to do with whether the methods are highly quantitative or not.

dbphoenix · Apr 17, 2017

edmond360 said:
The big players who buy and sell financial assets are doing so for ever-changing reasons, but they're obviously not doing it randomly. It makes little sense to me to look through the charts or price data to find some behaviour or weird pattern which correlates with those ever-changing reasons, and then have to do more work to make sure it's not just a spurious correlation. It is like trying to study animal behaviour from the tiny markings left in its bones, I get it when you have no choice, but in trading you have a living specimen out in the real world right there for you to go look at and ask questions to.

Unless you really really get how to accurately model financial data using mathematics, then you're unlikely to make any money from a back-tested / forward-tested protocol. The problem today is that quants are more likely to be computer scientists rather than mathematicians or engineers.

If you want more detail regarding what I do and how I do it, you're welcome to read my threads. But I see no point in doing a lot of copy-and-paste. Those who have read them will understand that the macro reasons never change. Though those who have no interest in trading price will not have been and will not be interested in them at all.

Which is fine.

dbphoenix · Apr 17, 2017

seekingTruth16 said:
It's strange that anyone could argue against what you have said. Obviously any form of testing is better than none, and back-testing followed by forward testing is likely to be of value and very unlikely to cause harm. What do you lose from doing it?

Does it get resistance because it sounds like too much work? Unless the point is that a lot of back-testing time will end up with no usable strategy. But that's better than losing money on an untested system.

Those I know in the industry wouldn't dream of putting money on something that wasn't back-tested (out of sample) and forward tested, and that has nothing to do with whether the methods are highly quantitative or not.

That's why I haven't talked about it further until the last few days (and only then because I want to do what I can to persuade new traders that there's more to this than just buying somebody's software). But, yes, it is too much work for most. New traders aren't attracted to this because of the work that's involved; they're attracted because of all the easy money

that's out there.

sigmund1 · Apr 17, 2017

Financial Psychology

It make no difference.

No difference at all.

It just doesn't matter if you believe in financial psychology or you don't.

Stupidity trumps common sense every time.

Gotta go, our Nan's just brewed up a fresh pot of Kopi Luwak coffee, just love that aroma.

foroom lluzers · Apr 17, 2017

Humans make mistakes , there are over 40 different trading mistakes , traders on live accounts will make a lot of mistakes.These result in losses between back tests and actual results and actual performances are much lower.

Just search hard on google and you will find all the 40 plus mistakes made by traders.

seekingTruth16 · Apr 17, 2017

foroom lluzers said:
Humans make mistakes , there are over 40 different trading mistakes , traders on live accounts will make a lot of mistakes.These result in losses between back tests and actual results and actual performances are much lower.

Just search hard on google and you will find all the 40 plus mistakes made by traders.

Is 'not back-testing' one of the 40?

kalott · Apr 17, 2017

foroom lluzers said:
If back testing on excel ,manual back testing or using automated software , then applying it on manual trading ,it will be totally inefficient back testing.This is because backtesting has no emotions

Results are much worse on actual trading .Most people can not replicate demo results or back testing results on live accounts.

if you can use a computer to backtest it you could probably use the computer to trade it and read a book about trading meanwhile

foroom lluzers · Apr 18, 2017

seekingTruth16 said:
Is 'not back-testing' one of the 40?

I am surprised this information is not even on the forums , the list of mistakes .
Van Tharp has a video on youtube about traders and mistakes.I suppose the idea is to keep everyone within their prisons of delusions.

Oh and V K sells courses only to real traders for 6 figures.

foroom lluzers · Apr 18, 2017

kalott said:
if you can use a computer to backtest it you could probably use the computer to trade it and read a book about trading meanwhile

It really is not that easy , grass is always green on the other side .Market is too intelligent /efficient to give you an edge , it is a zero sum game.see image ....it says it all

One of the more earlier wise posters posted about the difficulties for retail traders.He is telling the truth but nobody accepts his wisdom .

foroom lluzers · Apr 18, 2017

edmond360 said:
This concept of back testing and probability is barely beyond school-level statistics. The other concept of psychology is even worse, how the hell can your psychology make you money? It obviously can't, unless you believe in magic. The reason brokers and salesman are continually looking to drum these concepts into people's heads is because it makes them a profit over a long period of time in which the trader can't tell if it's working or not working... because it's random, it's sequences, stick at it, don't give up, your edge takes time to play out, keep trading, keep pushing that button, you don't need to predict the price, don't worry about anything just stick to the process, follow the rules, follow follow follow, and on and on you go. It's utter nonsense.

Only those traders who do lots of other analysis and understand the markets and understand economics, they can get away with school-level statistics because it's just the cherry on the cake for them. For everyone else, it's not enough to just use a systematic pattern or process. How can it be? Common sense alone should tell you it's not that easy. Unless you do all the hard work which banks pay their analysts to do, then you're not going to get very far with nothing but blind faith in some statistical edge you think you have uncovered.

Just look at the quantitative firms who also rely upon nothing but statistics, they only recruit people with PhD's. Why? They would only do it if financial data was a little more difficult to analyse than just counting up the outcomes of a coin-flip.

Your post is great about the small guy.

There is something we do not have , there may be automated fron running software , used by the big boys.It may have been developed by far more knowledge than we have access to.

Most back testing is useless

Veteren member

Guest Author

Junior member

Active member

Veteren member

Guest Author

Junior member

Veteren member

Attachments

Guest Author

Junior member

Active member

Guest Author

Guest Author

Active member

Veteren member

Active member

Established member

Veteren member

Veteren member

Attachments

Veteren member

Similar threads