Trading Probabilities Using Bayesian Theorem

[bearwatch]

Hi all,

I've done some research about probabilities and came up with this very interesting concept, finding historical/prior outcomes/knowledge and use it to evaluate the current market action to take the appropriate move, long, short, or flat, be it an entry/exit type for a particular market condition.

Does anyone have experience in this area and be willing to share as I'm interested in develop this idea further to enhance my systems. Thanks.

 

[TWI]

I did try to understand a book I had on Bayesian Analysis of Time Series Data but it only worked for the first 20 pages before it went a little above my head with the Maths ( I have a degree in Maths & Physics so it isn't like I cannot understand basic stuff). Certainly sounds interesting. I have found a lot of interesting statistical ideas however and unfortunately been unable to implement them in my trading models due to my mathematical limitations. I think this is where funds like Medallion score because they only employ Maths and Science Phd's who have a relevant thesis...It has to work because these guys crank out ~35% average/annum.
Please let me know if you crack it.

 

[SOCRATES]

...It has to work because these guys crank out ~35% average/annum.
Please let me know if you crack it.

If these guys are able to do this, it presents a dilemma, being to disseminate or not to disseminate, to openly discuss or not to discuss it at all.

 

[TraderPattern]

Excellent mathematical analysis is surely a good thing when it comes to trading. The results of the Baysiens Theories are evidently most encouraging. Thank you for sharing.

 

[SOCRATES]

Excellent mathematical analysis is surely a good thing when it comes to trading. The results of the Baysiens Theories are evidently most encouraging. Thank you for sharing.

Yes, agreed, but the problem with this stuff is that it is flat, whereas what is needed has more than one dimension, like interwining dna coils, if only to illustrate an idea..

 

[TheBramble]

bearwatch - I don't know to what 'results of Bayesian Theories' (sic) another member refers as certainly none have been offered here. Nor am I aware from my research of any in relation to trading which could in any way be considered 'encouraging'. But in my limited understanding of Bayes' theorem (which by the way measures correlation, not causation it is IMLTHO a blind alley. If you're interested, as I was, in this area, then you might want to pick up with Laplace who expanded on Bayes initial thoughts - quite independently it is thought.

The point where I let the Rev. Bayes go was when I realised the subjective nature of his ideas made it possibly not the best use of my research time. Note "it is not inconsistent for different persons to assign different Bayesian probabilities to the same proposition".

Sounds like the markets to me...

 

[bearwatch]

To start, I'm new to Bayes' concept so any input I may have fallacies so correct or point them out as we go along.

According to the theory, we start with a belief, but we want to make concrete by calculating them into #s. Basically, the gray area instead of the black and white. This is fair assessment of the market because everyone has beliefs but no one can put their finger on it and say "my belief has 50% chance of sucess".

Let's put this in a trading example. Say, we believe that up gaps tend to stay up in bull markets than bear markets. This is just a belief or hypothesis. So we can go and find out the probabilities of this is true and probabilities that it is false. Once we have this prior data/probabilities, we incorporate the new information in to form a new probability of success or failure.

That's the first part, the 2nd part. An example. Say in this same scenario, which entry type do we favor taking, buy at the open or wait for the gap to close to buy? Model comparison of this Bayes' calculation is what interests me because the first example is similar to other math work, count the (+)s and count the (-)s and you have your probability of the price continue going forward or stop and reverse in a bull or bear market. Based on the stats from the first example, we can determine which type of entry to make. This is similar to understanding what strategy do we use in what market condition.

The idea is to drill down in creating probabilities in many scenarios to determine what's the best action to take. I think this is where Bayesian Networks comes into play.

Am I explaining clearly or am I babbling?

 

[SOCRATES]

To start, I'm new to Bayes' concept so any input I may have fallacies so correct or point them out as we go along.

According to the theory, we start with a belief, but we want to make concrete by calculating them into #s. Basically, the gray area instead of the black and white. This is fair assessment of the market because everyone has beliefs but no one can put their finger on it and say "my belief has 50% chance of sucess".

Let's put this in a trading example. Say, we believe that up gaps tend to stay up in bull markets than bear markets. This is just a belief or hypothesis. So we can go and find out the probabilities of this is true and probabilities that it is false. Once we have this prior data/probabilities, we incorporate the new information in to form a new probability of success or failure.

That's the first part, the 2nd part. An example. Say in this same scenario, which entry type do we favor taking, buy at the open or wait for the gap to close to buy? Model comparison of this Bayes' calculation is what interests me because the first example is similar to other math work, count the (+)s and count the (-)s and you have your probability of the price continue going forward or stop and reverse in a bull or bear market. Based on the stats from the first example, we can determine which type of entry to make. This is similar to understanding what strategy do we use in what market condition.

The idea is to drill down in creating probabilities in many scenarios to determine what's the best action to take. I think this is where Bayesian Networks comes into play.

Am I explaining clearly or am I babbling?

No you are not blabbing, but you are not makig it clear either. But you are doing one thing, and that is to try to map in flat form what is not flat, but multidimensional, and additionally a thng that keeps on changing, like a living brathing entity, and that is the problem. I can percieve what it is you are trying to achieve, but it is the wrong mapping for the reasons above, and very probably of limited use as such.

 

[bearwatch]

I have seen one trader made a successful system out this method and have seen his equity curve. Can't prove it but his detailed explanation made a good impression on me. Well, no harm in finding out.

Socrates, how you do approach building successful mechanical systems? Do you probabilities in your logic? I can't say I'm a guru of mechanical system myself but would like to improve my game.

 

[TheBramble]

I have seen one trader made a successful system out this method and have seen his equity curve. Can't prove it but his detailed explanation made a good impression on me. Well, no harm in finding out.

If someone has made a successful system out of Bayes they most definitely bought something to the party themselves.

If you have the ability to do so, I suggest you ask this trader to tutor you or mentor you on his approach and methods. He is unlikely to be offended as he has gone some way it would seem to encourage you in this respect.

Of course, solitary original research on your part will yield benefits all of their own as well.

 

[SOCRATES]

I have seen one trader made a successful system out this method and have seen his equity curve. Can't prove it but his detailed explanation made a good impression on me. Well, no harm in finding out.

Socrates, how you do approach building successful mechanical systems? Do you probabilities in your logic? I can't say I'm a guru of mechanical system myself but would like to improve my game.

The vast majority of constructors of successful mechanical systems are driven by commercial considerations, that is to construct, improve, upgrade, and sell to the general public.

This is because the cost of reliable architecture can prove to be very expensive in terms of time, research, overcoming unforseen technical obstacles, programming, then all the marketing costs that follow, then the costs of further upgrading, technical support, etc., therefore the vast majority of constructors seek to recoup these costs by following the commercial route and making the final product freely available retail, and ultimately to make a profit out of the distribution of the package via commercial channels.

That is one route. The other route is to develop by research, verification, programming and to absorb all the costs by offsetting these developing costs to finality against trading profits, thus retaining total control of the end package.

The end package then becomes an exclusive construct, because it is not readily available commercially. That is, it is not available to the general public and what is more, its parameters, functionality, etc., are kept out of the public domain, This gives it added value, because the edge imparted by the package does not become proliferated and consequently weakened. In both cases we are talking about mechanical systems only.

Then as inevitably the package outlives its useful life, because it has built in obsolescence as a consequence of the objective not being mechanical but methodological. When a methodology is refined to replace the package it is then disposed of, usually by private treaty to an institutional counterparty still at the mechanical development stage of the idea or one seeking a mechanical solution to slot in with its needs.

You can see that the object is to use a package to illustrate a system in tangible form, to progress the idea to a method, which is an intangible form. Therefore in reply to your question, the requirement is to approach building successful mechanical systems holistically, by having an accurate overview in advance of construction as to what is required, with the ultimate intent of using the result (a successful mechanical result ) as a stepping stone to perfecting and refining edges within a method. Then the mechanical system becomes obsolete.

And, in reply to your second question, the more able the package is capable of mirroring probabilities realistically within accurate logical constraints, the more efficient it will be, and ironically, the faster it will become obsolete, and consequently disposed of.

Then ultimately what remains is the method. This is the ultimate objective and the most important, because thought is everything.

I hope and expect the above explanations satisfy your query.

 

[bearwatch]

As you can see my avatar, I look like that in real life-- a slow learner so I'll have to reread a few times and look up a few words to let it sink in. :cheesy: Thanks for the explanation.

What you're saying is no mechanical system will last correct? Constant revision, logic update and optimization is required in all methodologies. In a sense, even the Bayesian approach will not save it from test of time.

Isn't there a belief that the market plays itself out over and over again in a different but similar fashion, ie. "the more things change the more they stay the same"? Bayesian approach is in itself a self-contained and evolving dynamic structure. All and new info will modify itself to approach the market as it sees fit, that's what AI does-- constant learning and correcting of its behavior toward the outside environment. Isn't that we all do... evolve and learn and adapt with new information and input everyday? Isn't that how successful discretionary traders survive over time continually learn and adapt and thrive in new market conditions? I'm not saying machines have yet the capacity as humans to learn and adapt and survive but it is possible right?

 

[SOCRATES]

As you can see my avatar, I look like that in real life-- a slow learner so I'll have to reread a few times and look up a few words to let it sink in. :cheesy: Thanks for the explanation.

What you're saying is no mechanical system will last correct? Constant revision, logic update and optimization is required in all methodologies. In a sense, even the Bayesian approach will not save it from test of time.

Isn't there a belief that the market plays itself out over and over again in a different but similar fashion, ie. "the more things change the more they stay the same"? Bayesian approach is in itself a self-contained and evolving dynamic structure. All and new info will modify itself to approach the market as it sees fit, that's what AI does-- constant learning and correcting of its behavior toward the outside environment. Isn't that we all do... evolve and learn and adapt with new information and input everyday? Isn't that how successful discretionary traders survive over time continually learn and adapt and thrive in new market conditions? I'm not saying machines have yet the capacity as humans to learn and adapt and survive but it is possible right?

Yes, there is reading.....and....reading.... in the same way that there is seeing or looking....and observing. They appear to be the same thing but are very different. This is because the process is invisible to an observer observing someone doing it.

Doing this not only leads to understanding; it also serves another purpose which is similar to reverse engineering. It ultimately leads or can lead to for want of a better word, evolvement.

In this process of evolvement better ideas are replaced by even better ideas, until finally the best ideas possible make themselves manifest. It is then a question of tweaking here and there to adjust what is nearly perfect, to make it as perfect as is perfectly possible.

Machines, in the contemporary sense of the word, cannot do this. This is because machines have the missing ingredient of natural intelligence. They are constructed to fulfil functions.
They have not been evolved sufficiently for them to entrust themselves to fulfil functions of their choice without any supervision, because in the strictest sense of the word, they are dependent, and hence not naturally intelligent as such.

 

[bearwatch]

Using fuzzy logic sold by all these commercial vendors do not work?

 

[jmreeve]

I did try to understand a book I had on Bayesian Analysis of Time Series Data but it only worked for the first 20 pages before it went a little above my head with the Maths ( I have a degree in Maths & Physics so it isn't like I cannot understand basic stuff). Certainly sounds interesting. I have found a lot of interesting statistical ideas however and unfortunately been unable to implement them in my trading models due to my mathematical limitations. I think this is where funds like Medallion score because they only employ Maths and Science Phd's who have a relevant thesis...It has to work because these guys crank out ~35% average/annum.
Please let me know if you crack it.

Which book did you try to read?
A lot of text book maths is very poorly presented. Have found that were maths is presented logically and well explained it is usually a sign the author has a good understanding of the material. Most maths can be worked through with determination and an simulation package but the advantage of being a PhD genuis is that you don't have to try.

 

[SOCRATES]

Using fuzzy logic sold by all these commercial vendors do not work?

The only logic that works is that inside your head, provided it is the correct type of logic.

 

[ZDO]

What you're saying is no mechanical system will last correct? Constant revision, logic update and optimization is required in all methodologies. In a sense, even the Bayesian approach will not save it from test of time.

.. Bayesian approach is in itself a self-contained and evolving dynamic structure. All and new info will modify itself to approach the market as it sees fit, that's what AI does-- constant learning and correcting of its behavior toward the outside environment. Isn't that we all do... evolve and learn and adapt with new information and input everyday? Isn't that how successful discretionary traders survive over time continually learn and adapt and thrive in new market conditions? I'm not saying machines have yet the capacity as humans to learn and adapt and survive but it is possible right?

The problem with applying nearly all ‘statistics’ to trading is that most of us have no reliable tools to accommodate changing distributions - distributions that are making quantum rather than gradual changes! Every edge incorporated into a mechanical system will fade AND every edge incorporated into a mechanical system will come back (not necessarily within the same trading career ). Hence, “every system works some of the time” zdo. [see http://www.trade2win.com/boards/showthread.php?t=18143 ]. An edge that is going recessive means that another is waking up… However difficult, rotating through edges is far more effective than tweaking parameters of an edge (by fuzzy or whatever means). Even dormant, an edge is still an edge and once discovered, should not be discarded as useless because it is not the best tool for the current campaign. It’s not nearly so simple and obvious in trading, but it’s the same as taking a gun hunting and a pole fishing and not vice versa. Applying this - the best, not necessarily the luckiest, funds have at the top managers who can give apt weighting to multiple money managers, who then each carefully applies a single methodology. Individuals, with rare exceptions, are prohibited (even genetically?) from being able to concurrently apply and dynamically adjust across multiple methodologies. Further, imo, humans cannot ‘do’ / ‘be’ bayensian consistently. However it is feasible for an individual trader to successfully rotate serially through single methods, improving in each cycle (as Socrates alluded to with 'evolvement'). And, it is possible to concurrently implement a ‘portfolio’ of mechanical systems, each acting as a proxy for a methodology.

Whoosh …

zdo

 

[SOCRATES]

Whoosh...

Spot on.... My compliments to you ZDO.

 

[bearwatch]

ZDO,

What you're saying using 1 single trading system despite dynamic params changing is not optimal compared to a portfolio of strategies correct? I agree completely but eventually all strategy need change from time to time this is the reason I'm looking into this area.

Thanks for the great input!

 

[ZDO]

bearwatch,

re: “What you're saying using 1 single trading system despite dynamic params changing is not optimal compared to a portfolio of strategies correct?”
A portfolio of systems is not much better than a single (robust or not) system unless one has good ways of dynamically adjusting the relative weight given to each system.

-but-

One single trading system locks one to a certain level of knowingness / assumptions about the ‘structure’ AND locks one to a certain level of ‘observability’ / locks one into a discrete limited set of nodes. Does that sound like it would work ubiquituously in the market(s) you perceive / experience / trade? Bottom line -‘Tweaking’ parameters is best for those models where the structure is known and we also have full observability of the ‘nodes’. Does that sound anything like the market(s) you experience?

Multiple systems gives you some travel across several possible ‘structures’ AND ALSO gives you some breathing space to use different observable nodes / variables and for tapping to / through to various hidden nodes. (btw, system designers beware - add one hidden node and one is no longer working with the one original trading system.)

Why do you say “all strategy need change from time to time” ?

zdo