my journal 3

[Yamato]

The message box, both in "advanced" and "quick reply" modes, is not working properly on internet explorer. Fifty percent of what I type doesn't show, every other key I press doesn't produce a character on the message box. It's very frustrating. Instead it works on opera, firefox, chrome. Also, I checked if similar message boxes work on yahoo and google mail and they do. The only thing left to do is if the message box works on internet explorer in another computer, which would surprise me, because it would mean I have a problem on trade2win's message boxes only on this computer and only on internet explorer. Which would make me reinstall internet explorer, which is a pain in the ass. Let's see.

Ok, I just checked and damn me. It works perfectly fine. So I basically screwed up something on internet explorer that only affects this box on this web site. I can't even double-check by registering another t2w account, because I might get banned, like last time.

Ok, let's try to reset internet explorer's settings to default first, before having to reinstall it.



Ok, done. Let's see if it works now.

Damn. Still not working.

Now I will reboot my computer, and if it still doesn't work, the next step is to try system restore, to yesterday or the day before.

Ok, rebooting doesn't solve it either. Now I will try a system restore to 2 days ago.

I have to remember I just had an idea about starting the systems after a loss. It might not help at all, and it might be incoherent probability thinking: I know it doesn't make a difference on random events, such as dice and the roulette. But, hey, even on those betting on black after a red came out does not hurt. If anything it leaves your chances of getting it right to the original 50%.

And, if instead there's some probabilistic reason that enabling those seven systems one at a time, each one after a big loss/drawdown, puts the probability on my side, then why not do it? Being wrong about this hypothesis won't hurt me for sure, because if it's totally random like for coin tossing and dice and roulette, then I won't affect the next outcome either by waiting one outcome. If instead I am right and it's not random, and after a loss or big drawdown, it is less likely to happen another one, then I will benefit from it. So now my idea is to enable those 7 systems, but one at a time and each one after a loss or drawdown. If I am wrong, it won't hurt, and if I am right, I will benefit from it.

So now i'll do the system restore.

Oh, good. Now it finally all works. As to the culprit, the only two possible causes are:

1) I installed hotspot shield, a proxy software, in order to watch some SNL episodes on hulu.com and vidcube.com, which would not stream to italy, but then, once i did that, it was useless, because they only streamed for free for 1 minute and a half. Anyway, I later found out that for most episodes the videos can be found at good places and not just these two paying websites. So this is the link anyway, if anyone cares:
Watch Saturday Night Live online (TV Show) - on 1Channel | LetMeWatchThis

2) I was trying to get rid of google search autocomplete feature, and i followed the instructions in this video:

Google - How to turn off Google autocomplete - YouTube


But then today the mother ****ing autocomplete (google is getting worse, just like windows, office and a lot of other software lately) came back anyway, and maybe it also caused, out of revenge, this problem on trade2win's message boxes.

Yesterday, among the other things, while browsing on letmewatchthis.com, which is my favorite and only movie website, I discovered this very good tv show:
Watch Mob Wives online (TV Show) - download MobWives - on 1Channel | LetMeWatchThis

Mob wives is quality stuff. I haven't reasoned on why yet, but it's very engrossing. First of all, I think it's well-made (good editing of the interesting parts). And second of all, I am Italian, lived in the states for a few years, so this is related to things I both know and also am curious about. But then there must be other reasons. Some brainstorming, in random order:

1) there's action, because the husbands are in jail or go to jail during the movie
2) among my favorite movies are mafia movies such as goodfellas (the mob wives are just like ray liotta's friends and wife in that movie), the godfather, and so on.
3) I've always liked documentaries, and in many ways a reality show is a documentary (at least for the way these women talk, and think)
4) among the things that make me say that it's well-made there's the fact that this show doesn't just show the daily life and behaviour of these women, but also some moral dilemmas, such as "should I leave my husband now that he's in jail?", "should I cheat on him?", "is it ok for my husband to kill people?".

Speaking of reality shows and documentaries, yesterday I always watched this good movie:
Watch Cinema Verite online - on 1Channel | LetMeWatchThis

[...]

So, I was saying about the enabling of the seven systems, one at a time, after they have drawdown or a loss. Does it make sense? As I said, if it doesn't make sense, it still will not hurt, so I'll do it anyway. But let me see if there's a way to test this. [...]

No, I don't know of a scientific way of testing this. There is for sure, but I don't know and can't think of this. Not now. I have other worries and I am distracted. Tomorrow the roommate is coming back, and it's going to be unpleasant as usual. He'll be a clown all day long. Not as bad a clown as vito, but close to it. Even half as much a clown as vito is too much for me.

But I will proceed by rule-of-thumb, because I can't wait too long. I am gonna do it this way. I will enable them, as i said, one at a time, not after a drawdown, but after a big loss.

Let's at least analyze how they've been doing lately.



Ok, I am reasoning week by week, even though there might more than one trade per week, and then I'll have to apply this according to the same reasoning (week by week, rather than trade by trade). But usually there's just one trade per week.

Ok, system by system (this is just to help my reasoning, by no means a reliable statistic):

CL_ID_01: by waiting for a losing week (rather than starting from the first week), I would have missed a bunch of profitable weeks, and incurred in more losses. Bad outcome.
EUR_ID_05: same as above
GBL_ID_02: same as above
NG_ID_02: this would have saved me a big loss
ZN_ID_03: irrelevant benefit ( would have avoided a small loss)
ZN_ID_06: it would have worked (avoiding loss, without missing profit before)
ZN_ON_02: I would have avoided a big loss, but would have missed even more profit before it, so overall a bad outcome.

I noticed a few things:
1) the losses are sometimes clustered together, which would speak against the hypothesis of waiting till there's a loss to start trading.
2) waiting for a loss, obviously always gets you to avoid that loss, but makes you miss all the profit before it, which is often more than what you lose

So, all in all, I can conclude (by guesstimates) that it doesn't make sense to wait for a loss, because, even when losses are not clustered together (which would be like looking for trouble), for a profitable systems (and those I use are highly profitable), waiting around, as a rule, causes you miss more profitable trades than losses.

So I am concluding that it doesn't make sens to wait for a loss or a drawdown, but I am still thinking that it is favorable to start after a long drawdown, such as the one I am seeing right now on many systems (yellow-shaded cells). I don't know if I can keep my reasoning going, because so far it's been good, but I sense that there's something to be gained from starting after a long drawdown.

Indeed, "max drawdown" means the maximum fall from a previous peak, and statistically we all assume it will hold or not be exceeded by much. This is true for the same reason that we do not have the concept of "max drawdown" in dice or coin tossing, where everything is random, and you cannot have the expectation of "max drawdown" (the max drawdown recorded on a sample of coin tosses). If we do have the concept of "max drawdown" is because we think that it won't be exceeded (or close to it). Whereas in coin tosses, whatever happened is irrelevant. But if we're building trading systems, it means we believe the market is not random walk, and therefore we believe previous drawdown helps us predict future drawdown.

And if we believe that, then we could also say that it makes sense to start trading after a big drawdown, since a big drawdown is LESS likely to be followed by more losses.

But then there's these two guys, Ralph Vince and pecas, who say otherwise and say each trade is like a coin toss and drawdown is meaningless:
Portfolio management formulas: mathematical trading methods for the futures ... - Ralph Vince - Google Books
http://www.trade2win.com/boards/pla...ting-future-maximum-drawdown.html#post1691624

So, my final conclusion, regardless of what these guys say, is this:
1) it doesn't make sense to wait for a loss because it could be followed by more losses but especially preceded by a lot of wins, which could outbalance the future losses, but which you'd miss by waiting for that loss
2) it doesn't make sense to waiti for a big drawdown, because with profitable systems, such as those we should be trading, you're more likely to waste profit (same as above) while waiting for that drawdown, profit which could make up for the future drawdown
3) it seems ideal to enter after a big drawdown has happened, if you're lucky enough to start at that point...

...But then would it not make sense to wait for that drawdown? Well, no, because time is money, and time is wasted profit as a rule. So it's good to start after the drawdown but not good to wait for it. It's not making total sense to me, and maybe those two guys above are right, but this would in turn mean that the concept of drawdown is meaningless. But if this is the case, then the whole idea of predicting the future based on the past is also meaningless, and therefore trading systems are meaningless.

I'm gonna need a lot more math than I've covered so far. But at least now I have a better assessment of what I don't know.

It will help to read this goddamn book, The Mathematics of Money Management by Ralph Vince, even though it's too heavy for me right now. I need to do less, in my own way, and better, before starting to fill up my head with his notions. Actually you know what? Since he has a chapter on Markowitz, I will now remove Markowitz, which is hard as hell, and replace it with Ralph Vince. I mean on my signature, at the end of each post.

[...]

Here's a thread totally focused on my previous question of when to start trading (randomly or after a loss/drawdown):
http://www.tradingblox.com/forum/viewtopic.php?t=8980

Not much is concluded, even though the forum has got users with great expertise on these subjects.

 

[Yamato]

Today I've done only the khan review exercises but no statistics nor probability because my eyes are hurting from not sleeping enough and from spending too much time looking at the screen. I still need to record here every time I do or not do math, because otherwise I disrupt my schedule and discipline. Today it's the day i did less math in the last 2 months. But I still did something.

On the other hand I am watching Mob Wives, which requires more reasoning than doing math (crossing and double-crossing calculations, calculating which wife can boast the most crimes committed by the husband, etc.). Fascinating documentary. So far I've liked the most the fourth and last episode I've watched:
http://www.1channel.ch/tv-2720873-Mob-Wives/season-1-episode-4

 

[Yamato]

This thing about prices being random or not is really bugging me, and I feel I should read much more about it because it's the foundation of my testing and probabilities of blowing out in the future, but before reading any books about it (Vince or others), since many also disagree with each other, I need to study the basics of statistics and probability. So, despite not having slept, and since I can't sleep anyway, I am going to do some of the work I was going to postpone.

What I was thinking was this for example. If I start on a random day, my probability of blowing out is about 0.1, given my capital. But if I then withdraw the profits two months later, isn't it like restarting again, and wouldn't then that probability increase, or would it be just 0.1 all over again (given that I already got away the first time)? I would say that the chance of blowing out will then be 1 - 0.9 * 0.9 = 19% , which is equal to 100% minus the probability of not blowing out multiplied by itself... damn. It confuses me, each time I think about it. If I start today, and plan to withdraw in two months, then I have 19% of blowing out. But if I start after I already did not blow out the first time, then, given that outcome has already happened, my chance of blowing out will be, once again, 10%.

Ok, let me think about rolling dice and coin tosses. If, after tossing a coin and getting a head, you toss it again, the probability of getting a tail is not 75%, but again 50%. So in that case, too, if you toss in a sequence, your probability of surviving the drawdown is again 0.5 to the power. In all cases it is to the power, because, even with coin tossing, one thing is to say "after ten heads, at the tenth toss we bet there's going to be a tail" and another thing is to say "we bet from the start that out of ten tosses there will be at least one tail". In the first case you have just 50% in your favor, whereas in the second case you have 1-0.5^10=99.9% on your side: chance of no tails in 10 tosses - Wolfram|Alpha.

But something still escapes me. I am definitely confused about this, but let's adopt the attitude of "a little by little I'll understand more of it" rather than "the hell with this", as I've done in the past. At any rate, this doesn't affect my trading right now.

Also, I still wonder what Vince means when he says that drawdown is an irrelevant value (cfr.previous posts).

[...]

Ok, I did some probability exercises and stats and now i'll try to sleep.

[...]

Ok, here's another one (from the mentioned pecas' thread) talking about how previous trades are irrelevant:
http://www.trade2win.com/boards/pla...ng-future-maximum-drawdown-4.html#post1696278

Also, it is key that you realize that what someone above said about picking a piece of paper and then writing down trade result and then PLACING THE PAPER BACK is absolutely CORRECT. Your prior trades DO NOT effect subsequent trades. If I flip a coin 10 times and the first 4 are all heads does that influence the 5th time I flip a coin....NO....I still have a 50% chance of heads or tails. You can have as many as 10 heads or all 10 tails and obviously any combo in between. So ech coin flip (think trade) has NO bearing on future coin flips. You just know that if you flipped a coin say 2 million times that ON AVERAGE you would have 1 million heads and 1 million tails but there could be strings of each interspersed.

So let me get this straight. I can agree that past trades do not have an impact on future trades. All right. But with one important remark: we're talking about long and short trades. If instead we were talking only about long trades or only about short trades and on futures, the fact that your, let's say long trades keep being losses means the future is falling lower and lower and this will certainly affect the future likelihood of rising/falling (it would be more likely to rise). So let's get back to this: we're talking about a long/short system, and its previous success/failure doesn't affect the subsequent one. So then can we say, like I read on the title of a chapter in Ralph Vince's book, that drawdown is meaningless? According to the concept of trades being like coin tosses, we should, because you can't say that after 10 coin tosses, they're unlikely to continue. Wait. You can. You can say that it's not likely that you'll see x heads in a row. You can say that you'll see it happen once every 1000 tosses. And in the same way you can say that it's unlikely that a given system will have 20 losses in a row for x dollars. So we're talking probability again, finally it's starting to get clear. But then drawdown is NOT meaningless, neither for trading nor for coin tosses.

If we're saying that, out of 1000 trades, we've seen 50% of wins and 50% of losses, this is just like for a coin. They both a probability of 50% of coming out one way or another. And if we say that the past max drawdown is 15k dollars, then it's still similar to the stats for coin tossing. The difference is that we know the probability model for the coin, being that there is a fair coin with one head and one tail. But we do not know what causes the outcomes of a trading system. In one thing, we go from theory to practice (coins) and in the other case we have to do the opposite and find the theory and probability model behind the statistics. With coins we go from proabability to statistics and with trades we go from statistics to probability.

Let's say I am studying the coin tosses but I do not know they're coin tosses. This is similar to what I am doing with a trading system. Let's say there's only 10 outcomes available. I'll see 6 heads and four tails and say this has a 50% chance and so on... what I am getting at, which is not much anyway, is how do we know how to estimate the future based on the past for trading? Ok, let's say we have 200 trades, and we have profit/loss of each trade. After all the analysis is done, how do we know how much that data can predict the future? Is there a formula for it? How do we know how much past drawdown can help us predict future drawdown?

With coin tosses we know that it is impossible to predict the future max drawdown. And so the same way is for trading. But then how do we go about it? We don't really care to predict future max drawdown anyway. We care that, while we're living, we're likely to make money from a system.

Now, let's say that we're still back on coins. You're given this possibility: you want to bet that we're going to get at least one head out of ten coin tosses? Yes, by all means. It is convenient to bet. Are we sure that we'll make money? Not at all. We could get unlucky. Then if we're given the opportunity to keep on betting on this event (that out of ten outcomes there'll be at least one head), we should keep on doing it, because, as I said earlier, there's each time a 99.9% chance of winning:
http://www.wolframalpha.com/input/?i=chance+of+no+tails+in+10+tosses

But still we would not be sure that we won't blow out our account and go into debt forever by betting our money, dime by dime, on this almost certain event.

What i am getting at is that, both with coin tosses as with trading systems (which I now agree are equivalent, in terms of probability theory, provided we're not, as mentioned, trading just long or just short), we CANNOT predict max future drawdown, so this thread is misleading in its title:
http://www.trade2win.com/boards/pla...608-predicting-future-maximum-drawdown-4.html

The title is like "achieving immortality". It should have at least a question mark.

How should we title a useful thread like that? "Appraising probabilities of future drawdown" and also "optimizing portfolio to reduce most likely drawdown". I am not saying that it's not useful. Then how to answer these threads with a different title is an entirely different matter, and I could barely write a post in a thread title like that.

 

[Yamato]

I thought about it, and I felt I wasn't being careful enough so I got rid of the two riskiest futures in the portfolio: CL_ID_05 and ZN_ID_06. They require too much margin and have the lowest forward-tested sharpe ratio of the former seven of the portfolio, which are now only five:



With these five guys I feel a bit safer. From the table above, I can quickly notice that, out of the 56 weeks or so, I would have blown out my account only by entering in one of them. But then again, the again from week to week (friday to friday) is not as bad as the falls that could take place from one day to the other. Despite this, it is clear that, given that i chose these systems for their great performance in the forward-tested period, the back-tested sample is not as good, in terms of sharpe ratio. It seems to be better in terms of profit. I can't delve into it too much right now, but it seems that the adjusted drawdown (with today's prices) is lower than the absolute drawdown, so this could mean a lot of things, such as the higher underlying price and profit of one system offset more the losses of another, or that the lower underlying price of another system caused its losses to weigh less on the overall drawdown. Or that there's a mistake, but I can't check it right now. Not worth it anyway given that I am proceeding by guesstimates, and the overall approximation is much higher than this detail.

Also, I am going to try and wire a few hundreds more at the end of this month, and make the initial capital 4600 instead of just 4000. The expected chance of blowing out is about 14%, which is 1% more than the previous one, but it required more margin, so that exact portfolio would have stopped trading sooner. Also, with a lower forward-tested sharpe ratio for those two futures, that portfolio was less promising and more likely to underperform relative to the back-tested sample (in my opinion), on which i do all the statistics.

Back-tested performance (2002 to early 2010):


Forward-tested performance (last 13 months):


It all looks very healthy. Now I will wait one more month to see what happens to this portfolio.

I hope the euro will rise, so my wire transfer in euros will mean a larger capital. In fact by not transfering my money earlier it's as if I were long on the euro.

[...]

Damn. I can't sleep. I am gonna turn my alarm off. I am drinking Lambrusco at the moment. I'll probably go late.

In the meanwhile, while getting drunk, I am doing a search on "appraising future drawdown":
https://www.google.com/search?q=appraising+future+drawdown

Here's one good link I found:
http://www.aetheling.com/MI/MaxDrop.html

I don't understand much but it is right on target.

 

[Yamato]

previous portfolio

I stopped posting the previous portfolio equity line over a week ago (to focus on the future portfolio), but I just checked, out of curiosity where we'd be had we kept trading it, and here it is:



We'd be between the high of August 8th and the low of September 26th, when we stopped trading. This was total chance, the fact that we started trading it on August 13th (the equity line goes from before to after, because I keep a record of all forward-tested trades), just three trading days after the peak, and stopped trading it on the day it touched the bottom. Pure chance. There was no conspiracy. There were nine previous combinations where this did not happen.

In fact, another coincidence, that 10% is what I am predicting as the rate of blowing out for my future system. There's a 10% chance that, by starting at a peak (which obviously I will only identify later), I will blow out my account. The difference is that my account is extremely small, and that is why I risk blowing it out, whereas on that other big account, what happened is that we lost all the profit made in a year plus 11k of the 160k invested.

 

[Yamato]

Done with daily math: khan review, probability, statistics.

I came across this nice venn diagram of complex numbers:


I found it here, where there's very good information on the subject:
Numbers

Interesting link that compares the different types of charts/graphs and lists their advantages and disadvantages:
lesson6_voc1

 

[Yamato]

As the day I'll resume trading approaches (21 days away), I feel an increasing need to play it safe, and now I've further cut down on the systems I'll be trading. I chose, rather than a portfolio that may have suffered from an optimized (combined) drawdown, a portfolio that includes the four systems with the best individual performance (in two weeks I went from 7 to 5 to 4). These 4 systems:

1) have been doing great over many trades in forward-testing (even though the combined performance is now worse, but, as I said, I don't want to risk overoptimization of the portfolio):


2) have a collective margin requirement as low as possible because they all trade at different times. This is very important, because if I come across a drawdown, and the systems traded change as a result of insufficient margin, then I would not be trading that same portfolio I tested anymore, and all previous estimates would be invalid.

3) have done great, not just in forward-testing, but also in back-testing:


The equity lines and drawdowns are not as good as for previous portfolios, but hopefully less optimized and more promising. In fact this is just the sum of the 4 best performing systems in forward-testing, AFTER making sure that they also performed great in back-testing, and AFTER making sure I can afford their margin despite some drawdown. It's partly a coincidence, and partly I hope it means they're solid.

[...]

Nope. Some points I make are not good. These systems trade so rarely that they'll rarely if ever meet, and the only month at risk is the first one, so margin is not a concern.

Then, statistics do matter.

Third, I have excluded some worse-performing systems that have much less leverage, and those help even out the situation. Given that they tend to make money, a bunch of small systems are the best way to avoid the risk of blowing out, and if anything I should be trading a bunch of so so systems with a smaller leverage rather than excellent systems with a huge leverage. So I am reintroducing some of them, the best ones.

Here's the new situation:


Now risk of blowing out is back at 13% from the 18% it had reached with the four best systems. Obviously I am not even hoping this is a good assessment of the risk ahead, but I know it's a good assessment of the past, a better one than just measuring the maximum drawdown - it's more important to see the likelihood of the smaller drawdowns... of all the drawdowns, rather than just the single biggest drawdown. Measuring all the drawdowns is close to sharpe ratio, but not the same thing. We also want to know how likely we're to blow out the account with the capital we have, because even a sharpe ratio of 8 doesn't tell you the probability of blowing out your account. I'll get better at this as I'll learn more probability theory.

 

[Yamato]

Mob Wives Season 2 Ramona Rizzo - YouTube

'Mob Wives' Starts Second Season With a Bang - YouTube

 

[Yamato]

I got this wrong, because I skipped one step. It was extremely hard in terms of attention required. They are randomly generated and this time a function of function of function... with 17 steps got generated. I forgot a "minus" along the way, towards the end, and now... I am screwed and I have to do a lot more. Doing these things is like performing surgery. If I had been writing an essay, I'd say that I basically made a spelling mistake, no not even that. I forgot to type a letter because I was typing fast, and then I got an F on my whole essay because of forgetting that one letter. A minus is not even a big character - it could be compared to forgetting a comma on an essay and failing it because of that.



[...]

Done daily khan, probability, statistics.

Running out of probability exercises, so I found some good ones here:
Statistics and Probability Quizzes and Trivia -- Fun Trivia

 

[Yamato]

I was fed up with waiting, sick to my stomach with going to the office every day (I am literally having stomachaches from not wanting to work anymore), with no hope in sight, and I anticipated the wire transfer by three weeks. If it goes through, I will start trading in a couple of days. I was getting too close to dying out of boredom, so I might as well anticipate it a little bit. If it keeps me alive, it's worth it.

[...]

Ok, I went through various portfolios, given that I'll start sooner, and I came up with this one:


According to the past, I'd have 11.5% of blowing out (max loss/fall starting from each day/point of the equity line):


But let's assume it's even higher than that, and it's 25%. I'd have 75% of probability on my side.

This portfolio makes about 100% per month, but with that 25% possibility of blowing out on day one. The more it makes money, provided that I don't scale up, the farther I get from the risk of blowing out.

If I instead I were to scale up once a month, each first day, the probability of NOT blowing out would get multiplied like this: .75 * .75 * .75... and so on.

So at the start I'd risk .25 of blowing out, and the second time I'd risk the same (like in the coin toss experiment), but as a whole my probability of NOT blowing would be .75 times .75 = 56%.

That explains something in 2008 I did not know, when I brought my equity line from 3500 to 24000 in 3 months and then back to the point where I had started in even less time:



So, I know from facts that I can go up very fast, and if I keep on scaling up I will sooner or later go down, but now I also know from theory, exactly how quickly that can happen, after studying some probability theory.

Which explains why in some cases I managed to start from a few thousands and bring it to about 25k (and then back to zero), and it explains why other times I managed to start with a few thousands and blow out my account immediately.

But the safest thing might be yet another choice. Rather than investing in 11 futures (cfr. image above) and risking according to the past 11.5% of blowing out, which according to my estimates is twice as much, about 25%, I could choose a portfolio that has zero risk of blowing out, but obviously also a much slower growth. As far as my studies, I still do not know how to assess the preferable trade-off between speed and risk, so I am proceeding by rule of thumb.

A portfolio with 2% of blowing out and a growth of 1000 a month would be better than a portfolio with 20% of blowing out and a growth of 4000 a month.

So now I am going to work on this idea.

[...]

Ok, given that the margin will be low, this might be a good idea. I will start with fewer systems, and then quickly scale up to the portfolio above, once I reach 8000 of capital or so, because this way, according to the past, I will have 96% of probability on my side of not blowing out:



Another element of this activity was to not die out of boredom while playing it safe, so I now have to estimate just how safe I can be without dying from boredom. Also I have to keep into account that if I blow out my account, there'll be a lot more boredom ahead, so, even for the sake of entertainment, it's best to have less entertainment, and make sure it lasts.

So maybe here's the most boredom and safety I can mentally afford (using small contract on EUR and CL):


These have only half as much risk of blowing out, but take months longer to grow:


[...]

The other combination makes five times as much money (in the 8 years of backtesting) and it carries twice as much risk.

Is it worth it? I am not really saying is it worth it to use the safe one rather than the other one, because I'll scale up to the other one sooner or later, but the question is if it's worth it to post-pone using the big one for about one year. Considering in one year I might be dead.

If I trade the safe one I'll die trying whether I am lucky or not. Because it makes 700 dollars per month in back-testing, which in real trading it will be 400 dollars. And it will not blow out my account, most likely. But it will barely pay for the server, which is 300 dollars per month.

The riskier one will make... in backtesting 3700 per month... which in real trading will be 2000 dollars. If that one works, after one month, my risk of blowing out is the same as the safe one, which is in backtesting 5% and in real trading probably 10%.

I don't know. I don't have 20k, which would be the safe investment.

I am going to do this. Given that blowing out means losing only 2000, because IB will stop once I go below 2000, what I will do is use the riskier one.

If I get lucky or rather if I do not get unlucky, it will change my life, even though this time I will neither spend them nor scale up.

Otherwise, nothing will change.

With the safe portfolio, either way nothing changes.

[...]

Ok, I just checked my email and the transfer went through. Now, either way, a month of excitement is ahead. After which I will either blow out or I will have much more money. If it doesn't go well, I could even last just one trade.

But with my systems, probability is on my side. With discretionary trading, I would last under 10 trades. I know this for a fact, from years of losses. So let's just count on the probability I have on my side, rather than on compulsive gambling.

 

[Yamato]

starting at the end of a wednesday, so on a thursday... NOT!

I've been trying to not get confused with all these numbers, and as a consequence I've avoided a lot of probability calculations, but this one bears little risk of being wrong, and even if it were wrong it can hardly jeopardize my results, so it's a win-win situation.

Here's my reasoning. On different days of the week, there's different systems trading (not always, but there's a tendency), and this causes a different profitability based on the day I start trading my portfolio, so why not maximize that, too?

From my calculations (cfr. table below), if I start trading at the end of a monday, so at the start of a tuesday, my probability of not blowing out is 87%, whereas by starting on thursday (on the table at the end of a wednesday, which is "3"), I have a 92% probability of succeeding:


I guess I could start as early as Friday, but I can make myself wait until next Thursday. That way I'll also leave some room for other improvements.

Let's study how this hypothesis could be biased and wrong.

There's different amounts of traded weekdays. Could this lead to a problem? Nope, because we're using percentages. We're asking: "what happens if (in the relativized back-tested period) we start trading the portfolio on a given day of the week?".

The answer is the maximum losses reached on each given instance. And we're told that, by starting on Thursdays, we have the lowest losses (92% with losses under 2500).

From the table we also know that on Thursdays many more systems trade, because we can see that from the numbers of Thursdays that have trades (if a date doesn't have trade, it doesn't show).

So this is telling us that... something that is true by definition. I get it.

It's almost a meaningless finding, because given that our sample of trades is not 1 trade but 24 hours of trades (one day of trades), and given that the systems are all profitable, by sampling days that have more trades we increase, by definition, the probability of those days being profitable. In other words, if Thursdays have on average 20 trades, then they're more likely to be profitable weekdays vs another day that has fewer trades, since wins and losses sum up - but this only happens with statistics and not on the account, because the broker doesn't care how you end the day but second by second.

But this is not it. Now I have to verify the average amount of trades per day, because the fact that there's more Thursdays showing does show that more systems trade on Thursdays, but it could also have the consequence of dividing all these trades between more days, thereby lowering the average number of trades and making this bias null.

Ok, my hypothesis is invalid or at least unreliable:


Thursdays have 50% more trades, with an average of 3 instead of 2. So this means that you're more likely to come out of a thursday with a profitable balance, and therefore you obviously have better stats for thursdays (we don't know how much it affects it, but there's a bias). It would be perfect, if IB measured margin based on a daily calculation, but obviously they don't - they measure it trade by trade, second by second, tick by tick. So what looks good on these stats is irrelevant for IB, and so, ultimately, I can start on any day of the week, since starting on a thursday only appears to be good due to thursdays having more trades.

This confirms to me how easy it is to draw the wrong conclusions when you have a lot of data on your hands. This happened to what I was doing with the investors. We had 7 workbooks, each one with a dozen worksheets. They had me work on these workbooks every week for hours and hours... optimizing this and that... creating this and that... sheet after sheet... but then we had the portfolio optimization all wrong. Portfolio was chosen on the wrong premises: it was the combination of systems that worked best in the past, based on hundreds of tests. That's like curve-fitting a system (mistake that they showed me how to detect, by using the out-of-sample methodology). But we went ahead with this mistake anyway. I knew nothing about portfolio, they were experts, I worked my ass off for months with the given assignments... so hard that all critical thinking was gone. Didn't have time to think about all the work I was doing. We implemented the combination that had worked best in the past, and, not suprisingly, its awesome (backtested) ratio of profit to drawdown fell apart immediately. Within six weeks, maximum drawdown was exceeded, and maximum relativized drawdown was exceeded, and almost doubled... we stopped trading.

 

[Yamato]

Ok, I just woke up, and the money was on the account: 24 hours from the moment I requested the wire transfer. It was a great improvement by using IB's German account, as it used to take me a week to get everything done, and this time I didn't even pay a fee, because of the Single Euro Payments Area.

I enabled my 11 systems. Now I'll see what happens. I already feel better.

 

[Yamato]

Ok, the first trade is profitable (still open, for another few minutes). Much better than if I had traded discretionary. There it is: I am a trader again. I don't know for how long, but I can say I am back in the game. Let's make sure I keep thinking in terms of probability and not in delusional terms, because that would mean engaging in discretionary/compulsive gambling.

[...]

Pretty nice feeling. I come home and I find a trade open which is being profitable. I hadn't felt like this for a while. When the capital wasn't mine, it wasn't the same. Now while I am doing something else, working at the bank, walking or sleeping, there's someone who's at work for me: the program I created, and probability.

I'm gonna do a little math and today I will rest, to celebrate this. As usual, I walked this morning a bit, to the taxi station and got off the cab a bit early this afternoon, and walked back home, so I could save a few euros. Right now I'm showing a balance of -3750 euros on my bank account, and +4200 dollars on IB. This speaks clearly about where hopes are. If anything, I will not eat, but I'll keep everything I can on the IB account.

[...]

Ok, the first trade was closed profitable, but the second trade didn't get triggered because the margin was busy on the first one. I expected this. The price of the second one got away from the entry level, and now I am hoping for a fall in order to implement a manual entry. This borders with discretionary trading, I know, but, emotionally, I could not enter at a higher price (only 7 ticks when I saw it, and now it's 25 ticks) - only at a lower one. I've fixed things so one system will exit half a minute early and free up margin for the other system's entry, which I have anticipated by half a minute (I swapped the time of entry of one with the time of exit of the other). I should have planned better this detail. Now the risk is missing the second trade altogether. But my eyes are tired. From working too much.

I regret not entering. On the other hand, being the average trade 105, I thought that by entering late I'd miss out on all profit even if it went ok. Maybe it was faulty reasoning, because the average trade is not the average profit, but the average of all wins with losses. So the average win is much higher. That is why total automation is the only way I can go. Damn. Even 100 dollars less could make the difference between blowing out and surviving. Damn.

Now I am hoping price (the EUR) will come back down.

[...]

35 ticks away. The price went up and never looked back, and I am stuck outside of the trade. This is really showing me, once more, the power of full automation. I had one thing that wasn't fully automated, due to having little margin, and here i am, hoping for things. Fascinating. What sucks though is that I could have avoided this, with a more careful planning (which i did later, by swapping entry time with exit time, but I might miss one trade because of doing it later).

[...]

That's it, the day is almost over. It's been all right. I did miss one profitable trade though, and it could even be my fault because I didn't insert it manually when it was 5 ticks away (now it's 32 ticks away). I hope this few hundreds dollars won't make the difference between blowing out and surviving. I'll just keep going as if it never happened. In a different state of mind, and with less knowledge, years ago, I would have engaged in revenge trading to make back the profit missed. In probability terms, this is wrong so studying it a little big has been useful. It's like if you don't get a trade right now, you're going to be lucky on the next one. I might indeed get lucky in case a trade will be missed, in the long run, missed trades should compensate each other. Oops, wrong again: actually, since the systems are profitable, I will miss more profitable trades than unprofitable ones. So they will not compensate one another. The more trades I will miss, the more profit I will miss.

Now of course I will not come here and write about every single trade that goes through (today it was four, and it would be too tiring already). As a summary, it was a good first day... or rather: an "ok" first day.

 

[Yamato]

I had to walk 5 kilometers to work, because there was a taxi strike here in Rome. For the first time I didn't mind, because I saved 10 euros, 2 euros every kilometer I walked. I think I will take the subway back home, because I slept well (the best investment right now is sleeping well). So today I'll have spent 1 euro on transportation. This means that I can relax for the rest of the time (two weeks) before my monthly salary gets credited. Yeah, I'll walk to the taxi station and get off the taxi a bit early, as usual, but even if I don't do that, I will be ok until the 27th. Otherwise I'd be worried of getting too in the red, as I have a limit of -4200 euros.

Now that this account is taken care of, let's hope the IB account will grow. Yesterday I made 130 dollars from the trades, plus the dollars I made because the EURO gained 1%, which is 40 euros (I'm keeping my currency in euros). The EUR trade I missed due to lack of planning ended up making only 20 ticks, so, given that I would have traded the mini contract, all I missed was a profit of 6.25*20=120sh (minus commissions). But since in the meanwhile there was a trade by another system that made 26 dollars, all I missed was a profit of 95sh dollars. Which does suck still, but it's not likely to make the difference between blowing out and not.

 

[Yamato]

Ok, back home. I took the subway, so today I spent only 2 euros on transportation, counting 1 euro for the newspaper I bought to read on the subway.

There are no trades so far, for the day. But the EUR lost what it gained yesterday, so I lost that 40 euros made on the exchange rate.

By the way, the exchange rate and stock prices: these are two instances of events not related to probability in the long run. In other words, if I place a random long trade and close it after one minute, then yes, the profit is random, and we could even say that it's 50% chance of success (except for commissions of course). But if I keep staying long, day after day, in a normal political situation like this one, the eur cannot just keep falling. So, whereas for a die we would say that getting a head has an equal chance of 50%, toss after toss, regardless of the number of preceding tails, for the EUR, we cannot say the same. The chance of having a profitable long trade increases proportionally with the increasing of the sequence of the preceding unprofitable long trades (provided the trades are in an uninterrupted sequence, and losing a trade means that the eur/usd exchange rate gets lower and lower). With coins it's one way, with stock indexes and currencies it's different. Coins do not care what sequence they have shown previously, whereas investors do care how low stocks and currencies have fallen.

This could be misleading and discourage us from applying probability to trading systems, and one would wonder why I keep on saying "I have x probability of blowing out" after I just said that probability doesn't apply to the futures I trade. The reason is in the time span and frequency of my trades. My trades 1) are both long and short, 2)last a few hours and 3) are far apart. This is equivalent or at least very close to spinning the wheel each time. If instead my trades 1) lasted months and 2) only went long they would definitely be related to one another, and the probability of a profitable long trade starting today and ending in a year would increase greatly depending on how successful the previous trades were, especially on the currency markets. Basically the EUR is not as likely to go from 1.28 to 0.78 as it is likely to go from 1.28 to 1.78. You look at the range of the last few years, and make your estimate based on that. And, overall, this can give you an edge, and at any rate it's clear that this situation is not equivalent to spinning a wheel. It's basically the difference between spinning a wheel click by click, number by number, which is going long for extended periods of time on the EUR, and that's why in both of these cases (the wheel and the eur) we can say probability does not apply. You're spinning the wheel, but it depends how much you spin it. For my systems, there's a lot of spinning, because i long and short, my trades last little time, and they're far apart.

Pretty good last 48 hours all in all. No fees paid on the wire transfer. Little money spent on transportation. Some money made on the trades yesterday. Financially, a good two days. And, most important, 2 days of trading without blowing out my account. Now my probability of blowing out is a little less than when I began. It's like for tossing a coin. If you have to toss heads twice in a row, and have two tosses. When you're about to begin, you have 25% in your favor. But, after you toss once and you get heads, then you have 50% of probability in your favor. It's like that guy said, on mathgoodies: P of B given A, is equal to P of A and B...
P(B|A) = P(A and B) / P(A)... so P(B|A) is equal to 0.25/0.50=50%.
http://www.mathgoodies.com/lessons/vol6/dependent_events.html

So by the same rationale, my probability of blowing out decreases little by little as my capital increases. It was 11.5% at the start (IF the past actually repeated itself and the best systems kept performing just as well, which they don't) and now it's 11%. When I'll have made 1000 dollars, if it happens, it will be down to 7%.

The stomachache I had, from having a future of sitting at the bank, is gone, maybe because I started trading, or maybe it's totally unrelated and the cause wasn't psychological.

I did some math already, will do some more, and from now on I will stop writing about the math I am doing, because it was useful until here, but after 3 straight months of math exercises done every day, I can safely assume that it's become a habit. From here on, it'd be a waste of time to keep writing a post that says "done some math". But I will write here if I find out something interesting related to math.

 

[Nonameo]

I'm really glad to see you're back focusing on your trading again after so long. Your ultimate goal has always been to become a successful trader, so its good to see you going directly after that goal.

Also great to see you're finally taking the subway. The money saved there could fund the testing of your systems.

I now read your journal in anticipation of trading related news.

 

[Yamato]

Thanks for the feedback.

Well, right. Of course I've always gone after this goal, but without capital the results weren't showing.

I have only recently understood that taking the subway and making efforts to save money is part of becoming a successful trader, not just for the actual money saved by not taking a taxi, but because you learn the value of money, and apply that knowledge in every other area of your life, including trading. You can't manage money in trading unless you know how to manage it in real life. Or rather, taking the subway is a confirmation that you're learning the value of money, that you want money and are understanding how hard it is to get it and to keep it. This has only begun recently, when I've really experienced not wanting to be where I wanted to be because of not having money. The first few years at my job were acceptable, the life in rome was acceptable, and success was in sight, but now the bank has become less acceptable, more boring than ever, and life in rome is not good, and success did not happen. This made me understand that I am here, suffering, because I made mistakes, and that, differently from what I thought, money is not that easy to make and to keep. Another thing that made me realize I learned the value of money is when i told this friend that i would stop going taking her to the restaurant once a month, she got offended and tried to make me feel cheap, and i felt relieved that she had insulted me and would disappear for a while. I am really learning the value of money, and the value of people, too.

I've been a spendthrift until recently (I just looked up the translation of the term, from italian), both in trading and in real life. When you're a spendthrift, and don't appreciate or miscalculate the value of money, getting back at the market and revenge trading (or seeking thrills) is more important than keeping your capital. Or rather, unless you're thrifty when you come to the markets, it takes you a long time to realize that money is easily made and easily lost in the markets. It takes a while, whether you're discretionary or automated, to realize that the money you made today needs to be preserved to face tomorrow's drawdown. Rather than realizing you're a spendthrift, you think that you still haven't figured out the markets and lose your capital as a consequence.

The problem is that, even with an edge, if you spend the money you win, or reinvest the money you win by scaling up, you're going to go down when there's a loss, you won't be willing/prepared to accept it and you'll lose even more, by trying to make it back. Even automated trading is not possible unless you're willing/prepared to take a loss, because you'll step in when there's a loss you cannot afford, and by doing that you'll lose even more. So what's important is to be stingy and not spend/need money, and that way what seems to be "extra" money stays on the account, not spent and not reinvested, and then you find out it wasn't "extra" but necessary when the drawdown hits you.

I wish I had realized and known all these things earlier, but I modestly accept the saying "it's never too late" and modestly accept whatever money will come now from the markets. You can't always win, and this is the price I pay for not having learned things the hard way ever. I never had to look/apply for a job, never worried about housing and similar problems, and I never really felt the value of money and sacrifice. So the price I pay is not getting things done sooner as far as trading. I didn't learn things the hard way, so the drawback is that I learned things very slowly. Had I been stingy from the start (many people are stingy since they're teenagers), and I mean it as a quality, I wouldn't have had a lot of problems I've had. But then again I might have never started trading. And yes, I'd have more money by now (I am hearing all the objections from the banned readers, so I am answering them).

 

[Yamato]

SNL first show ever:
Watch Saturday Night Live online (TV Show) - on 1Channel | LetMeWatchThis - Season 1 and Episode 1
Saturday Night Live (season 1) - Wikipedia, the free encyclopedia
SNL Transcripts: George Carlin: 10/11/75

 

[Yamato]

weekly update

The week was breakeven. I got neither lucky nor unlucky, but lucky with the timing, because if I am breakeven and the systems lost 1000 in the past two weeks this means that had i started two mondays ago, I would be down 1000 now. So I am relieved. The past doesn't prevent future losses, but certainly I didn't incur those losses, which would have been certain had i started two weeks ago.



The blue-shaded values are only part of what i lost, because all (weekly) trades for a system get summed up in one cell.

 

[Yamato]

probability applied to drawdown: questions

I have to solve a couple of unanswered questions as far as probability theory applied to drawdown, not because it's fun but because I can apply it to my trading, therefore my profit, therefore my life. That's the only thing that can make me study math.

So, the first question is something that gets answered in various contradictory ways in financial forums: is it best to enter after a drawdown or it doesn't make a difference?

This in turn is connected to another question: does past drawdown help predict future drawdown?

In my little study, I will use a comparison of coin tosses vs. trades.

Let's immediately clarify that we're comparing trades from automated trading, and not from discretionary trading, so psychology does not have any part of the decision-making process, in which case the previous outcome of trades would most likely affect the future outcome, though I don't know in which way, whether you tend to trade better/worse after a string of losses/wins.

Another thing to clarify is the 1) frequency, 2) distance, 3) long/short direction of my trades, which makes them very similar to the tossing of a coin. If they were all long and lasted years, and weren't far apart, then of course after a string of losses, a win would be increasingly likely.

Now, are there really no differences between my trades and a coin?

First of all, but this is not a problem, given that my systems have a 60% probability of winning if not higher, and a ratio of average win to average loss of more than 1, let's roughly make the pretty accurate estimate that for my systems each trade is like rolling a die, whereby an outcome of 1 through 4 means a win and 5 or 6 mean a loss.

Problem 1:
an important difference could be (I hope not) that the losses and wins are correlated. Depending on the markets' behavior the frequency of losses might increase. This might not affect our reasoning and tests, focused on something else, but it has to be remembered. You see, if you create 1000 systems, probability would seem to protect you (though never guarantee you) from losing on all of them at once, but if they're highly correlated, you would still lose much more than you expect judging from probability estimates. But, as i said, this might not be a problem for these tests, just something to be remembered.

Problem 2:
Something else not useful here, but to be remembered is that there probably is something similar to a survivorship bias for my systems.

Problem 3:
Another thing: the systems might lose their edge over time, either because the market changes or because other people use them, because they've also discovered their edge. So the probability measured in the past would not be the same as that of the future.

Problem 4:
This is much more relevant for my tests here. I know for a fact, from tossing a coin a million times, on excel, that the probability, with the law of large numbers, is indeed 50% for heads and tails. I cannot say the same for my trading systems. There are thousands of trades, not millions. So the probability of the past is not as easy to assess for trades as it is for coin tosses.

So, back to my original questions: is it best to enter after a drawdown or it doesn't make a difference? Does past drawdown help predict future drawdown?

Let's say that a drawdown of 10 consecutive losses (even by different systems) wipes out my capital. Given the probability of my systems and the die of being profitable (from 1 to 4 included), my probability of not incurring those 10 losses in a row is... 0.33^10...

I thought of something else. I'm going to take all the back-tested trades I have, measure the average loss and average frequency of losses, and see if I can come up with a better estimate than the die comparison.

Roughly 57% of wins in back-testing.

Average win 473 vs average loss of -373.

This gets really complex for me. It's going to be hard to not make mistakes at this point. I am going to go back to simplifying things, because here the only objective is not correct assessment of my probability of drawdown, but only whether it makes sense to compare the drawdown of coin tossing and die rolling to the drawdown of trades.

So, back to my original questions: is it best to enter after a drawdown or it doesn't make a difference? Does past drawdown help predict future drawdown?

So we can actually even get back to coin tossing, which is far simpler than die rolling.

With coin tossing, we know from both tests and theory that, after 10 consecutive heads, your chance of getting head or tail is still 50/50.

So, if trades are like coin tosses and in my case they seem to be (given the frequency/duration/direction of my trades), it should be clear that entering after a drawdown, unlike many say on financial forums, does not make a difference. If you run millions of tests (I did), and then you check all the coin tosses following 10 straight heads, you find that there's roughly 50% for each heads and tails.

So this is already a big finding, because even yesterday I was saying "hey, I am glad the drawdown is behind me". That in itself doesn't mean anything. It is counter-intuitive, but, given the nature of my trades, "drawdown behind me" is not a valid concept, no matter how many times it recurs in my head.

If all the problems listed above are not problems, and the drawdown is random, then we cannot assume that entering after a long drawdown will keep us from a future drawdown. Given that we do not know how big those problems above are, especially the correlation of systems, it's safer to assume that the results are random than to assume that all the losses happen at once. It's closer to reality. Both approximations are not exact, but hypothesizing the randomness and non-correlation of my trades is closer.

So, back to my original questions: is it best to enter after a drawdown or it doesn't make a difference? Does past drawdown help predict future drawdown?

Did I answer them yet?

Yes, the first one has the answer of "it doesn't make a difference", hard to believe but it seems the case.

But then past drawdown might still help us predict future drawdown, in probability terms. Not that losing yesterday tells us we'll win tomorrow, but the frequency of past losses tells us the frequency of future losses, and the frequency of strings of losses (drawdown) tells us the future likelihood of that, too. This is the field of inferential statistics, because we do not know the probability model for trades, but we try to assess it based on the data we have.

What's the difference between probability and statistics?

In probability, we start with a model describing what events we think are going to occur, with what likelihoods...
[...]

The statistician turns this around:

1. Rules ← data: Given only the data, try to guess what the rules were. That is, some probability model controlled what data came out, and the best we can do is guess — or approximate — what that model was. We might guess wrong; we might refine our guess as we get more data.
2. Statistics is about looking backward.
3. Statistics is an art. It uses mathematical methods, but it is more than math.
4. Once we make our best statistical guess about what the probability model is (what the rules are), based on looking backward, we can then use that probability model to predict the future. (This is, in part, why I say that probability doesn't need statistics, but statistics uses probability.)

[...]

So, recapitulating, entering after a drawdown is useless (given the nature of my systems).

But, to make sure that the concept of "drawdown" is not useless altogether, we have to verify if we can use it to make predictions.

So, resuming my (simplified) comparison of trades to coin tosses, let's say that 5 consecutive heads (or 5 consecutive average losses) are going to blow out my account.

This happens only if I start trading and then that string of losses happens. Because otherwise I have the time to build up my capital and survive that string of losses.

So, to judge my chances of survival, with coins I use a known probability model and with trades I use my back-tested data. And I come up with this same conclusion: the risk of blowing out by getting a string of five losses when I start is 1/2 * 1/2 * 1/2 * 1/2 * 1/2 = 1/32.

Both with coins (model assessed from theory and tests) and with trades (model assessed from tests), this is telling me how likely I am to blow out.

If instead I win once, then I need to lose 6 times in a row, in order to blow out. Damn. I need to add that, too. Well, for trading, all I could do and all I did is measure past trades and measure the fall from any point of the equity line and I came up with my assessment, but with coins we have infinite data, so let's keep on seeing all the implications of it, and pretend we're talking about trading, and then I'll see if I can get any insights.

At the start, by tossing five more times, I have a 1/32 chance of blowing out. Before going any further let's test this both in theory and in practice, on excel.

[...]

Ok. This works:
View attachment assessing_probability-theory_vs_practice.xls

The comparison comes to mind again, because in terms of trading drawdown, by losing 6 times in a row after a win, I still blow out my account. And by losing seven times in a row, after winning twice I still blow out my account. Not only that, but if we use coin tossing, we know that, in the long run, we're always at zero, if we were to lose one unit with heads and win a unit with tails. So, things are different here. Because with trading, our wins are bigger and more frequent than our losses. This makes me regret not using the dice rolling analogy, but this is much clearer for other purposes.

So, anyway, the excel workbook (cfr. file above) showed that, both according to theory and practice (10,000 coin tosses), the probability of blowing out by starting at any given time and rolling five times in a row is 3.125% or 1/32. This drawdown would certainly destroy us by scaling up, because, if we keep on scaling up, out of every 30 trades on average, we're bound to come across a string of 5 losses (given a 50% win rate).

But if you do not scale up, as I am planning to do, and always assuming that you make more money with a win than with a loss, then my chance of blowing out is almost exactly at 3.125%.

So, what was the question again? Right: does past drawdown help predict future drawdown? Yes, both with coins and with trades.

I (approximately) seem to have answered my doubts and questions, even though I know that I may have forgotten about something I was wondering about, that soon new doubts will arise in my mind, and I will find out that I hadn't understood everything. This is a complex field, with many counter-intuitive facts, and you can't figure it all out just by reasoning. Reading, and taking tests is useful. For example I found some new tests here:
Statistics and Probability Quizzes and Trivia -- Fun Trivia

They're anything but easy, especially if you have the timer on.

[...]

I just got out of the bath. I was thinking about this counter-intuitive nature of drawdown, coin tosses, and probability.

What's really hard to grasp is the first question I had asked, and answered this way: after a drawdown you're not any safer than you were before it.

But that's the way it is (provided the trades are made the way they are in my systems, long and short, distant, and of limited duration).

If you toss a coin and get five heads in a row, you feel: "come on, what is the chance of getting another string of five heads in a row, AFTER it already happened?". Well, it is exactly as before it happened, 3.125%. It's hard to grasp. When you're trading systems, not just me, but I have heard it from others, you feel that you're going to be safer by entering right after a big loss. Instead you're not. It's so counter-intuitive.

[...]

Something else worth mentioning, but not as counter-intuitive, even though it is often mistakenly assumed, is that no matter how much capital you have and how little you risk, and no matter your win ratio (unless it is 100%), then you're never guaranteed to not blow out your account. You will decrease the probability, but you're never guaranteed. So, in this sense, we can't really divide traders between reckless ones and safe ones. The difference is only between the level of risk one takes. To some extent, we are all running the risk of blowing out our account.

[...]

Wait a minute... I can take it a bit further. We have established that, as far as my systems, we can compare trades to coin tosses.

But then we noticed how, as far as coin tosses, if the sample is large enough, practice is almost identical to theory (cfr. file above), and therefore this means that if the sample is large enough we can derive a probability model from practice (i.e. "statistics", "data", "backtested data"). Indeed it's clear that practice increasingly resembles theory according to the size of the sample: 10 coin tosses will not resemble theory as much as 10,000 coin tosses.

But then we also said that my problem with trades is that the sample is not large enough to verify all potential outcomes. But, if our trades can be regarded as coin tosses (except for the win/loss ratio, and size of wins and losses), then we could just multiply them by 100, and mix them together in random ways, to see what we come up with. So I will do this little test, on excel.

Then I will compare the results and see if they match my probability estimates from the same trades, only this time not randomly mixed but according to the back-tested sample's chronological order. I've always been careful to not play with too much data and formulas, in order to avoid drawing the wrong conclusions. Better no conclusions than wrong ones. But in this case, I think I can go a little further with this study, without the risk of coming to the wrong conclusions.

First of all, let us see the hypothesis drawn from the 8 years of backtested relativized data, as far as my likelihood to blow out my account by starting to trade on any given day of the previous equity line:

According to the past, I'd have 11.5% of blowing out (max loss/fall starting from each day/point of the equity line).

Let's see if, by mixing up all the 1715 trades and multiplying them by 100... no wait. Those are the trading days. The trades are many more than that, and to do things properly, I should mix the individual trades and then recalculate the drawdown.

I will first get all the relativized (profit/loss of) trades, probably thousands of trades.

Then I will multiply them by 100... I can't. Excel limit. Well, I'll multiply them as many times as possible. Then I will add a column and assign to each row (each trade) a value returned by the rand() function. Then I will copy/paste those values as "values", thereby removing the formula (otherwise they keep changing). Then I'll order them by number. Then I will apply the maximum fall function, to see the max fall ensuing an entry at that trade. I could even plot a chart. I have a rough idea that what I am doing here is somewhat related to monte carlo sampling, but I don't have the skills nor time to find out.

[...]

Ok, getting there.

The total trades for the 1715 traded days are 3530. Given that excel has 65,000 rows (my version - the other one sucks) I will be able to multiply the trades by 18. Good enough. Let's do that.

Done. Damn, here's the findings (I shaded in blue the safe area, of 77%):


It clearly shows an increase in potentially harmful days, in this case trades.

As the previous picture showed, on a day by day basis (which is not IB's method of measuring margin), the drawdown threatened to wipe me out 11.5% of the days I could start trading the portfolio.

On a trade by trade basis, this "monte cazzo" sampling I did (cfr. what I wrote above about my experiment) show a 23% probability of blowing out. This is definitely closer to reality than my figure. And this is provided that trades are distributed randomly. Because otherwise, I could get very lucky or very unlucky, depending on when I begin.

But let's conclude this study by saying that my probability of blowing out my account is definitely closer to the 23% value I found today (probably the increase is mostly caused by this change: using trade by trade vs day by day) than to the 11.5% value I had found.

I am getting closer and closer to the point of it not being worth it.

On the other hand, I wonder if it counts, in probability terms, the fact that I blew out my account so many times previously. Probably not, because I was using different methods. Had I used the same exact method, in the long run, no matter how risky, the method would have worked. After all, given that I started with 4000, and I'll blow out at 2000 (minimum required by IB to trade), this is like an expensive lottery ticket, which has a 25% chance of not being a winning ticket, and a very large potential profit to the upside. Another feature is that I cannot touch the profit for a long time after it happens because otherwise I reduce the potential to the upside.

[...]

Damn!

I had second thoughts about what I said, that it was probably due to the change in timeframe (trade by trade rather than day by day). I checked, and look:


I was totally wrong: the change in timeframe did practically nothing. We only go down 1% by changing the timeframe, so it's totally irrelevant. The difference is coming entirely from resampling.

So why did risk double from 11.5% to 23% by resampling the trades? Because... it might be that the systems I chose, the portfolio i put together... is overoptimized. The trades are not like a bunch of trades put randomly together, as they might be in the future, but a bunch of trades by systems that I selected because they produced, in the past, trades that fit well together. Did I do that? Not really, because I chose the best systems available, but it is true that I did check if the trades fitted well together, and they did. If they hadn't I would have discarded them.

So, resampling (or whatever it's called) clarified that this portfolio looks better in the past than it might turn out to be in the future, in that, even if the future trades turned out to be as good as the past ones (which is unlikely), those trades, if randomly distributed, have a 23% chance of blowing out my account.

This is my finding. I'll have to digest it in the future days, and draw some more conclusions from it.

One immediate idea is this: if trades are not randomly distributed... or rather: if the favorable chronological order/distribution of the trades in my back-tested sample is just random luck, then I am by all means wrong to be selecting this portfolio. If instead the favorable sequence of trades derives from the fact that when one system loses the other wins, then it's not as wrong, but still we do not know if we can count on this happening in the future.

However, all in all, I don't think we can count on systems to positively compensate one another, and win and lose alternately. I think the best we can ask of them is to not lose all at once, and, even better, to lose randomly. It is just too much to expect one to win when the other loses, and if this has happened in the back-tested portfolio it is likely to mean that we chose the one lucky combination of systems that makes this happen, that is we curve-fitted the portfolio.

So, I'll have to watch out for this. Despite this, even with the resampling, this portfolio is acceptable, considering I am not betting the farm, but just my 2000 dollars.

Here's the resampling tests I've been talking about (blue sheets), with a comparison to the original trades (yellow sheets):
View attachment monte_carlo_sampling_of_my_trades.zip

This is not it. I went back and ran the same test two more times, to really make sure that mixing up all the back-tested trades for the selected portfolio would yield similar results, and it did. The first time I resampled my chance of survival was 77.1%, the second time it was 78.8% and the third time it was 78.1%. This is totally reliable. And it means that if the back-tested trades from my systems were mixed randomly the chance of blowing out by entering the equity curve at a random point would be 22%. In the past. In the future we don't know.

But now the next natural step is to apply this method to the forward-tested period. This resampling really seems a good way to avoid curve-fitting.

Ok, here's the forward-tested trades and what they say about the likelihood of blowing out: 6%:


Not much. I think that the resampling will show about 20% of chance of blowing out.

Wrong. It got a bit worse but not much worse, 11% of blowing out days:


All in all, I could have a shot. If I lose, I lose 2000. If I win, the upside is unlimited. My guesstimate is that I have about 75% of making it and not blowing out.


IMPORTANT:
One last important thing before I forget. From all these tests it would seem that, no matter how pessimistic, you could never blow out your account if you had a capital above 20k. This is not true. You're never completely safe, even if the systems never stopped working. If you resample the trades enough, or if you let enough years of real trading go by, even assuming the systems keep working and having the same performance, you'll come up with a sequence of trades that blows out your account. The only thing that prevents you from blowing out is that you stop trading sooner than the blowing out opportunity arrives. As capital increases and/or as your number of systems increases, your chance of blowing out grows smaller and smaller, but never goes to zero. And as time goes by, if the system keeps getting traded (even assuming it keeps having the same edge, as I said several times before), each time it tests its luck. The only reason we saw the biggest drawdown at 10k in 8 years of regular sample and at 19k in 160 years of resampling is that they were precisely x years and not infinite years. Another thing to mention though, is that as capital increases, a drawdown of 18k will not be enough anymore to blow out the account, and you'll need a bigger and bigger drawdown to blow it out, but while your chances of meeting with a bigger drawdown will increase as time goes by (simply by trading longer), also your capital will increase, putting you in a safer and safer position, farther and farther away from the risk of blowing out. So yes, the chance of blowing out will still be there, even though infinitesimal. But even now, if you had a capital of 20k, you'd have an infinitesimal chance of blowing out your account using these systems. You'd make 10% a month. I am not saying this to attract investors, just saying it to myself to remind myself of how i blew it, given that in several occasions I had that capital. If it'll happen again, after all I've seen happen, I'll settle down. No way that I'll spend it like I did before, no way that i'll scale up like i did before... I'll just be extremely happy that I am making anything at all and on my own, and without worries.