my journal 2

[meanreversion]

Personally I trade futures and backtest everything on a 15-minute timeframe with data from disktrading.com, which I found to closely match IB data. So I would say that I am happy with both. On top of it, trades from systems backtested usually match the forward-tested ones, so the data is reliable from all points of view and all cross-examinations, at least as far my low precision is concerned, because I wouldn't know how well it rates on a tick by tick timeframe. So I have no advice other than the data sources everyone knows already, such as tickdata.com which charges you over a thousand dollars per symbol so it is beyond my reach anyway.

I've used disktrading as well, for FX data.

FX is OTC, so good data is tough to get. But of course futures are exchange traded, so the data should be good as gold.

And equities, they're exchange traded as well, so the data should be ... hmmm ...

Bad data always shows excess profits for a mean reversion strategy, and excess losses for a breakout system.

Travis - is this your forename/surname/favourite band?

 

[Yamato]

Yeah, travis is travis bickle, the protagonist of my favorite movie. This guy here:

http://www.youtube.com/watch?v=QBvg3PkI-PU

The drugs are kicking in.

I almost can't write any more.

I almost cannot keep my eyes open. One last effort to get up and brush my teeth.

Damn, I brushed my teeth, and woke up all over again.

Now i have to write another 20 minutes of stuff.

Here's another song:

No more complaining for today. Something poetic to write maybe.

No, nothing like that either.

Another beer could help, definitely.

I can't reach that beer on the piano.

I ran out of alcohol as usual.

I'll just keep on quoting songs and posting videos.

we are the champions, we're are the campions, no time for losers... this doesn't make any sense. A champion could lose, too. That's why he's supposed to be fighting until the end. If he's won why would he be fighting till the end? So a champion could be a loser as well, so he definitely has time for losers, too.

I'll keep posting. After all this is "my journal". So I can't be spamming my own journal. Nor going off topic on my journal.

Brainstorming type of thing.

What makes this girl go into a music career and write her songs and sing on stage? I think she's ugly, so she had to achieve some success to get in bed with the guys she liked. What do you think? I am trying to make money for the same reason. Achieve success to get in bed with some girl who never wanted me. It would have been faster to content myself with a less pretty girl.

Nothing to say about this nowhere song.

I don't know what it means.

Yup, I feel just about satisfied and I will end with this one or maybe a couple more. No more words, but more feelings about this one than about others with words. This is saying a lot of stuff about me and about who's writing it: it's saying "let's stay away from loud places, with loud stupid people, let's stay alone, in the middle of nature, and think and be sad or simply not be bothered".



Nothing. Still can't sleep. Should've watched a movie instead of doing more complex tasks tonight, which I don't remember what they were, but a movie would have been better.

Too late to start one now.

I need one more music video to end the day on a nice note.

Yeah, this is it. I am a little crazy, like the guys in this movie. I feel comfortable with that idea, and being around bizarre people. I'd be more comfortable in the cuckoo's nest than around my colleagues at the office.

 

[Yamato]

13k right now... and the dream goes on.

http://www.youtube.com/watch?v=MSeGTrPBuOE

 

[Yamato]

http://www.watchingthatmovie.com/watch-150931-Sunshine

 

[Yamato]

summary of the week

The systems are above 13k and are now making 1k per week quite reliably. I feel good about running systems that make good things happen on a regular basis. I never got to this point. Yes, I made this profit before, all in a sequence, but I've never gone beyond a capital of 31k, and instead things here are happening, and without losing my calm. Ok, I have to write it all over again. I've gotten to three straight months of profit before, I've gotten to this capital invested before, but I was out of control and didn't know what I was doing and wasn't trading systems this good - I was just being lucky and gambling - now I am doing this thing scientifically in comparison. I am feeling different now, and we're going to take it a lot further.

--------

Annualization of sharpe ratio of my forward-tested systems. Tomorrow I'll have to annualize the sharpe ratio table and related chart (the one I keep posting) and I'll post it, and mention the changes, only if I agree with the annualization of it suggested by the investors. Otherwise I'll post it my way.

------

Creation of new systems, or rather: applying old systems to new futures.

I ran out of new systems to enable and I have decided to create more systems, based on 3 to 4 more futures, that I have not been trading. I will do it all at once, to save energies (I think it's called "economy of scale"). First I'll decide what futures, then I'll buy the data, then I'll test the existing systems on the selected futures.

If the Nasdaq is not much more correlated to the ES than the CL is, then that is one candidate.

I am debating on the other candidates with the investors. I am also thinking about Silver but volume needs to be good. If volume is good and they're not correlated I could do a bunch of metals.

------

News on the home front. We're moving to another apartment in the same building. It bothers me, I don't see the point of it, but the advantage is that I won't hear the neighbour bitch anymore.

------

Problems continue with the "office improvement" team. I don't want to see those idiots around me. I have been working non-stop for years, while others slack off, we have requested people and tools, were given none, and we don't need two idiots teaching us how to optimize our work. I feel very insulted by their presence around us, which will continue for the next two months and maybe for a whole year. On top of having a stupid job, these guys are not bright at all nor cultured. It is a very bad sign for my bank, the fact that resources are wasted this way. 4 people are doing the job of 10 people, and we get sent these idiots trying to speed things up even further. At the same time, there's dozens of slackers doing nothing, and they're left alone. I feel insulted, and I can't hide it. We've had a meeting, and I didn't nod in approval when the head of compliance was telling us about the great potential of the "office improvement" team. I now think he's an idiot, as well.

In part though I have to admit that i need an enemy to hate and an obsession to fixate on. Having lost Vito, I am now focusing on the next enemy to destroy. They are evil, and they must be defeated. Them and the head of compliance. I am not going to kiss any asses like the others. The enemy is the head of my department. I can't get myself fired, but I will now think he's an idiot. Sorry, man. You've really gone too far this time. If they'll ask me what I think I will say it, too. Yeah... of course, I am not expected to get an appointment with him and tell him that he's an idiot. The hell with that. I am not out to save my bank. But I definitely did not show any enthusiasm when he was describing the qualities and potential of the "office improvement" team. Obviously he was wrong, and he's too powerful to be told about it. Now, only the butt-kissing people get promoted so he's now surrounded by people who tell him he's right. I am not asked my opinion. I can only tell my boss and some other guy higher up. But they won't tell him that he's wrong nor what I think. No point in me telling him what I think - he wouldn't care. I would just get hurt and gain nothing. Like a Chinese student in Tiananmen Square going against a tank. Screw them all - I am saving my bullets for better causes - myself first of all. I don't have to defend the bank from its own parasites, including the managers themselves. Certainly, I am not there because I belong to the butt-kissing crowd, but the hard-working group. Let them do whatever they want to do. It's not in my power. But it's in my power to bad-mouth all the stupid people I see around me.

 

[Yamato]

weekly update and more

Ok, let's get started on all my work for the weekend.

WEEKLY UPDATE
First of all, some self-congratulations for the growing equity line, which has done nothing but grow in the last 3 months:



Those first awful 6 months are definitely behind us. I always knew I had good systems - I just didn't know exactly which ones they were.

The pattern of equity growth now seems to be two weeks up and one week down, with the weeks up being an average of 2000 and the week down being an average of -1000. This is a profit factor of 2000*2/1000=4. Self-congratulations once again.

Now let's do that sharpe ratio work I had to do.

MY SHARPE RATIO VS. TRADES MADE SCATTER PLOT
First of all, let's look at the most recent sharpe ratio without that annualisation the investors asked for. As usual I have marked with green circles the systems we're trading and this time there's no red circles because I am not going to ask the inclusion of any new systems:



Ok, this is a tough one and yet an interesting one. I think I have found out what the problem is and why the investors did not like my scatter plot.

MISUNDERSTANDING ON MY SCATTER PLOT
They most likely thought that my scatter plot meant: "a system's performance is given by its sharpe ratio and its frequency of trading". This makes sense, because say you have two systems with sharpe ratio of 2. Say one system trades once a year, and the other one trades once a week. The first system will be useless, and the second system will be a gold mine.

So that has to be kept in mind, too. However, all I mean with my scatter plot (and yes, it is less information than one might have expected) is: "a system's performance is given by its sharpe ratio and its sharpe ratio is reliable to the extent that the system has traded".

Now I will try to find a way to make everyone happy, also because the frequency of trading is no small matter, and should be included in a chart.

We cannot have a chart with three variables.

On the one hand, we could say "let's get rid of the reliability of sharpe ratio" and only show systems that traded more than 10 times, but we'd be losing some important information. A sharpe ratio from 10 trades is not the same thing as a sharpe ratio from 100 trades.

So... in fact there's no way I am getting rid of my part. Let's see if their information can be included somehow and how important it is.

WHY MY SCATTER PLOT WILL STAY AS IT IS
What they're saying (I am assuming) is that we should measure how often a system trades because the same sharpe ratio could have different implications depending on the frequency of trading. And i've agreed that a system trading once a year is worthless. At the same time, guess what. If that system trades once a year, I will never have the 10 forward-tested trades needed to correctly appraise the sharpe ratio, so that problem is ruled out. Another point is that if a system trades only once a month instead of once a week, yes it will be worse than equal sharpe ratio trading more frequently, but it will still be worth trading, and since here we're not having to choose between systems but we can enable all the good ones (in fact we already ran out of good systems to enable, as can be seen from the scatter plot itself), the frequency of trading is an important matter and yet an irrelevant matter for our situation. So I will definitely preserve the information of sharpe ratio and its reliability, and archive this matter as a misunderstanding.

Now I will focus on creating new systems on new futures, but I will start a new post because this is a huge endeavor.

 

[Yamato]

looking for new futures to build new systems on

The simple reason for adding futures instead of building new systems on traded futures is as follows. It's easier because i'll apply the existing strategies on the new futures, and it's a more effective diversification, provided that the new futures are not too correlated to the existing ones (so I'll be looking out for that).

But first some music:

Resuming from here:
http://www.trade2win.com/boards/trading-journals/85510-my-journal-2-a-64.html#post1095702

It's a post I have written a year ago, when I was doing the same search. But now I am ready to add more futures, with the extra knowledge I have gained in a year, and the extra capital to invest.

So here i am again, looking at futures listed by volume:
http://www.futuresmag.com/Issues/2010/March-2010/PublishingImages/2010TradersViewChart.pdf

For a variety of reasons, such as volume, exchange and advice from investors, here's, from the link above, the futures that I will analyze to understand behaviour and nature (what they are, their correlations, etc.):

STOCK INDEXES (FUTURES)
E-mini Nasdaq 100
check correlation to ES and CL

MEDIUM-TERM INTEREST RATES
5 Year Treasury Note
2 Year Treasury Note
check correlation to ZN and among each other

SHORT-TERM INTEREST RATES
Eurodollar
check correlation to ZN and find out what the hell it is

METALS
Here the investors suggest copper and silver. Of course I shouldn't have to repeat for potential objections that we're already trading Gold. Let's also see if there's anything else with good volume and come up with a list.

The metals are largely traded (according to that futuresmag.com link above) on other exchanges, but these are the CME ones (the ones I care about):
http://www.cmegroup.com/trading/metals/

Let's first of all analyze the volume:
http://www.cmegroup.com/trading/metals/files/MoMU.pdf

Silver and Copper are ok. The others have a problem.

Let's now check correlation between... no let's do the correlation part later. So here's my final list for metals:
Silver
Copper
check correlation to ES, EUR, GC and among each other




NO LUCK WITH THESE...

ENERGIES
Here the investors suggest heating oil. It's got a low volume and now I am comparing it to CL to make sure it's not too similar.



Ok, everyone. I've checked its correlation, weighed pros and cons, and I will drop it. Volume under 100k. Not good compared to all the other futures we're evaluating and given its large correlation with CL. Bad months (pain in the ass for roll overs), bad unit of measurement. We've got better opportunities to invest my energies on. This will have to wait until the next turn.

CURRENCIES
The investors suggested the DX (US Dollar) but for a variety of reasons I will not add it. Not that I don't have the data, because in fact it is the only one I already have bought. The reason I don't like this is that, unlike the others it's on ICE first of all, rather than CME. Second of all, and most important, it's not a currency but a basket of currencies - it doesn't feel right, at all. It's like trading a basket of currencies... the weighting could change in the future... it's a mess. I don't trust this. I won't even get started on it.

GRAINS AND OILSEEDS
I'd have loved (since a year ago) to add some of these, but the data from disktrading.com is not good (only intraday), the data from tickdata.com is 1300 dollars per symbol, so I am giving up on these. I can't afford 1300 dollars per symbol. On top of it, they have a low volume, weird expirations, weird units of measurement.

SOFTS
With Sugar I've had a problem similar to that I had with the other guys above. I explained it in detail here:
http://www.trade2win.com/boards/trading-journals/85510-my-journal-2-a-71.html#post1107792

The work described above will take me an entire week. The objective is to come up with about 5 more futures to get data for, to back-test existing systems on, to automate new systems on. Ultimately I am expecting to bring my total number of forward-tested systems to about 100, the number of traded systems to about 30, and the unprofitable months to zero. Right now it's about two red months per year, in my guesstimates. Yes, my equity curve doesn't show that, but it took me six months to understand what were my good systems and to disable the bad ones, and it took me nine months to enable all the good ones.

Recapitulation. I will have to study, understand and compare (correlation studies, as listed above) these six futures:
E-mini Nasdaq 100
5 Year Treasury Note
2 Year Treasury Note
Eurodollar
Silver
Copper

Each future will produce about 5 systems, for a total of about 30 new systems. This thing will take months.

 

[Adamus]

Re: weekly update and more

We cannot have a chart with three variables.

I believe you can. Try a scatter plot on a horizontal plane, with each plot as a column showing the 3rd variable on the y axis. Isn't that readable?

 

[Yamato]

Yeah, I saw it coming, and was going to correct my sentence, but then decided against it, because "can" has many meanings. What I meant is that I cannot build such a chart and keep things simple and clear, which is always my requirement, so as far as my journal writing is concerned, I can say "I cannot do it" (given my requirements). Since it's my journal, I can't keep on explaining out loud each time things I am taking for granted.

 

[Adamus]

Yeah, I saw it coming, and was going to correct my sentence, but then decided against it, because "can" has many meanings. What I meant is that I cannot build such a chart and keep things simple and clear, which is always my requirement, so as far as my journal writing is concerned, I can say "I cannot do it" (given my requirements). Since it's my journal, I can't keep on explaining out loud each time things I am taking for granted.

Couldn't help but reply because I don't understand your loyalty to your scatter plot.

 

[Yamato]

I can't help but notice that I've already written why my scatter plot is excellent. You see, sharpe ratio summarizes the whole performance. And the number of trades (per system) measures the reliability of that sharpe ratio value. Nothing else is needed and what's needed is all on that scatter plot. Also, my father has helped me with that scatter plot, so now it's magic. Actually I didn't even know what a scatter plot was until he told me.

 

[Yamato]

maybe I am a little bit sad

Now that all this has been done, the systems were created, the profit is building up... I feel a little bit sad. Sad about the fact that this knowledge cannot be shared because you're all my opponents. And the wealth that can be produced from this is all depending from the wealth you guys are losing. So here I am teaching and preaching but the truth is that I actually don't really want you to learn or else I could not make any profit any more.

http://www.youtube.com/watch?v=JjdDq0IqBDc

In fact, a trading (public) journal is a contradiction, because, being a zero-sum game, I am not supposed to be teaching anyone anything about what I am doing. And when I write this stuff, deep down inside, I hope that my trading systems will not be understood by others. So it could be a good journal, pleasant to read, but hopefully it's not something that will teach anyone anything.

http://www.youtube.com/watch?v=BagUvGGO0DM

But maybe I am not sad about that. Maybe I am sad about the fact that, after all this work, I still have not made a dollar (because that's our arrangement), and even when I will receive my share, it will still take ages before we build up to... retirement.

Yeah, maybe I am being impatient. That's what killed my account. And maybe things are working out because on the other side there's people who are being patient instead.

We'll see how long my patience/impatience will last. If I can wait another scaling up round (months away), then my monthly profit will exceed my banking salary. When i'll see that happen, I might probably calm down. The good thing is that, despite arguments, impatience, anything, we just keep going and going and this can only be good.

If I had been on my own, things would be worse. Let's just be happy with that.

 

[Yamato]

studying the six new futures

All right, let's move forward and leave these worries behind. Something good will come out of this.

Resuming from here:
http://www.trade2win.com/boards/trading-journals/85510-my-journal-2-a-219.html#post1496418

Work to do:

STOCK INDEXES (FUTURES)
check correlation to ES and CL for...
E-mini Nasdaq 100

http://futuresource.quote.com/quotes/chart.action?symbol=ES+1%21&compareTo=NQ+1%21&topsyms=ES&month=1%21&year=2011&chartAggregation=V&chartStyle=BAR&chartSize=745x550&chartDensity=LOW&userStudies=&x=25&y=13&chartMinutes=15
http://futuresource.quote.com/quotes/chart.action?symbol=ES+1%21&compareTo=NQ+1%21&topsyms=ES&month=1%21&year=2011&chartAggregation=V&chartMinutes=15&chartStyle=BAR&chartSize=745x550&chartDensity=HIGH&userStudies=&x=22&y=9



Sorry, fellas. This dude is too correlated to the ES. Not even need to check the correlation with CL. It all ends here. We're looking for markets to diversify in, not for markets as close as these two. Good thing I checked again. But, once again, I am discarding the Nasdaq.

And now we're left with 5 more futures to go.

MEDIUM-TERM INTEREST RATES
check correlation to ZN and between each other for...

5 Year Treasury Note
Sorry, fellas, but this is almost identical to the 10 Year Note, just as I expected:
http://futuresource.quote.com/quotes/chart.action?symbol=ZN+1%21&compareTo=ZF+1%21&topsyms=ZF&month=1%21&year=2011&chartAggregation=V&chartMinutes=15&chartStyle=BAR&chartSize=745x550&chartDensity=HIGH&userStudies=&x=29&y=10

2 Year Treasury Note
Too close, not as close, but too close, once again:
http://futuresource.quote.com/quotes/chart.action?symbol=ZN+1%21&compareTo=ZT+1%21&topsyms=ZT&month=1%21&year=2011&chartAggregation=V&chartStyle=BAR&chartSize=745x550&chartDensity=HIGH&userStudies=&x=45&y=9&chartMinutes=15


And now we're left with 3 more futures to go. I am really glad that i got out of the way some useless futures, so I didn't waste any energies. This is what I am best at: getting to the point quickly. Some people never get to the point. That's why I don't need the office improvement team around me for two months. These guys are required by the bank to take 2 months to solve our problems: if there are no problems, they will invent some. If there's problems that can be solved in 2 days, they will make up some more bull**** so they can stay two months. If there's problems left after 2 months, they will pretend there aren't. The whole method is wrong, and the people involved with this, are dishonest.


SHORT-TERM INTEREST RATES
check correlation to ZN and find out what the hell it is for...
Eurodollar
http://en.wikipedia.org/wiki/Eurodollar
Wow, I understood very little of it.
http://www.cmegroup.com/trading/interest-rates/stir/eurodollar_contract_specifications.html

Wow, careful not to confuse it with the LIFFE one, as the one i want is called "GLOBEX Euro-Dollar" on IB's margins page:
http://www.interactivebrokers.com/en/p.php?f=margin

Symbol for IB TWS is GE and overnight margin is about 900 dollars.

Let's see what it's correlated with... is there a LIBOR future? Does it have more volume? No, it doesn't exist. How about Euribor? Let's see.

Listen - it seems the Euro-dollar is inversely correlated with the ZN, and that it has no volatility basically - that is why it has a low margin:
http://futuresource.quote.com/quotes/chart.action?symbol=GE+1%21&compareTo=ZN+1%21&topsyms=GE&month=1%21&year=2011&chartAggregation=D&chartMinutes=&chartStyle=BAR&chartSize=745x550&chartDensity=HIGH&userStudies=&x=42&y=12

These charts suck but there seems to be no correlation to nothing.

OK, TEST PASSED SO FAR. I WILL KEEP STUDYING THIS.

No, wait...
http://www.wikinvest.com/futures/Eurodollar_futures

I am reading this and it seems like it's entirely based on fundamentals. Also, I don't like the way the chart looks.

Ok. I have asked questions left and right. I am now down to 3 futures. I can't afford to discard this one unless it's really the case.

This is the section on t2w where I can search on this subject:
http://www.trade2win.com/boards/money-markets/

Ok, here's one guy who developed an ATS on eurodollar:
http://www.elitetrader.com/vb/showthread.php?s=&postid=2389062&highlight=eurodollar#post2389062

On stockcharts.com here's the chart:
http://stockcharts.com/h-sc/ui?s=$XED

Ok, I've written a post here:
http://www.trade2win.com/boards/money-markets/121966-eurodollar-system.html

I've written private messages asking questions to forum members. I will wait till monday to see it trade live on TWS. I think this will be a good candidate for a system. I will know after another week of research. Now let's move on.


METALS
check correlation to ES, EUR, GC and among each other

Silver
Far enough from ES:
http://futuresource.quote.com/quote...sity=LOW&userStudies=&x=37&y=11&chartMinutes=

See below for distance from Copper

Wow, gold and silver are quite close:
http://futuresource.quote.com/quotes/chart.action?symbol=SI+1%21&compareTo=GC+1%21&topsyms=GC&month=1%21&year=2011&chartAggregation=D&chartStyle=BAR&chartSize=745x550&chartDensity=LOW&userStudies=&x=42&y=8&chartMinutes=

Ok, still far enough.

Copper
not too correlated with gold:
http://futuresource.quote.com/quote...550&chartDensity=MEDIUM&userStudies=&x=51&y=7

nor to silver:
http://futuresource.quote.com/quote...5x550&chartDensity=LOW&userStudies=&x=24&y=10

inversely correlated with ES:
http://futuresource.quote.com/quote...5x550&chartDensity=LOW&userStudies=&x=30&y=10

----------------------

Summary

Most likely all three futures will pass. I still have a doubt about Eurodollar. Most likely, I'll be able to get 20 good systems out of these 3 new futures, bringing the total to just a little lower than 100.

Even though it is just three futures, it will still take a couple of months to get everything done.

 

[Yamato]

more on eurodollar

Continuing from above.

I've asked around and I was told that the eurodollar has to be traded a bit far away and not the closest month.

Also, it was suggested to me that it moves a bit like the JPY, indirectly suggested. Amazing resemblance:
http://futuresource.quote.com/quote...x550&chartDensity=HIGH&userStudies=&x=61&y=11




Let's now compare its expirations to one another:
http://futuresource.quote.com/quotes/chart.action?symbol=GE+M1&compareTo=QED+U1&topsyms=QED&month=U&year=2011&chartAggregation=D&chartStyle=BAR&chartSize=745x550&chartDensity=HIGH&userStudies=&x=36&y=16

Look, it's true. The contract in June (M) moves less than the September one (U):



However I won't be able to do this if I backtest on disktrading.com data, because I will be given the next contract (in this moment June for example) and so I can't automate a system different from the one I back-tested.

Wow, no wonder they say to trade a further expiration and no wonder volume is so high for those:
http://futuresource.quote.com/quotes/chart.action?symbol=GE+Z2&compareTo=QED+M1&topsyms=QED&month=M&year=2011&chartAggregation=D&chartStyle=BAR&chartSize=745x550&chartDensity=HIGH&userStudies=&x=28&y=3

Check out the difference between June 2011 and December 2012:



Check out volume for the various expirations, on page 16:
http://www.cmegroup.com/daily_bulletin/preliminary_voi/VOIREPORT.pdf



The really big volume is not where we would expect it, which is next June, but a whole year from now. I wish I could test this thing on the contract a year ahead with automatic rollover, but i don't think this is possible. This would be a bummer as they say.

But wait. Let's read how they pick the roll over method... yes, got it!

Futures data in our database is in continuous format - there is one file per
symbol named after the currently most active contract and containing data
from the previous contracts for the time they were most active. We do this
by always collecting data for the most active contract in one file and
closely comparing the trading volume to the volume of the forthcoming
contract. When the new contract becomes more active than the current one, we
"rollover" by re-naming the file after the new monthly contract and resuming
data collection for the new one, because it is now most active.

We don't do any price adjustments on the rollover dates.

This is just perfect for me: I will be trading a year ahead and will be testing a year ahead, because that is indeed the most active contract.

I will definitely get started on these 3 new futures: eurodollar, silver, copper. I might take a week off - due to work anyway - and, before asking disktrading for the data, I will rest my eyes for a little while.

WARNING: Eurodollar dropped with explanation here:
http://www.trade2win.com/boards/trading-journals/85510-my-journal-2-a-220.html#post1496944

 

[Yamato]

some correlation with the euro

I was looking at the equity curve and all my correlation work:



I think I can really say that my systems profitability is correlated with the EUR going up, or rather the systems make money when the dollar goes down (which is the same thing). Then one could say that trading going long on the EUR is equivalent to trading my systems. But the good part is that when the EUR goes down, they lose less than they make when it goes up.

But when you look at the chart above, you must remember that we started with about 10 systems, some of which suck, then gradually dropped 5 of them, and gradually added 11 of them, bringing the invested capital from 15k to 30k. So everything is evolving and the past correlation might decrease in the future. I don't care if that chart is not a scientific representation of the past live performance of the present portfolio - yes, that chart has some limits. But we're all proceeding by guesstimates to some extent. And I am not going to throw the baby out with the bath water because i'd be losing important information (like someone who recently said to me my scatter plot was "rubbish" only because he had misunderstood my work and expected it to represent something else).

 

[Yamato]

thinking about other markets

I've realized that the basket of futures is so perfect that maybe some agricultural products should actually be added, while i am in this selection process.

With the present situation, I'd be trading 3 interest rate futures...

...wait, I just thought of something and double-checked correlation between ZN and eurodollar June 2011, and I found out something terrible:
http://www.trade2win.com/boards/trading-journals/85510-my-journal-2-a-220.html#post1496808



It is almost identical to the ZN. Now, since we dropped the Nasdaq for being too close (even less than this one) to the ES, then it doesn't make any sense to spend time and work on a future that moves like another one. So the eurodollar is officially dropped as well.

Now, here's the present situation. I'll be trading these futures: 2 stock indexes, 2 interest rates, 2 energies, 6 currencies, 3 metals.

This is just perfect but I now need to think about adding those agricultural ones, or at least really make sure it is hopeless. Which ones are they and what's the deal with them?

Sugar #11

Corn

Soybeans

Wheat

Let's check out the volume first of all once again:
http://www.cmegroup.com/daily_bulletin/preliminary_voi/VOIREPORT.pdf
http://www.futuresmag.com/Issues/2010/March-2010/PublishingImages/2010TradersViewChart.pdf

Volume seems ok, so let's review what was stopping me from using them, one by one:

ICE's Sugar #11, SB at ICE (aka NYBOT)
from:
http://www.trade2win.com/boards/trading-journals/85510-my-journal-2-a-71.html#post1107792

It only trades half of the time. Did you know that? Forget building a system on it. It's got all its weird expirations, and it only trades half of the time, it has lower volume than the others... forget about it.

I am not sure this is good enough anymore. I need to investigate more about sugar at ICE.

CME's Corn ZC and Soybean ZS
from:
http://www.trade2win.com/boards/trading-journals/85510-my-journal-2-a-66.html#post1100610

Ok. I have worked days and now three hours for this result. Looked up about 30 web sites and several forums, and I have come to the conclusion that, if I want to trade Corn and Soybeans, I need to buy the data from one of the above data vendors, and be willing to pay hundreds of dollars for each symbol. I cannot do any back-testing on the 5 hours I got from disktrading.com. Usually I am happy with them, but in this case, I guess my usual 15 dollars per symbol won't do it.

So far the only problem seems to be the hours for which they trade or the hours for which I have data. I really should get around this problem, maybe with the help by investors or by other people reading the journal. Now, why did I discard Wheat?

I don't know. I searched but I never wrote anything about it. It's traded at CME - ECBOT and its symbol is ZW:
http://www.cmegroup.com/trading/agricultural/grain-and-oilseed/wheat_contract_specifications.html

It's the one with the least volume and that is why i had discarded it. I know sugar #11 is fine, but I am not subscribed to ICE right so I'll wait to study it in detail (first I need the data to backtest it anyway). Here's the situation for the other three:



This is great news because I have finally found out that I do indeed need to add these 4 guys: soybean, corn, wheat and sugar.

I would be tempted by Cotton as well:
https://www.theice.com/marketdata/reports/ReportCenter.shtml?reportId=26

No, wait... ridiculous and sleazy. These guys are listing the volume for all months put together so that sugar shows 93,198 and cotton 41,765, but if you look on TWS, then things change and volume is less than half of that.

I will discard all ICE futures merely based on the problems this exchange gives me with all its tricks and deceiving menus.

So we're left with these three from ECBOT: ZC, ZS, ZW. I really have to get these done, in terms of data. Screw sugar and ICE-NYBOT.

One peculiarity of these 3 futures is that volume is, more than anywhere else i've seen, focused on those 5 intraday hours:



Hours for which I actually already have the data, but I don't consider it good enough because I absolutely need the remaining hours, so I can back-test my systems properly on the data.

This is just pefect now. I will have systems on:
2 futures each for: stock indexes, interest rates, energies,
3 futures each for: metals, agriculturals
6 futures for: currencies

Now: I will not move forward to any other phases until I will have gathered all the data for the two additional metals (last thing to do) and first of all for the very problematic corn, wheat and soybean. This time I need to do these first, because i can't afford to leave them behind me again. There must be a way to find the data for it.

 

[Yamato]

the investors/advisors/auditors/inhibitors/teammates

Just a few thoughts about the investors. Sometimes I feel upset because we're not investing a million right now instead of just 30k and, as mentioned a few days ago, I wonder if I shouldn't, like in the movie Limitless, get in touch with a mobster to lend me 100k, and do this thing myself.

But then i realize that, with all their limits, the investors are not replaceable with just a loan, because they're not just investors. They are also "compulsive gambling - inhibitors" and also "euphoria-inhibitors". Since I started automated trading in June on their account I have not placed one discretionary trade. "Of course" you might say, but this inhibition from gambling would not be a given if we were talking about my account or maybe even the account of someone who didn't know about trading systems and didn't monitor me like they do. So in this respect they are not replaceable by a loan, or by other non-traders investors.

Then they're also advisors/coaches. Out-of-sample, sharpe ratio, verification of back vs forward trades, sticking to selected systems until proven wrong, profit cushion money management... all these concepts and more were taught to me by them.

And finally, but maybe I have one more later. Finally they're auditors, who verify my performance. I always said I had great systems, but then in the first six months, I did not show it. The auditors' function was very important in that it forced me to admit that only 20% of my systems were good enough to be traded. Without auditors (and investors) I would have gone on saying I had great systems, but without being able to prove and, if given a chance, I would have selected the wrong ones, as was the case with 40% of the systems I selected in June 2010.

There was another one and it's this: the sense of not being alone but being part of a team, that appreciates you. This is important in that if I were by myself, I would not make the right choices and wins and losses would both get to my head, making me go from reckless to depressed.

So, one needs to weigh all the pros and cons, and in this case the term "investors" is a misleading one, because in fact in my case they're first of all "compulsive-gambling inhibitors". Even if I had 30k of my own I should ask them to monitor me.

The cons also exist and, besides the mentioned slow pace in scaling up, I won't list them, but let's just say there's enough disagreement to have a big argument every 3 months. Also because the matters of disagreement are always the same and we rarely see eye to eye about them.

 

[meanreversion]

Re: thinking about other markets

You can get clean minute-bar data on grains going back 7 or 8 months from Interactive Brokers.

Beyond that, I'm not sure where you'd look. I've come across a lot of crap data in my time, and a few months of clean data is better than ten years of crap data, in my opinion.

I know I've mentioned this before on t2w, but the tick data from eSignal is appallingly bad. As such, backtesting any breakout systems will give poor results, whilst backtesting mean reversion systems will always give good results (but of course neither set of results is relevant anyway).

Obviously the unusual aspect of grains is that they disappear for an hour or two before resuming trading at 9.30am Chicago time. Also, note that corn went limit up a couple of times last week .... handle with care ....

 

[Yamato]

Thanks for the concise and very to the point and precious information.

Tickdata.com is too expensive (over 1000 dollars per symbol), but if you bought the data already, I could maybe split the cost with you, because I don't need tick data: only 15-minute timeframe. My systems are not that sensitive: as i said before, I work with a shovel rather than with tweezers, and I know and weigh all the related implications. I am aware I am proceeding by guesstimates, and I have developed a rule-of-thumb sensitivity for what can be done reliably and what cannot be done. As a consequence, knowing my limits, data reliability has never represented a problem.

If you know you're not a good shot you're not going to place an apple on your son's head and suff like that. I apply this principle to my trading. I know the limits of my precision and I don't put myself in situations where I will get hurt because of it. For example, I never ventured into high-frequency trading.

Other than my proposal, to you or other readers, to negotiate the collective purchase of tickdata.com data, I am left with a doubt from your post. And this doubt rests largely of my mentioned precision. Given that I can't follow such details as "limit up" problems, I would tend to discard anything with such problematic implications.


So Corn would tend to be placed in the trash can according to my rules, because of this "limit up" peculiarity.

Now I will do a search to find out if also wheat and soybean have the same problem - otherwise I'll have to ask you.

http://www.investopedia.com/terms/l/limitup.asp
http://commodities.about.com/od/glossary/g/limit-up-down.htm

Oh, wait, this answers my question:
Corn, wheat soybeans all close limit up Friday

But also ES can go limit up, so the real question is the frequency of limit up situations rather than which futures. Or better: the frequency of it according to each future.

Then I'll know if I can keep the 3 agricultural futures or not.

Hold on... I am getting there...

From:
http://www.cmegroup.com/trading/agricultural/grain-and-oilseed/corn_contract_specifications.html



Now I want to have a list of the limit up situations on each and every future I have considered trading. There's got to be a list of those somewhere.

Found the right thread on elitetrader.com:
How many days locked limit up/down?

Getting closer and closer to discarding those 3 agricultural futures: expensive data, bad units of measurement, bad volume, limit up problems, bad hours...

Wow, they got me scared about Silver as well:


But cme.com now doesn't list any limits for it:
http://www.cmegroup.com/trading/metals/precious/silver_contract_specifications.html

Furthermore, I don't see any limit up candlesticks for the past few months of Corn:
http://futuresource.quote.com/quote...50&chartDensity=MEDIUM&userStudies=&x=15&y=16



Getting there...

Ok, listen. I am dropping them.

Look:
http://www.cmegroup.com/trading/agricultural/grain-and-oilseed/soybean_contract_specifications.html

Soybean has a limit, too.

Wheat has a limit as well:
http://www.cmegroup.com/trading/agricultural/grain-and-oilseed/wheat_contract_specifications.html

Then, I did this search (damn server redirect - it sent me to google.it and I had to do it all over again):
http://www.google.com/search?q="corn+limit+up" 14,200 results
http://www.google.com/search?q="wheat+limit+up" 2,880 results
http://www.google.com/search?q="soybeans+limit+up" 1,020 results

Instead here it doesn't seem to happen, also considering that people talk a lot more about these other futures:
http://www.google.com/search?q="gold+limit+up" 155 results
http://www.google.com/search?q="nasdaq+limit+up" 3 results
http://www.google.com/search?q="oil+limit+up" 375 results (not so good actually)
http://www.google.com/search?q="silver+limit+up" 5 results

Ok, final choice: I am done with the agricultural futures as well. I am officially discarding ZC, ZS, ZW.

I will now build my systems merely on silver and copper and have no concerns about data whatsoever, because I know where to get these two.

Now i should get drunk to celebrate. I went from potentially 10 futures to just 2. All my selection work was very useful in that I didn't embark on a mission that could not be accomplished.

So here's what my final portfolio will look like:
2 futures each for: stock indexes, interest rates, energies
3 futures each for: metals
6 futures for: currencies

Ok, now I'll go get drunk and watch a movie. Self-congratulations once again.

 

[Yamato]

adding e-mini nasdaq 100

Continuing from above and reconsidering decision made here about dropping e-mini nasdaq 100:
http://www.trade2win.com/boards/trading-journals/85510-my-journal-2-a-220.html#post1496596

It's unbelievable - the seeds in my head kept growing and growing, and I came to this subconscious reasoning and conclusion: "I will add e-mini nasdaq 100 as well". All of a sudden, something came to my consciousness, i don't know how, and I went and checked the nasdaq's correlation with ES and it's less than eurodollar with ZN, which is why I had discarded that one. And I said: hey, I am trading ES and YM, which are almost identical, yet I found it useful because some systems work better on one and some on the other... so why am i not trading the nasdaq?

And i kept pondering... "Now that I've selected so few futures to test, (copper and silver), I can afford to reconsider my previous decision...". And all this happened within 10 seconds, at least consciously.

So, you see: it was worth it to spend a few days just wondering what futures I should start trading, rather than starting immediately with one of them, as others suggested.

And with nasdaq the cost/benefit ratio is very advantageous as it trades on CME and has no peculiarities (expirations are even less problematic than metals).

So here's what my final portfolio will look like:
2 futures each for: interest rates and energies
3 futures each for: metals and stock indexes
6 futures for currencies

Silver, copper and nasdaq were also advised by the investors, and it is quite a coincidence that I ended up selecting them, practically on my own, after this long process - but the truth is that I never would have come up with them if i hadn't been told about them by the investors, so here's another example of their function as "advisors" (cfr. two posts ago). There were six more suggested to me that I did not select: heating oil, 2 yr and 5 yr treasuries, and 3 more agricultural ones, which i actually had selected and discarded on my own as well. These 3 that were selected really kick ass. Too bad for the agricultural ones, but they have way too many problems.

With a little luck I'll be able to create 20 systems on these 2 new futures and bring my total right under 100 systems, with traded systems at about 30.

...THERE IS NO END TO WHAT WE CAN DO TOGETHER, (together)
THERE IS NO END. (there is no end, there is no end)
THE WILLOW TURNS HIS BACK ON INCLEMENT WEATHER,
AND IF HE CAN DO IT,
WE CAN DO IT, JUST ME AND YOU...

http://www.youtube.com/watch?v=ne5bKbW4rxg

...WITH A LITTLE PUSH, WE COULD SET IT OFF,
WE CAN SEND IT ROCKETING SKYWARDS.
WITH A LITTLE LOVE, WE COULD SHAKE IT UP,
DON'T YOU FEEL THE COMET EXPLODING?



That's right, and i am the rocket scientist.

I am tired - I am very tired.