my journal

[Yamato]

Ok, so we've established that I won't fix "closed". The other important information is the open, and that at least is reliable. Now I'll work on the rest, given that the system already works. But we want to really make sure that, besides "closed", it's not making other mistakes.

 

[Yamato]

Ok. I am done with the opening gap system on the ES. It trades rarely, only when the gaps are big. It only goes long. It makes a lot of money but it trades rarely. Very rarely. Like once a month, and on some years it doesn't even trade. I will have to remember to trade this system no matter what forward test profit I get, because it almost always makes money but it trades rarely, so I can't afford to miss any of its trades and I can afford to trust him even without forward testing.

Wow, it trades only once a month and some years it doesn't even trade... it's going to be pretty boring to look at. However: it's one of 50 systems. Besides, if I apply this system to all the futures I follow, this is going to trade like twice a week, which is not bad at all. All right, I am pretty satisfied.

http://en.wikipedia.org/wiki/Austin_Powers:_The_Spy_Who_Shagged_Me
http://www.letmewatchthis.com/movie-1334-Austin-Powers-The-Spy-Who-Shagged-Me#

 

[Yamato]

I am going to bed. More work on the system, all day long. I managed to simplify it so much that now it consists of just two lines of code. This way, it definitely cannot be overoptimized, and definitely has an edge if it works with so little code. Trades only about 60 times in 8 years. And for 3 years it doesn't even trade. 60/5 = 12. It trades about once a month, on the years it trades. Now I will apply it to a bunch of other futures, which will further tell me if it works and if then makes even more sense. That way I will have about 2 trades per week. Which would still be boring to use, but then I get 10 trades per day from the other systems. This will bring my trades from 50 to 52, adding a 4% of trades, and even more profit, maybe like 10%, because it has more of an edge than the other systems.

 

[wprins]

Hi again Travis. Been reading your latest posts. A few comments as they occur to me, for what they're worth:

1.) About typing as fast as you can think -- I assume you're approving of being able to type fast/touch typing, in which case I'll simply say I concur. Touch typing is something I taught myself years ago because I decided that it's as basic as reading and writing, and a computer programmer that can't type properly is about as good as a writer that can't read. (OK, I exagerate, but you get the point.) And there's sadly many programmers in our industry who are woefully bad typists.

2.) About Excel -- I've read several times now about you using Excel. I'll simply mention/suggest that you have a look at something called "Resolver One". It's a spreadsheet integrated with the Python programming language. If you don't know Python, you *reeaaaalllly* need to give it some serious attention. It's incredibly powerful in a lot of contexts. Anyway, resolver one is here and here's a stock trading thing that someone built for a Resolver one competition. Anyway...

3.) About Tradestation: Maybe you can help me with a small question I've got, if you don't mind and it's not trouble. I'm looking at converting some Tradestation indicator logic to other platforms, but I'm a little stuck at nailing down the exact formula used to calculate "smoothedaverage()" which appears to be a tradestation builtin. I've tried to google this but not found anything conclusive. If you could specify the meaning of that function in fundamental mathematical terms then I'd be most appreciative.

4.) About breaking your rule and tampering/interfering: Don't sweat it. Just decide to re-stop interfering again from this day forward. And stick to it.

5.) About the new system, good going. You've almost convinced me to build my own implementation to experiment... however I'm currently at a far more rudimentary basis than you, and want to firstly get my current system which mostly works automated so I can test it on more markets etc. Having a day job doesn't help with this of course.

Have a good weekend/week.

 

[Yamato]

I will comment on your comments as I read them. Thanks for taking the time to read me and commenting.

1) Yes, me too. I taught it myself when I was 8 and using a the typewriter (there weren't many computers around back then). I think my father encouraged me to do it. He probably bought me that manual. Thanks, father. I put stickers on the keyboard, one color for each different finger that was supposed to be used. It was great to learn this in a few weeks, and then use it for the rest of my life, being able eventually to type without looking at the keyboard, while seeing my colleagues and classmates who were using 2 fingers all across the keyboard and had to look at the keys while typing. In a way I am like a pianist now (which I studied as well, but I quit after just 3 years).

2) Very interesting, thanks. I will use it only when I need it (to speed up my excel sheet, which right now is still acceptable in terms of speed), because that'd taking time away from creating maybe 30 more systems. It's not just something I install in 5 minutes and forget about it. So for now I will apply the saying "If it ain't broke, don't fix it".

3) Prepare yourself to be disappointed by this answer. Also, keep in mind that I decided not to work on a very important function that screwed up all my systems MERELY because I didn't have the mental energy/capability to do it. Still, I will now at least take a look at this function that you wish to work on. Here it is, the "smoothedaverage" function:

Inputs: Price(NumericSeries), Length(NumericSimple);
Variables: Sum(0);

If Length > 0 Then Begin
	If CurrentBar = 1 Then Begin
		Sum = Summation(Price, Length);
		SmoothedAverage = Sum / Length;
	End
	Else Begin
		SmoothedAverage = (Sum[1] - SmoothedAverage[1] + Price) / Length;
		Sum = Summation(Price, Length);
	End;
End;

Remember, once again, that I am not good at math nor formulas but I will try to translate into English for you a code that looks like Greek to me (and I am not that good at Greek). Let me know if I can (try to) do more, or better. Here it goes, as I read it, with my comment in red. Please don't laugh at me if my explanation will turn out to be useless (as I predict it will). Don't forget that I am just a political science major who taught himself some vba. By the way, I don't believe complex averages or indicators are any good. I believe in keeping everything simple (also because I couldn't do otherwise even if I wanted).

Inputs: Price(NumericSeries)[COLOR="Red"]this should be the [I]close[/I] of the bar, which depends on the timeframe you're using[/COLOR], Length(NumericSimple)[COLOR="Red"]how many bars back you want to go in computing the average[/COLOR];
Variables: Sum(0);[COLOR="Red"]this is needed here when you're using a variable that you will later define with a formula[/COLOR]

If Length > 0 Then Begin[COLOR="Red"]the formula is processed only if the bars available are > 0[/COLOR]
	If CurrentBar = 1 Then Begin[COLOR="Red"]this part I don't understand[/COLOR]
		Sum = Summation(Price, Length);[COLOR="Red"]too complicated[/COLOR]
		SmoothedAverage = Sum / Length;[COLOR="Red"]my brain is melting[/COLOR]
	End
	Else Begin
		SmoothedAverage = (Sum[1] - SmoothedAverage[1] + Price) / Length;[COLOR="Red"]I've given up[/COLOR]
		Sum = Summation(Price, Length);[COLOR="Red"]bleah...[/COLOR]
	End;
End;

I thought I would be able to explain more of it, but I got lost half way into the code. That's how bad I am, believe it or not. However maybe it's been a blessing because it forced me to create very simple systems. In a sense I am "stupid" as far as formulas, in that I don't understand... partly ignorant as well. I am stupid but orderly. Being orderly will take you very far even if you are stupid. I think if I had been taking engineering classes back in college, everyone would have definitely perceived me as "stupid". But I took political science classes, so now I actually feel like a genius because I can write some vba.

4) Yeah, that's exactly what I've done. But the temptation now will be to break the rule once a month. You see, a big part of the problem is my lack of capital and therefore lack of trades by the systems. If I had so much capital as to use all my systems at once, I would be seeing 10 trades per day, and I wouldn't bother to add anything to it. It'd be growing very fast by itself. But I see it sitting there, unused, and not growing. You see, I could very well place plenty of trades every day, but say I place a trade on one of the many CL systems: those trades could bring me 2000 or lose me 2000. If I incur into a loss, my account will be wiped out in one week. That is why I cannot trade systems that I could actually financially trade: it wouldn't be safe. So I actually have the capital to trade all systems, but it's not safe to do so because it's too small. It's a new concept for me, still difficult to grasp. It's as if I was in front of Mike Tyson: I have the power to punch him, but it's not safe to do so, because I may incur a drawdown and be punched back, and I am not strong enough to withstand a mike tyson drawdown.

This is not said with the purpose of asking capital to anyone, because as I said repeatedly, I am still not responsible for my own actions, so I can't even handle my small capital (which I've lost over and over again even in the past year).

5) I understand. I also have a day job. My brain is melting, my eyes are hurting. They squeeze me like an orange at work, trying to get the most out of me, since everyone else just refuses to work. Yesterday I told my boss "Oh, good, finally I am done and I can relax a little", and he reacted by saying "Don't worry, there's plenty of work for you", as if he was upset that I had worked so hard that I could finally take a break. On the other hand he treats me very well otherwise. It's as if he wanted me to pretend I am busy, which I never do. If I am taking a break I let him know. If I am not busy, I let him know. The others are always pretending they are busy. So I end up doing 50% of all work, and the other 4 people do 10% each. I am exaggerating. I do 75% and they do 25% all together.

Anyway, the opening gap system, which I've been working on during the past hour, while replying to your post, actually is best when it trades an average of 2 times a year, for 3 years out of 9. So that's like 6 trades in total. I know I can trust this system and that it's not overoptimized, but it feels unnatural to run a system that may not trade for a few years. Amazingly, though, if I let it trade 120 times instead of 6, it just makes the same amount of money (and I guarantee that it's not overoptimized). If I let it trade 600 times, it doubles the money, but that doesn't make sense either to use. I mean 6 trades make 5000 dollars. And 600 trades make 10 thousand dollars. Which one would you prefer? Obvious the first one. So I am puzzled about what I should do: implement a system that trades on average once a year or what?

 

[Yamato]

Ok. Now I've moved the rarely trading opening gap ES system to the YM. It does just as well but interestingly it doesn't exactly trade on the same dates. Strange because the two indexes pretty much move together. However, that's good and it doesn't mean there's anything wrong with my testing. It means also that now I will have not one but two trades per year to keep me entertained. Now I'll extend testing to all other futures. If I'll come up with something that overall trades once a month, I'll be ok with it and implement the whole thing.

Ok, done on the CL as well. It works almost as well, with similar settings (gaps need to be an even bigger fraction of price). Now this guy trades 15 times in 5 years (some data is missing). So that'd be about 30 times in the past 9 years. Also it trades, not just more often, but at different moments than the other two systems. So now we have a total of about 45 trades in the past 9 years or so. So that gives us already 5 trades per year at different times. Very good. And we're talking about high probability and high profitability trades. I am confident that I'll be able to add another 50 trades, bringing my total to 1 trade per month, which is good enough to be trading this system.

Gold's next.

 

[Yamato]

 

[Yamato]

Ok. Gold does not work whatsoever and I can't understand why, but that doesn't take away any reliability from the other three systems. Now GBL is next.

 

[Yamato]

Everybody's talking and no one says a word
Everybody's making love and no one really cares
There's Nazis in the bathroom just below the stairs
Always something happening and nothing going on
There's always something cooking and nothing in the pot
They're starving back in China so finish what you got

Nobody told me there'd be days like these
Nobody told me there'd be days like these
Nobody told me there'd be days like these
Strange days indeed -- strange days indeed

Everybody's runnin' and no one makes a move
Everyone's a winner and nothing left to lose
There's a little yellow idol to the north of Katmandu
Everybody's flying and no one leaves the ground
Everybody's crying and no one makes a sound
There's a place for us in the movies you just gotta lay around

Nobody told me there'd be days like these
Nobody told me there'd be days like these
Nobody told me there'd be days like these
Strange days indeed -- most peculiar, mama

Everybody's smoking and no one's getting high
Everybody's flying and never touch the sky
There's a UFO over New York and I ain't too surprised

Nobody told me there'd be days like these
Nobody told me there'd be days like these
Nobody told me there'd be days like these
Strange days indeed -- most peculiar, mama

 

[Yamato]

J'avoue, j'en ai bavé, pas vous, mon amour
Avant d'avoir eu vent de vous, mon amour
Ne vous déplaise, en dansant la javanaise
Nous nous aimions, le temps d'une chanson

A votre avis, qu'avons-nous vu de l'amour
De vous à moi, vous m'avez eu, mon amour
Ne vous déplaise, en dansant la javanaise
Nous nous aimions, le temps d'une chanson

Hélas, Avril en vain me voue à l'amour
J'avais envie de voir en vous cet amour
Ne vous déplaise, en dansant la javanaise
Nous nous aimions, le temps d'une chanson

La vie ne vaut d'être vécue sans amour
Mais c'est vous qui l'avez voulu, mon amour
Ne vous déplaise, en dansant la javanaise
Nous nous aimions, le temps d'une chanson

http://en.wikipedia.org/wiki/Serge_Gainsbourg
http://en.wikipedia.org/wiki/Anggun

 

[Yamato]

GBL is working well. It works best if I let it run the other way for the first 15 to 30 minutes of trading and then bet on the closing of the gap.

Surprisingly (since, unlike the others, the GBL doesn't have an upward bias), it doesn't work if I let it go short. So, so far, all opening gap systems work only if I go long.

The ZN will be next.

So far, by testing the 4 profitable systems (GC didn't work) in the past 8 years (considering data missing, it's about 8 years on average), with the best settings, I've gathered a total of 94 trades and a combined profit of 7500+5000+3000+3000=18500. This means already 1 trade per month (94/8=12), and I have another 4 futures to go. It also means 18500/8/12=200 extra dollars a month, at practically no extra cost (capital isn't used very much in those hours). I am confident that eventually I'll have a "multi-system" (applied on different futures, which actually means 8 different systems) that'll trade twice a month, making about 400 dollars a month every 4000 dollars of capital, which means about 10% a month, which is totally acceptable. If I renounced to these 10% a month systems, I would have nothing right now. I used to try and build just one 100% a month system, but none of them worked: they were all over-optimized. So I've shifted the focus to building many 10% a month systems.

Many's the times I've been mistaken
And many times confused.
Yes, and I've often felt forsaken
And certainly misused.
Oh, but I'm all right, I'm alright,
I'm just weary to my bones,
Still you don't expect to be bright and bon vivant
So far away from home,
So far away from home.

I don't know a soul who's not been battered,
I don't have a friend who feels at ease,
I don't know a dream that's not been shattered
Or driven to its knees.
Oh, but it's alright, it's alright,
For we lived so well so long,
Still, when I think of the road we're trav'ling on,
I wonder what's gone wrong,
I can't help it, I wonder what's gone wrong.

And I dreamed I was dying.
I dreamed that my soul rose unexpectedly
And looking back down at me,
Smiled reassuringly.
And I dreamed I was flying,
And high up above my eyes could clearly see
The Statue of Liberty,
Sailing away to sea,
And I dreamed I was flying.

We come on the ship they call the Mayflower,
We come on the ship that sailed the moon.
We come in the age's most uncertain hours
And sing an American Tune.
Oh, and it's alright, it's alright, it's alright.
You can't be forever blessed.
Still tomorrow's goin' to be another working day,
And I'm trying to get some rest,
That's all I'm trying, to get some rest.

 

[Yamato]

 

[Yamato]

http://en.wikipedia.org/wiki/Hoagy_Carmichael
http://en.wikipedia.org/wiki/Ned_Washington
http://en.wikipedia.org/wiki/Norah_Jones
http://www.youtube.com/user/shadavmim
http://en.wikipedia.org/wiki/The_Nearness_of_You

 

[Yamato]

http://en.wikipedia.org/wiki/Ennio_Morricone
http://en.wikipedia.org/wiki/Sergio_Leone
http://en.wikipedia.org/wiki/Once_Upon_a_Time_in_America

 

[Yamato]

Ok, done on the ZN. I'll test the forex futures next weekend.

The ZN works as well, but with slightly different settings. For one thing, the optimal gap required is larger, almost twice as much as the GBL's. That's fine, because I would have expected the ZN to have a wider range than the GBL. The other thing is that whereas the GBL and all other gap opening systems close their positions very quickly, after a little over an hour. The ZN instead likes to keep its position open all day long. Which I am skeptical about, but tests prove that it makes a lot more money, and increasingly so (as the hours increase, the profit gets better: a sign of non-randomness, I think), so it seems reasonable to let it do it. Besides, it's good to vary among systems, so they can't all be wrong at once. Capital-wise, the fact that the ZN stays open for so long is the least of problems because it is the future requiring the lowest margin.

The last thing I need to mention (mostly to myself) is that the "closed(1)" function issue may be very relevant for this system, because the ZN does move quite a bit in those 8 hours that the closed(1) function gets wrong (it polls the close at 23.45 EST, instead of 4 PM EST).

Other than this, this ZN guy only goes LONG, like everyone else in the opening gap business, and makes about 4500 dollars in 9 years and just 34 trades, so I can't really throw it away.

The other systems made 18500 in 94 trades, so if I add everything, I get 23000 in 94+34=128 trades. And I am already trading more than once a month. ****ing closed(1) problem... damn it... screw it... the idea of not dealing with it keeps bugging me. I'll ask someone on elitetrader or google it.

Anyway, other than that problem, everything is good, and I will test this thing on the other forex futures next weekend as I said.

 

[Yamato]

http://www.elitetrader.com/vb/showthread.php?s=&threadid=180385

CloseD(1) problem entirely solved. However I will now have to re-optimize all my systems. I will postpone doing it on all 37 because half of those don't even use CloseD, and the others are not that sensitive to differences. But I definitely will have to re-optimize all 5 finished opening gap systems in light of this new method to solve the CloseD(1) problem (by setting a different schedule for the symbols you test). All this will take place next weekend.

 

[wprins]

Inputs: Price(NumericSeries), Length(NumericSimple);
Variables: Sum(0);

If Length > 0 Then Begin
	If CurrentBar = 1 Then Begin
		Sum = Summation(Price, Length);
		SmoothedAverage = Sum / Length;
	End
	Else Begin
		SmoothedAverage = (Sum[1] - SmoothedAverage[1] + Price) / Length;
		Sum = Summation(Price, Length);
	End;
End;

That's perfect thanks - I can read that and it's unambiguous. :clap:

Have a good week.

 

[Yamato]

Well, I am glad it was useful, because I understood almost nothing of it.

 

[Yamato]

Ok, back at work. Too much slouching at the movie theater has given me a backache, so I am trying to sit straight up today. My chair sucks though.

However, right now I am in a good mood overall. The boss has been asking me for easy tasks on excel that I can take care of in 5 minutes. Then another guy, also with a high-ranking position, asked me for pivots on his excel database, which I've built but that he has to fill with data (the most tiring part of the job). Everything I did today was on excel.

It seems like everyone coming to me for help has a higher ranking position than me, gets paid 4 times as much as me, but they do the manual labor, and I am asked to do the intelligent and quick part. They do quantity and I do quality. Also because they've got to do something, and since they can't do the hard stuff (which is easy for me, and takes me little time), they're stuck with the easy and tiring and repetitive stuff. Everything is reversed. This is not the way things should be. I either should not have a low ranking position, or they shouldn't a high ranking position. Also, I am unhappy that I don't get to have a single, like some of them.

Instead I am stuck with annoying and noisy co-workers that keep me from focusing, and that's why eventually I will flip and quit my job. If I ever reach what... 200 thousand, I will quit immediately.

Overall good mood though. Also because yesterday, unexpectedly, I solved the CloseD(1) problem. You just limit the future contract schedule, to get rid of the non RTH trading - especially since I don't need it for the opening gap systems. I think the gap size requirement will need to be increased. Indeed, if I use the close at 4 PM EST rather than midnight, gaps will tend to be bigger (prices will have twice as much time to move away from the close), and those 30 awesome trades that my filter was allowing will turn into 60 so so trades. So I need to increase that value, most likely.

Now imagine if I hadn't fixed this CloseD(1) tradestation problem: I would have had back-testing with 30 very good trades, and real trading with twice as many trades and not nearly as good. Pretty poor testing, isn't it? Pretty poor programmer, too. And yet, amazingly, notwithstanding my limited quality and precision (despite the fact that I'm doing my best), I've been getting profitable months ever since I started automated trading two years ago. This tells you how much of an edge automated trading has over discretionary trading. A crappy automated trader will make money from the start (like in my case), whereas a discretionary trader who's been trading for 12 years may still be unprofitable (like in my case).

 

[Yamato]

Home finally. Got some good news as well. I don't feel like trading discretionary, despite my relapse on friday. Also because I am making some money today on an automated trade.

Why can I hold my peeing for so long, without even noticing I have to urinate, and then suddenly once I get home I feel the irresistible urge to pee immediately, or even in my pants by how bad I need to pee? It must be because I stop holding it, since I know the toilet is so close. In the same way, when I feel I'll soon quit my job, the days become unbearable, whereas I didn't seem to mind work until the day I decided that I would quit. I think the same applies to every activity you don't like: if you decide you'll put up with it, you can. If you decide you won't put up with it, it becomes unbearable. The only thing changing is your mind, and sometimes you don't even realize it and actually think that the heat became unbearable because it got really hot. Instead it often becomes unbearable just because you notice it.

Anyway, today and for this whole week no testing. Too tired from work. I used to be able to stare at my computer 16 hours a day, but lately my eyes have started to hurt. And now it's definitely easy to choose to stay away from my computer, despite many things leading me to it. Now I'll go rest my eyes in front of the television screen. I am serious. It is not as bad. Until the day I'll quit my job and live by the sea.