more systems to "fix"
My next little project, for today, is to find a few more systems to work on. Not necessarily systems that have exceeded their drawdown, but systems that have failed to deliver and that also trade a lot: above 150 trades per year.
I cannot accept from such systems to have delivered anything less than 1000 dollars. They can't stay at break-even for 300 trades. That's unacceptable.
Those will be the systems I will fix (most likely) on top of the list I've established here:
http://www.trade2win.com/boards/trading-journals/85510-my-journal-2-a-230.html#post1546004
CL_ID
CL_ON_4
EUR_ID_3
EUR_ID_6
GC_ON_2
JPY_ID
JPY_ID_2
JPY_ON
ZN_ID_2
Once I'll find these new systems to fix, I will get to work on all of them at once, so finally I'll get to the final automation part.
[...]
So, ok. Back from watching tv.
I need to select systems that have traded at least 50 times. That is the first thing. There's 22 of them.
Then I need to focus only on those that have made less than 2000 dollars of profit. There's 13 of them.
Ok, let's see what they are:
CL_ID
CL_ID_2
ES_ID
ES_ON
EUR_ID_2
EUR_ID_3
GBP_ID_4
JPY_ID
JPY_ID_2
JPY_ON
YM_ID
ZN_ID
ZN_ON
Of these, some were already included in previous selection of systems to be fixed. I am going to get rid of them, and produce a new shorter list to be analyzed:
CL_ID_2
ES_ID
ES_ON
EUR_ID_2
GBP_ID_4
YM_ID
ZN_ID
ZN_ON
There's 8 of them. Let's see their trades and profit:
HTML:
system Profit trades made
CL_ID_2 1665.5 59
ES_ID -1465.4 123
ES_ON -229.6 152
EUR_ID_2 -1563 90
GBP_ID_4 1325.45 69
YM_ID -1014.2 170
ZN_ID -240.415 74
ZN_ON 825.605 362
Each one of them has to be studied in detail to see if it can and should be fixed. By the end of today I will study them and post my thoughts here. The rest of the work will be done in the next few days.
Let us remind ourselves once more that
these systems have not failed yet. They have
not exceeded their previous maximum historical drawdown and in some cases they're even profitable. So what we first have to really make sure that, after so many trades, we're fixing them because
they have failed to deliver and failed in performance compared to what we expected of them.
Not surprisingly, all 8 disapponting systems are from before I learned to use the out-of-sample testing methodology. This is in part because I didn't allow the recent systems to be examined (and the out-of-sample was only used since 10 months ago). And it is also because the new systems didn't place enough forward-tested trades yet. But my estimate is also that the systems created with the new method (the out-of-sample) will fail less than the ones I created without it.
[...]
Back.
I am going to study the situation of those 8 systems now. What do I need? I need the report and the forward-tested trades, to be compared together.
Let's get started. This is more fun than watching tv anyway.
CL_ID_2
Holy cow. This is a perfect example of a system that has not exceeded drawdown in forward-testing (only because we started in July 2009) and yet it has exceeded it by far in the out-of-sample - which I did not use in my testing. Look (the red line divides the end of my all-sample testing):
This is crap. I either fix it, or I'll keep it as it is. But I hope to fix it. It was crap to begin with, which doesn't make things great.
ES_ID and YM_ID (same system on similar futures)
These are all right, but I want to work on them to make them excellent. WIth the knowledge I have now, I can do it. What I mean to say here is that any time we have a frequently trading system like this one that makes little money, we can turn it, with an extra filter, into an excellent system that trades half as many times.
ES_ON
This is the twin of YM_ON which is excellent. I need to keep it as it is to monitor the not yet explained differences in performance between the two systems trading very similar futures. The one on YM made 3600 and the one on ES made 200 dollars. Of course we're trading the one on YM, but I need to keep them as they are to monitor this phenomenon.
EUR_ID_2
Here's another one that trades a lot and that has not failed and can be improved.
But hold on... if I fix these systems that still have not exceeded their past drawdown by 1.5 times, then I would even get busy fixing many other systems in the future before making sure they have failed.
You know what? I am going to keep all these systems except those that have blatantly failed. Also because I have plenty of work on my hands already.
SO, REVIEWING ALL MY CHOICES OF THIS POST:
CL_ID_2 has blatantly failed and so it has to go. The others have not failed yet, so they stay.
Basically it was simpler than I thought. I am merely looking at the out-of-sample which results of the period I didn't have available when I built these systems by using up the whole sample (to optimize them on).
Three more to go:
HTML:
GBP_ID_4 1325 69
ZN_ID -240 74
ZN_ON 826 362
After all this work, we need a music break:
GBP_ID_4
This is a special case in that it has not failed but it has had a big drop in 2008 which could be avoided by increasing the range requirement, and raise it as high as the one used for the twin systems EUR_ID_4 (the same code but with a higher daily range requirement). So I will fix it just that way, and keep the trades it made in forward-testing but I will add a note to it.
This is how it looks like with the original code:
This is the way it looks like with a small change in range:
Definitely worth changing, all the more so since that way it will be identical to EUR_ID_4. The trades decrease by 60% and profit decreases by 20%. I'll take it.
Now the two ZN. I am tired.
ZN_ID
I am still stressed out about work, and still upset and offended that i've had to deal with such crap and demeaning bull****. Why am I in a situation where I have to fight for such basic things as good work and things being done orderly? Why am i in a place where people, in order to kiss up to the bosses, get so low as pretending they appreciate the work of a stupid retarded lazy bitch sent to us by management? How can i still be in a place like this? Why did I achieve so little in life to have to be around such stupid lazy despicable individuals? Answer: compulsive gambling. Without the compulsive gambling of the past two years I would not be at the bank now. I would not be around such despicable people. Stupidity is their number one problem I despise, second one is laziness, and third one is a low sense of duty. You should try to get promoted by hard work, not by kissing up and lying and pretending something is good and well done when it is not, and yet things at my place work just the other way: people who pretend and kiss us and lie get promoted, and I get treated like I have some problems. What problems? I can't lie and pretend?
Anyway, back to
ZN_ID
Here's another system I cannot touch, because it hasn't failed at all. It did exceed the max historical drawdown but by less than 1.5, so if it hasn't failed on the chart, it hasn't failed on the drawdown point of view... and there's no immediate fixing it, then I have to keep it. Look:
Actually, if anything, dude, this is an excellent one, despite it having been unprofitable so far. No way that i am changing it. We had even selected it for real trading for a few months.
And now, finally:
ZN_ON
Ok, this dude has nothing wrong at all, but it simply has not made money in the past two years, and it's not looking bad either on its equity.
However, since it trades almost every day, I can easily find something that fixes it and makes it trade once a week and make a lot of money instead of trading every day and breaking even for two years in a row. This is the real pearl because it provides a lot of trades for free basically and every day. And if I can add a magic filter to it that adds a little bit of an edge - this is going to be excellent. But I have to be very careful here to not abuse the out-of-sample because by now I now it quite a bit. I must not find a solution that works only because it knows what the out-of-sample is like. I mean, I have to find something on the in-sample and only get to try once if it works or not, preferably. I can't just try adding to it 10 different filters that work on the in-sample and then pick the one that also works on the out-of-sample, because that would be totally cheating.
So... the summary of all this writing and thinking (it helps me to work harder if I post it here), is that I will now have to fix precisely 9 failed systems (appraised yesterday) and 3 other non-failed systems but which could greatly benefit from my fixing. Any system that won't get fixed will keep being traded just as it is. I will now if the fixing works from the out-of-sample verification.
Total systems to ultimately automate or re-automate is now going from 40 to potentially 53. Yes, we can. I was at merely 40 systems when i started this journal, and now I am going to add 53 systems in just a matter of two months. Nothing can stop me now. Just a possible killing rampage at the office (by me).
Both for working on the systems and to avoid a killing rampage which would land me in jail, I need to do this:
1) leave at 3 pm according to my part-time schedule. This is something i have not done in a long time thanks to the "efficency" team, which has slowed me down by an average of 3 hours per day.
2) **** the boss. Basically if he asks me to make any changes, I have to make him understand that he has to remove me from my position, and I won't make any worsening "improvements" just because he's kissing up to the management and listening to the efficiency team. Today I told him he can give my responsibilities to someone else if he wants to screw my work of 4 years. I won't be the one destroying my own work. If he is so stupid and so butt-kissing that he would rather screw the efficiency than tell these idiots to get lost, then he can put someone else in charge of my work and I'll mop the floor instead. There's no way I am undoing the good work I have done in the past years to please some retarded guys who don't know what they are doing and saying. He's an idiot. I've realized my boss is an idiot, today. I am leaving at 3 pm every day from now on.