For how long are you looking for? I can ask. I know there's a bund trader at his fund, who shouts and screams every time eurjpy falls 15 ticks and bund hasnt moved up yet.
- Status
For how long are you looking for? I can ask. I know there's a bund trader at his fund, who shouts and screams every time eurjpy falls 15 ticks and bund hasnt moved up yet.
I thought about that L after, but it confuses me that the L in GBL stands for Long term and L in CL Light.
Yamato
Legendary member
- Messages
- 9,840
- Likes
- 246
FGBL data I need
I use tradestation 2000i and I would like to have an ascii .txt file on a 15-minute timeframe for the last 10 years or so (or whatever is available of course).
It would look like this:
I use tradestation 2000i and I would like to have an ascii .txt file on a 15-minute timeframe for the last 10 years or so (or whatever is available of course).
It would look like this:
Re: FGBL data I need
I know he uses tradestation and something called EasyLanguage. Cant promise anything, but i will ask.
I know he uses tradestation and something called EasyLanguage. Cant promise anything, but i will ask.
Adamus
Experienced member
- Messages
- 1,898
- Likes
- 97
Re: back from work, back to work
I thought FMT was a black box, yet you backtested the idea - you mean you coded something up and tested it, or you bought the indicator and tested it? I haven't worked out what it is yet, since I haven't had time to try, although it might be better value in terms of time expended just to pay the money for it, as long as it's explained so I can re-code it in NinjaMutantTurtleTrader. It sounds like you've used it already, or am I misunderstanding you?
I pointed it out on the journal here because I thought Travis might have done the same as you - lucky you're reading here. I haven't looked at the other two you mention.
I thought FMT was a black box, yet you backtested the idea - you mean you coded something up and tested it, or you bought the indicator and tested it? I haven't worked out what it is yet, since I haven't had time to try, although it might be better value in terms of time expended just to pay the money for it, as long as it's explained so I can re-code it in NinjaMutantTurtleTrader. It sounds like you've used it already, or am I misunderstanding you?
I pointed it out on the journal here because I thought Travis might have done the same as you - lucky you're reading here. I haven't looked at the other two you mention.
Yamato
Legendary member
- Messages
- 9,840
- Likes
- 246
humongous discoveries: dropping 5 systems out of 10
Awesome work done in the last few days.
Back-testing and comparing of forward-trades (done on excel+tws) with back-tested trades (done on tradestation).
This lead me to dismiss some of the systems I thought were better than actually were.
For example: did you know that the gaps in the last few years have not been filled as quickly in the first 2 hours of trading on the ES-YM?
It's a fact. The Opening Gap systems, that caused us almost 2000 dollars of losses in our traded period, were not what I made them out to be. They've only lost money in the last 2 years. Only I didn't know it, because I only bought the latest ES-YM data a week ago.
Out of 10 systems, due to either "back-testing vs execution" discrepancies or to the fact that a system has not worked in the out-sample, I have considered dropping 5 of them. And we're talking about dropping 5 out of the supposedly best 10 systems I had!
Quite depressing, but also relieving to have found out.
We replaced the CL_ON_2 with the much better CL_ID_3, as I had already suggested here:
http://www.trade2win.com/boards/trading-journals/85510-my-journal-2-a-152.html#post1270894
We might or might not replace the GBP_ID_3 with the GBP_ID_5, because the former sucks in that it just loses in the 2-years out-sample and doesn't match in the "back vs forward" comparison and so we're dropping it, but the latter, despite being great in both forward-testing, back-testing and out-sample, also doesn't match as far as the "back vs forward" comparison. This is due to the mentioned tradestation "session end" bug, which despite renewed efforts I have not been able to solve yet, and probably never will. I would trade because it's good and it will make money - I know it for sure - but the investors might say no, because it's not faithful enough to the back-testing.
I feel like I know my systems like the back of my hand, but I have to admit to myself that dropping 5 out of 10 means it was just an impression. You will never know your systems well enough. Especially when you've got dozens of them.
So, recapitulating:
1) 3 Opening Gap systems most likely disabled (waiting for investors' ok) because they basically stopped working, at least on the ES-YM (I don't have the GBL data yet to check the GBL_ID_2).
2) GBP_ID_3 dropped and maybe replaced by GBP_ID_5 (waiting for investors' ok).
3) CL_ON_2 replaced by CL_ID_3.
And now I'll try to relax to avoid having too much bags under my eyes tomorrow.
Awesome work done in the last few days.
Back-testing and comparing of forward-trades (done on excel+tws) with back-tested trades (done on tradestation).
This lead me to dismiss some of the systems I thought were better than actually were.
For example: did you know that the gaps in the last few years have not been filled as quickly in the first 2 hours of trading on the ES-YM?
It's a fact. The Opening Gap systems, that caused us almost 2000 dollars of losses in our traded period, were not what I made them out to be. They've only lost money in the last 2 years. Only I didn't know it, because I only bought the latest ES-YM data a week ago.
Out of 10 systems, due to either "back-testing vs execution" discrepancies or to the fact that a system has not worked in the out-sample, I have considered dropping 5 of them. And we're talking about dropping 5 out of the supposedly best 10 systems I had!
Quite depressing, but also relieving to have found out.
We replaced the CL_ON_2 with the much better CL_ID_3, as I had already suggested here:
http://www.trade2win.com/boards/trading-journals/85510-my-journal-2-a-152.html#post1270894
We might or might not replace the GBP_ID_3 with the GBP_ID_5, because the former sucks in that it just loses in the 2-years out-sample and doesn't match in the "back vs forward" comparison and so we're dropping it, but the latter, despite being great in both forward-testing, back-testing and out-sample, also doesn't match as far as the "back vs forward" comparison. This is due to the mentioned tradestation "session end" bug, which despite renewed efforts I have not been able to solve yet, and probably never will. I would trade because it's good and it will make money - I know it for sure - but the investors might say no, because it's not faithful enough to the back-testing.
I feel like I know my systems like the back of my hand, but I have to admit to myself that dropping 5 out of 10 means it was just an impression. You will never know your systems well enough. Especially when you've got dozens of them.
So, recapitulating:
1) 3 Opening Gap systems most likely disabled (waiting for investors' ok) because they basically stopped working, at least on the ES-YM (I don't have the GBL data yet to check the GBL_ID_2).
2) GBP_ID_3 dropped and maybe replaced by GBP_ID_5 (waiting for investors' ok).
3) CL_ON_2 replaced by CL_ID_3.
And now I'll try to relax to avoid having too much bags under my eyes tomorrow.
Last edited:
Yamato
Legendary member
- Messages
- 9,840
- Likes
- 246
systems I'd take a bullet for
I think it was wprins who used to have this quote in his signature:
I don't like quoting people other than movies, but this quote stuck to my mind.
The author seems to be Joel Rensink:
http://cdn.forexfactory.com/attachment.php?attachmentid=265018&d=1245924370
That quote never applied to my trading but I am getting closer. My trading used to be "change strategy / tamper with systems as soon as you incur a loss" but now I am getting closer to being able to say that I am trading methods that I absolutely will take... not a bullet for, but at least a drawdown, a 1000 dollars drawdown.
So here's my list of good systems and my motivations.
Essentially the reasons are these two:
1) It did well in the OUT sample (from summer of 2008)
2) the forward testing is either the same (the trades are matched) or it trades better than the back-testing.
According to these two rules, I will now analyze the 7 systems (also see previous post) I would take a drawdown for.
"5 survivors" (traded from the start):
1) GBL_ID
We can skip this guy because it performed awesome in the year of forward-testing, trading almost every day. Yet if we wanted to do some back vs forward matching, I'd need new data, but no one has given it to me yet (not disktrading, not others I asked). But we can trust this guy blindly. It traded over 100 times, with great performance. This system is one of my 3 best systems.
2) EUR_ID_5
it does good in the out sample
execution matches back-testing perfectly
3) YM_ON
it does awesome in the out sample
execution matches back-testing
4) ZN_ID_2
Good out sample
Good matching
5) ZN_ON_2
good out sample
pretty bad matching - BUT forward testing trades better so i am ok with it
Replacements for 2 dropped systems:
6) CL_ID_3
This one is only in back-testing mode (OUT sample from spring 2008) because I automated it 3 weeks ago: it's my 61st and last system. But it does so well, that I am going to trust it. Very hard to find a LONG-only strategy that only loses 7k during the 75% drop of 2008.
7) GBP_ID_5
It looks really really good in the OUT sample as does everywhere else:
Forward-testing matches half of the time, and in the other half it does better:
I think it was wprins who used to have this quote in his signature:
I don't like quoting people other than movies, but this quote stuck to my mind.
The author seems to be Joel Rensink:
http://cdn.forexfactory.com/attachment.php?attachmentid=265018&d=1245924370
That quote never applied to my trading but I am getting closer. My trading used to be "change strategy / tamper with systems as soon as you incur a loss" but now I am getting closer to being able to say that I am trading methods that I absolutely will take... not a bullet for, but at least a drawdown, a 1000 dollars drawdown.
So here's my list of good systems and my motivations.
Essentially the reasons are these two:
1) It did well in the OUT sample (from summer of 2008)
2) the forward testing is either the same (the trades are matched) or it trades better than the back-testing.
According to these two rules, I will now analyze the 7 systems (also see previous post) I would take a drawdown for.
"5 survivors" (traded from the start):
1) GBL_ID
We can skip this guy because it performed awesome in the year of forward-testing, trading almost every day. Yet if we wanted to do some back vs forward matching, I'd need new data, but no one has given it to me yet (not disktrading, not others I asked). But we can trust this guy blindly. It traded over 100 times, with great performance. This system is one of my 3 best systems.
2) EUR_ID_5
it does good in the out sample
execution matches back-testing perfectly
3) YM_ON
it does awesome in the out sample
execution matches back-testing
4) ZN_ID_2
Good out sample
Good matching
5) ZN_ON_2
good out sample
pretty bad matching - BUT forward testing trades better so i am ok with it
Replacements for 2 dropped systems:
6) CL_ID_3
This one is only in back-testing mode (OUT sample from spring 2008) because I automated it 3 weeks ago: it's my 61st and last system. But it does so well, that I am going to trust it. Very hard to find a LONG-only strategy that only loses 7k during the 75% drop of 2008.
7) GBP_ID_5
It looks really really good in the OUT sample as does everywhere else:
Forward-testing matches half of the time, and in the other half it does better:
Last edited:
Yamato
Legendary member
- Messages
- 9,840
- Likes
- 246
need to add new systems: great
Talked to investors. We're not trading enough systems and the present curve looks bad: I need a few more candidates. Let's see what we're left with.
We cannot consider the new 20 (from 41 to 60), because they've only traded for a month or so and I don't trust them enough yet.
We're only left with the usual "prospective 20" systems, from which I have to eliminate some because we just discarded them or used them (see posts above).
I won't even list them and only focus on the candidates we have left. Furthermore, we get rid of the CL_ON_3 and ES_ON_2, because, from 2008, they still did not get out of the drawdown, even though they did great in the last year.
Besides for not having the historical data, I am getting rid of GBL_ON because of this (drawdown lasts for almost one year):
Wow, what a disaster. One by one I got rid of all my "prospective systems". I think I am getting a bit too strict in selecting them.
Here's what we've got left:
GC_ON
Great matching and great OUT sample with low drawdown (even though bad performance in our year of forward-testing):
ZN_ID
Ok, pretty good matching and good equity line and out sample. This is what we need:
If my candidates are accepted, we'll be trading 9 systems and the equity curve for the last year will look like this:
WARNING: Beware that the forward-testing equity line above is affected by the fact that some systems were not forward-tested from the start of the period.
For the last 8 years of back-testing (WITH the OUT sample) it looks like this:
I did not do it on purpose but this combination in 8 years of back-testing only has a 7000 dollars drawdown, which is amazing. I've never seen a smoother equity line. Even in the rough periods, it almost shows no reaction.
Any further systems should not be added before we double the contracts for these systems.
Talked to investors. We're not trading enough systems and the present curve looks bad: I need a few more candidates. Let's see what we're left with.
We cannot consider the new 20 (from 41 to 60), because they've only traded for a month or so and I don't trust them enough yet.
We're only left with the usual "prospective 20" systems, from which I have to eliminate some because we just discarded them or used them (see posts above).
I won't even list them and only focus on the candidates we have left. Furthermore, we get rid of the CL_ON_3 and ES_ON_2, because, from 2008, they still did not get out of the drawdown, even though they did great in the last year.
Besides for not having the historical data, I am getting rid of GBL_ON because of this (drawdown lasts for almost one year):
Wow, what a disaster. One by one I got rid of all my "prospective systems". I think I am getting a bit too strict in selecting them.
Here's what we've got left:
GC_ON
Great matching and great OUT sample with low drawdown (even though bad performance in our year of forward-testing):
ZN_ID
Ok, pretty good matching and good equity line and out sample. This is what we need:
If my candidates are accepted, we'll be trading 9 systems and the equity curve for the last year will look like this:
WARNING: Beware that the forward-testing equity line above is affected by the fact that some systems were not forward-tested from the start of the period.
For the last 8 years of back-testing (WITH the OUT sample) it looks like this:
I did not do it on purpose but this combination in 8 years of back-testing only has a 7000 dollars drawdown, which is amazing. I've never seen a smoother equity line. Even in the rough periods, it almost shows no reaction.
Any further systems should not be added before we double the contracts for these systems.
Last edited:
Yamato
Legendary member
- Messages
- 9,840
- Likes
- 246
Great! All my changes were okayed.
Now I can start the 9 systems.
Then I'll get drunk and go back to sleep.
My long journey to profit... hopefully I won't die before becoming profitable.
I need some music, to distract myself. I'll get drunk and play some mp3s while I post the corresponding youtube videos.
http://www.youtube.com/watch?v=SsgGQLo6MbU
Steven Tyler
Joe Perry
Dream On
http://www.youtube.com/watch?v=tXbE4x9rfds
http://www.youtube.com/watch?v=XqxYy8OOTOE
http://www.youtube.com/watch?v=hSGuMfv24xA
http://www.youtube.com/watch?v=64fILOaIbw8
http://www.youtube.com/watch?v=MrHBHT5tqiI
Anche se... non lo so, non vorrei... ma però.
No, non credo proprio che sia così.
Sarebbe comodo sì, ma io non sono come te.
Anche se... dove andró? Non saprei, o non sarò.
No, questo è un amore grande, sì.
Vuoi che ti dica così, ma io non sono come te.
Ciao, sai cosa ti dico: "ciao". Io posso stare senza te.
Senza piú tanti "se", senza tanti "ma perché?"
senza un amore cosi, io posso stare, sì.
Ciao, sai cosa vuol dire "ciao"? Vuol dire un'altra come te.
E mai più tanti "se", e mai piú nessun perché.
Ci si illude ancora, sì.
No, questo é un amore grande, si.
Vuoi che ti dica così, però io non sono come te.
Ciao, sai cosa ti dico? Ciao. lo posso fare senza te.
Senza più grandi "se", senza grandi "ma perché?",
senza un amore così, io posso fare, sì.
Ciao, in fondo basta dire anche "ciao",
io sto meglio senza te.
Senza piú tanti "se", senza tanti "ma perché?"
senza un amore così, io posso stare, sì.
Ciao, sai cosa vuol dire "ciao"? Vuol dire un'altra come te.
E mai più tanti "se", e mai più nessun perché.
Senza un amore così, io posso stare, sì. Ciao.
http://en.wikipedia.org/wiki/Being_There#Score
Now I can start the 9 systems.
Then I'll get drunk and go back to sleep.
My long journey to profit... hopefully I won't die before becoming profitable.
I need some music, to distract myself. I'll get drunk and play some mp3s while I post the corresponding youtube videos.
http://www.youtube.com/watch?v=SsgGQLo6MbU
Steven Tyler
Joe Perry
Dream On
http://www.youtube.com/watch?v=tXbE4x9rfds
http://www.youtube.com/watch?v=XqxYy8OOTOE
http://www.youtube.com/watch?v=hSGuMfv24xA
http://www.youtube.com/watch?v=64fILOaIbw8
http://www.youtube.com/watch?v=MrHBHT5tqiI
Anche se... non lo so, non vorrei... ma però.
No, non credo proprio che sia così.
Sarebbe comodo sì, ma io non sono come te.
Anche se... dove andró? Non saprei, o non sarò.
No, questo è un amore grande, sì.
Vuoi che ti dica così, ma io non sono come te.
Ciao, sai cosa ti dico: "ciao". Io posso stare senza te.
Senza piú tanti "se", senza tanti "ma perché?"
senza un amore cosi, io posso stare, sì.
Ciao, sai cosa vuol dire "ciao"? Vuol dire un'altra come te.
E mai più tanti "se", e mai piú nessun perché.
Ci si illude ancora, sì.
No, questo é un amore grande, si.
Vuoi che ti dica così, però io non sono come te.
Ciao, sai cosa ti dico? Ciao. lo posso fare senza te.
Senza più grandi "se", senza grandi "ma perché?",
senza un amore così, io posso fare, sì.
Ciao, in fondo basta dire anche "ciao",
io sto meglio senza te.
Senza piú tanti "se", senza tanti "ma perché?"
senza un amore così, io posso stare, sì.
Ciao, sai cosa vuol dire "ciao"? Vuol dire un'altra come te.
E mai più tanti "se", e mai più nessun perché.
Senza un amore così, io posso stare, sì. Ciao.
http://en.wikipedia.org/wiki/Being_There#Score
Last edited:
Yamato
Legendary member
- Messages
- 9,840
- Likes
- 246
why the GBL_ID is MUCH better than the GBL_ON
Continuing from here:
http://www.trade2win.com/boards/trading-journals/85510-my-journal-2-a-155.html#post1273718
Ok, dude.
Here's what's bugging me: the GBL.
The guys at disktrading.com are totally ignoring me and I can't bother them either, because they're the only good data vendor I know at that price. Damn.
So all i've really got is the forward-tested trades, which are pretty good, but not good enough for me.
But let's look at those and let's compare my two main GBL systems, GBL_ID and GBL_ON, which are strikingly similar in profit and trades, but there's a hidden element: two systems identical and in profit and trades are not the same in performance.
Let's look at them in detail.
Equity of forward tested period: looks similar.
The profit, trades, drawdown are all similar. Yet I still maintain that GBL_ID is better and that is why I am not trading the other. But I have to investigate on why, because I don't remember what was the reason.
Ok, here is all the details:
Biggest loss: GBL_ID is almost twice as good: THAT IS ONE REASON.
1) GBL_ID -741 dollars
2) GBL_ON -1251 dollars
Max drawdown: similar values
1) GBL_ID -4,047 dollars
2) GBL_ON -3,816 dollars
Trades: similar values
1) GBL_ID 148
2) GBL_ON 146
Incidentally, you'll be surprised to know that the two systems are totally different, in hours of trading, duration of trades, strategy.
Profit Factor: GBL_ID is much better
1) GBL_ID: GROSS PROFIT 23,775 divided by GROSS LOSS 15,243 = 1.56
2) GBL_ON: GROSS PROFIT 34,368 divided by GROSS LOSS 27,594 = 1.25
What I could not spot on the chart with the naked eye is now evident with the profit factor. One system goes up steadily, whereas the other goes up and down. I suppose this is the same thing measured by the much hated Sharpe Ratio.
I should ask the investors to check the Sharpe Ratio out for me, since I cannot do it.
In other words if you had a system that makes 102k and loses 100k, showing a profit of 2k, we could not say that system is the same as another system that makes 2k and loses nothing. In our case, we're somewhere in between of course and this was just an extreme example to show my point. My point is that 2 systems with the same amount of trades, the same profit and even the same drawdown as in my case, are not necessarily as good as each other. And it is because of how often that same drawdown happens. This is measured successfully by the Profit Factor and the Sharpe Ratio.
Let's give yet another clearer example to clarify things in my own head first of all, since that's why I am writing.
We have two systems with a profit of 2000, drawdown of 2000 and 5 trades. Everything is apparently the same, as with my two GBL systems. But here's their trades:
1) first system:
+1000
+1000
+1000
-2000
+1000
5 trades, profit 2000, drawdown -2000. Profit factor 4000 / 2000 = 2
2) second system:
+2500
-2000
+2500
-2000
+1000
5 trades, profit 2000, drawdown -2000. YET: Profit factor 6000 / 4000 = 1.5
The simplification above is a perfect summary of what goes on with my two GBL systems. They both reach the same profit destination but the first one reaches it with fewer zig-zags - even though the largest zig-zags are of the same length (2000).
Everything seems the same by profit, number of trades, drawdown and even the naked eye on a chart, as happened with the chart above. But the Profit Factor alerts us that the two systems are not the same, that one is more regular and the other one is potentially more random, unpredictable and unreliable. Not as good, basically, despite a deceivingly similar performance.
Whereas with the 102k gross profit vs 100k gross loss system it is evident that the system is not reliable, with smaller numbers that is not as evident and we need the profit factor to alert us (since I can't deal with the Sharpe Ratio).
That difference in profit factor makes the whole difference in how reliable a system is. The higher the profit factor the more we can trust it, regardless of absolute profit.
Let's take the more extreme example I mentioned earlier. One system makes 102k and loses 100k. Absolute profit of 2k and profit factor of 1.02. Another system makes only a profit of half as much: 1k. But it has a gross profit of 1100 dollars and loses 100 dollars. Absolute profit of 1k and profit factor of 11. Which system do we trust more? The one that makes 2k or the one that makes 1k? The choice is obvious, despite the fact we'd be discarding a more profitable system.
But why do we discard a system that makes more money? Because we are thinking that it was lucky and that in the future it might not deliver what it delivered in the past. We don't want to trade systems that succeeded in the past but systems that will succeed in the future. And that is why we cannot only look at profit as a measure of performance. Right: winning is not enough. A winner today may not always be a winner tomorrow, and someone in second place might be more likely to be tomorrow's winner.
Ideally we prefer a system always getting second place rather than a system getting one year first place and the other year third place. This is extreme, but you know what i meant.
It's as if you are seeing two guys shooting a target. And one guy is shooting all over the place, in all directions and suddenly he hits the target. The other guy is always one centimetre away from the target and he never hits it and so he loses, whereas the other guy wins. They shoot the same number of times, and one guy wins. Who do we trust better? The guy who hit the target after hitting meters away from the target 90% of the time? Or the guy who always missed but was always within 1 centimetre from the target? We trust better the guy who always missed and lost the competition.
I am not done. More examples.
View attachment who_is_better.bmp
Who do we trust better and invest our money on? The system that makes the most money (green) or the system that makes the least money (red)?
We trust red because we think green got lucky. This is to show once and for all that profit alone is not enough to measure performance.
And we have finally appraised that my GBL_ON systems, despite appearing the same as GBL_ID in every aspect, is half as good as GBL_ID, and this is merely measured by one of 4 performance indicators I use, profit factor. And that is because I don't want to use Sharpe Ratio, which sucks because it's got the square root, which always freaks me out, because my Math only goes up to seventh grade.
Continuing from here:
http://www.trade2win.com/boards/trading-journals/85510-my-journal-2-a-155.html#post1273718
Ok, dude.
Here's what's bugging me: the GBL.
The guys at disktrading.com are totally ignoring me and I can't bother them either, because they're the only good data vendor I know at that price. Damn.
So all i've really got is the forward-tested trades, which are pretty good, but not good enough for me.
But let's look at those and let's compare my two main GBL systems, GBL_ID and GBL_ON, which are strikingly similar in profit and trades, but there's a hidden element: two systems identical and in profit and trades are not the same in performance.
Let's look at them in detail.
Equity of forward tested period: looks similar.
The profit, trades, drawdown are all similar. Yet I still maintain that GBL_ID is better and that is why I am not trading the other. But I have to investigate on why, because I don't remember what was the reason.
Ok, here is all the details:
Biggest loss: GBL_ID is almost twice as good: THAT IS ONE REASON.
1) GBL_ID -741 dollars
2) GBL_ON -1251 dollars
Max drawdown: similar values
1) GBL_ID -4,047 dollars
2) GBL_ON -3,816 dollars
Trades: similar values
1) GBL_ID 148
2) GBL_ON 146
Incidentally, you'll be surprised to know that the two systems are totally different, in hours of trading, duration of trades, strategy.
Profit Factor: GBL_ID is much better
1) GBL_ID: GROSS PROFIT 23,775 divided by GROSS LOSS 15,243 = 1.56
2) GBL_ON: GROSS PROFIT 34,368 divided by GROSS LOSS 27,594 = 1.25
What I could not spot on the chart with the naked eye is now evident with the profit factor. One system goes up steadily, whereas the other goes up and down. I suppose this is the same thing measured by the much hated Sharpe Ratio.
I should ask the investors to check the Sharpe Ratio out for me, since I cannot do it.
In other words if you had a system that makes 102k and loses 100k, showing a profit of 2k, we could not say that system is the same as another system that makes 2k and loses nothing. In our case, we're somewhere in between of course and this was just an extreme example to show my point. My point is that 2 systems with the same amount of trades, the same profit and even the same drawdown as in my case, are not necessarily as good as each other. And it is because of how often that same drawdown happens. This is measured successfully by the Profit Factor and the Sharpe Ratio.
Let's give yet another clearer example to clarify things in my own head first of all, since that's why I am writing.
We have two systems with a profit of 2000, drawdown of 2000 and 5 trades. Everything is apparently the same, as with my two GBL systems. But here's their trades:
1) first system:
+1000
+1000
+1000
-2000
+1000
5 trades, profit 2000, drawdown -2000. Profit factor 4000 / 2000 = 2
2) second system:
+2500
-2000
+2500
-2000
+1000
5 trades, profit 2000, drawdown -2000. YET: Profit factor 6000 / 4000 = 1.5
The simplification above is a perfect summary of what goes on with my two GBL systems. They both reach the same profit destination but the first one reaches it with fewer zig-zags - even though the largest zig-zags are of the same length (2000).
Everything seems the same by profit, number of trades, drawdown and even the naked eye on a chart, as happened with the chart above. But the Profit Factor alerts us that the two systems are not the same, that one is more regular and the other one is potentially more random, unpredictable and unreliable. Not as good, basically, despite a deceivingly similar performance.
Whereas with the 102k gross profit vs 100k gross loss system it is evident that the system is not reliable, with smaller numbers that is not as evident and we need the profit factor to alert us (since I can't deal with the Sharpe Ratio).
That difference in profit factor makes the whole difference in how reliable a system is. The higher the profit factor the more we can trust it, regardless of absolute profit.
Let's take the more extreme example I mentioned earlier. One system makes 102k and loses 100k. Absolute profit of 2k and profit factor of 1.02. Another system makes only a profit of half as much: 1k. But it has a gross profit of 1100 dollars and loses 100 dollars. Absolute profit of 1k and profit factor of 11. Which system do we trust more? The one that makes 2k or the one that makes 1k? The choice is obvious, despite the fact we'd be discarding a more profitable system.
But why do we discard a system that makes more money? Because we are thinking that it was lucky and that in the future it might not deliver what it delivered in the past. We don't want to trade systems that succeeded in the past but systems that will succeed in the future. And that is why we cannot only look at profit as a measure of performance. Right: winning is not enough. A winner today may not always be a winner tomorrow, and someone in second place might be more likely to be tomorrow's winner.
Ideally we prefer a system always getting second place rather than a system getting one year first place and the other year third place. This is extreme, but you know what i meant.
It's as if you are seeing two guys shooting a target. And one guy is shooting all over the place, in all directions and suddenly he hits the target. The other guy is always one centimetre away from the target and he never hits it and so he loses, whereas the other guy wins. They shoot the same number of times, and one guy wins. Who do we trust better? The guy who hit the target after hitting meters away from the target 90% of the time? Or the guy who always missed but was always within 1 centimetre from the target? We trust better the guy who always missed and lost the competition.
I am not done. More examples.
View attachment who_is_better.bmp
Who do we trust better and invest our money on? The system that makes the most money (green) or the system that makes the least money (red)?
We trust red because we think green got lucky. This is to show once and for all that profit alone is not enough to measure performance.
And we have finally appraised that my GBL_ON systems, despite appearing the same as GBL_ID in every aspect, is half as good as GBL_ID, and this is merely measured by one of 4 performance indicators I use, profit factor. And that is because I don't want to use Sharpe Ratio, which sucks because it's got the square root, which always freaks me out, because my Math only goes up to seventh grade.
Last edited:
Adamus
Experienced member
- Messages
- 1,898
- Likes
- 97
Re: humongous discoveries: dropping 5 systems out of 10
After seeing your analysis to compare trading system live results vs backtested results, and after doing it myself for my own systems, I've come to acknowledge that there's often a considerable difference, and if you don't look at enough trades, they could seem to be totally different systems.
I also tweaked some of my trading systems to reduce the difference in the backtest results when run on different data. Do you reckon it's right to assume that these tweaks will also reduce the difference between the live results and the backtest results over the same period?
I actually find it quite difficult to reduce the difference. I started out thinking I should be able to apply logic to the situation, but a lot of what I thought should work actually didn't. The techniques that I ended up applying, partly based on stuff you told me earlier, were:
- don't use highs or lows. Use weighted closes, or the average of the close and the high or the open and the high, whichever is nearer.
- don't use ATR except on long time frames or long periods
- base all averages or indicator inputs on average bar values, not OHLC points
- avoid direct comparisons when not strictly necessary and use >= or <= instead of > and <
You see what I mean about it being difficult. There was a big new technology a while back that has settled back down into obscurity now, called Fuzzy Logic that I think would be useful here.
Almost forgot to say, I've got a correction for your English. Actually it's not a correction, it's just an improvement. Not many people say "it's relieving", rather they'd say "it's a relief". It's actually so uncommon that I misread it first time and thought you'd put "reliving".
After seeing your analysis to compare trading system live results vs backtested results, and after doing it myself for my own systems, I've come to acknowledge that there's often a considerable difference, and if you don't look at enough trades, they could seem to be totally different systems.
I also tweaked some of my trading systems to reduce the difference in the backtest results when run on different data. Do you reckon it's right to assume that these tweaks will also reduce the difference between the live results and the backtest results over the same period?
I actually find it quite difficult to reduce the difference. I started out thinking I should be able to apply logic to the situation, but a lot of what I thought should work actually didn't. The techniques that I ended up applying, partly based on stuff you told me earlier, were:
- don't use highs or lows. Use weighted closes, or the average of the close and the high or the open and the high, whichever is nearer.
- don't use ATR except on long time frames or long periods
- base all averages or indicator inputs on average bar values, not OHLC points
- avoid direct comparisons when not strictly necessary and use >= or <= instead of > and <
You see what I mean about it being difficult. There was a big new technology a while back that has settled back down into obscurity now, called Fuzzy Logic that I think would be useful here.
Almost forgot to say, I've got a correction for your English. Actually it's not a correction, it's just an improvement. Not many people say "it's relieving", rather they'd say "it's a relief". It's actually so uncommon that I misread it first time and thought you'd put "reliving".
Yamato
Legendary member
- Messages
- 9,840
- Likes
- 246
It was a relief for me to have looked it up and have found that... I have bad news for you. My use of "relieving" is correct:
http://www.google.com/search?hl=en&...q="it's+relieving"&aq=f&aqi=&aql=&oq=&gs_rfai=
5000 people use it. That's a correct expression by my book.
Keep monitoring my English though, as you might have better luck next time.
Do you have the GBL data by any chance? Do you know anyone who has it?
The reason I ask you is that our buddies at disktrading.com have not bothered to send me the promised data yet.
Regarding your questions:
Yes, I think so ("these tweaks will also reduce the difference between the live results and the backtest results over the same period").
You see, the simpler your systems, the less...:
1) ...overoptimized they risk being
2) ...there will be discrepancies between two data sets (of the same future)
3) ...there will be discrepancies between two data sets tested on two different platforms, such as the situation of live execution vs back-testing (in my case at least).
I am glad you followed my advice, but I didn't suggest many of the things you did do (e.g.: "Use weighted closes, or the average of the close and the high or the open and the high, whichever is nearer.").
What I advise is simply to reduce the parameters to as few parameters as possible.
In my systems, as I said in a previous post, I use any two (three in some cases) of these parameters (at a time):
1) time of the day (always used)
2) price above or below a moving average
3) price above or below a previous close (yesterday or x days ago)
4) day of the week
5) etcetera
For example, using ATR is like using 5 parameters at once to my mind. So is using the RSI, because you're using the last 13 closes or something like that. Also, one thing is to use a simple moving average and another thing is to use MACD. You get my point.
If a system is simple, you could even get a 50% of discrepancies, but the same profit, because the trades are made according to the same strategy, so it doesn't matter if they are made on different days.
For me it is pretty easy to keep things simple because I always refuse (it's like an allergy) to use complex ingredients which I don't understand completely and since I don't understand a lot of things, I always keep things simple. For example, I am allergic to RSI, MACD, Sharpe Ratio... I never use things whose implications I don't fully understand.
http://www.google.com/search?hl=en&...q="it's+relieving"&aq=f&aqi=&aql=&oq=&gs_rfai=
5000 people use it. That's a correct expression by my book.
Keep monitoring my English though, as you might have better luck next time.
Do you have the GBL data by any chance? Do you know anyone who has it?
The reason I ask you is that our buddies at disktrading.com have not bothered to send me the promised data yet.
Regarding your questions:
Yes, I think so ("these tweaks will also reduce the difference between the live results and the backtest results over the same period").
You see, the simpler your systems, the less...:
1) ...overoptimized they risk being
2) ...there will be discrepancies between two data sets (of the same future)
3) ...there will be discrepancies between two data sets tested on two different platforms, such as the situation of live execution vs back-testing (in my case at least).
I am glad you followed my advice, but I didn't suggest many of the things you did do (e.g.: "Use weighted closes, or the average of the close and the high or the open and the high, whichever is nearer.").
What I advise is simply to reduce the parameters to as few parameters as possible.
In my systems, as I said in a previous post, I use any two (three in some cases) of these parameters (at a time):
1) time of the day (always used)
2) price above or below a moving average
3) price above or below a previous close (yesterday or x days ago)
4) day of the week
5) etcetera
For example, using ATR is like using 5 parameters at once to my mind. So is using the RSI, because you're using the last 13 closes or something like that. Also, one thing is to use a simple moving average and another thing is to use MACD. You get my point.
If a system is simple, you could even get a 50% of discrepancies, but the same profit, because the trades are made according to the same strategy, so it doesn't matter if they are made on different days.
For me it is pretty easy to keep things simple because I always refuse (it's like an allergy) to use complex ingredients which I don't understand completely and since I don't understand a lot of things, I always keep things simple. For example, I am allergic to RSI, MACD, Sharpe Ratio... I never use things whose implications I don't fully understand.
Adamus
Experienced member
- Messages
- 1,898
- Likes
- 97
OK thanks for the run-down. I do try to keep things simple but am obviously not as successful as you at it. The kind of stuff you did when you made that pivot point system was more like the way I work.
I think it's possible that you're right about a system still being profitable despite having 50% discrepancy in the trades compared to the backtest. It just isn't intuitive, at least not to me. It's quite worrying. I guess I feel it's telling me I'm not in control.
re the "it's relieving" vs "it's a relief" - is 5000 hits vs 160,000 hits. So you can take your pick as you have. I won't go so far as to say it's bad English, it just doesn't sound right to my ears. If you insist, let's run a poll on the forum here.
Adamus
Experienced member
- Messages
- 1,898
- Likes
- 97
Is that a knife you're holding up or some sort of veiled Al Pacino style threat? It's difficult to see across the internet.
I forgot to mention it, presumably because I was too busy trying to prove my point about use of the word relief - as in, I can't get no relief.
I have only subscribed to forex data, so unfortunately for you, I can't provide you any Bund data.
Yamato
Legendary member
- Messages
- 9,840
- Likes
- 246
Yes, good memory. It has to do with Al Pacino. It was inspired by this quote:
http://www.imdb.com/title/tt0086250/quotes
http://www.imdb.com/title/tt0086250/quotes
Yamato
Legendary member
- Messages
- 9,840
- Likes
- 246
Been thinking...
Been thinking about money management and risk management.
About the importance of the size of the drawdown and the frequency of it, both measured by profit factor and sharpe ratio (which I understand much less).
Given that no systems are infallible, risk/money management are very important.
Having profitable systems without having the proper risk and money management is like having a knife without knowing how to use, such as giving a knife to a child.
You're going to play with it until you get hurt. With the knife and with your systems.
Another part of risk and money management is having capital, because otherwise you'll be encouraged to ignore risk and money management, hoping to increase your capital faster.
And you can do it. You can do it once, twice and three months in a row, but sooner or later you'll blow out your account. Not necessarily because of the drawdown itself but because of your inability to mentally accept a drawdown you weren't prepared to see: you will start discretionary trading, trying to make your losses back, you will second-guess your systems, being afraid of future failures... you'll blow out your account, one way or another.
When your drawdown is as big as 50% of your account, unless your account is 2000 dollars, you will start second-guessing and doubting your systems for sure - even though they won't be necessarily failing.
That is why you have to know your systems and their performance: in order to not get scared when nothing unusual is happening.
When you'll incur a loss by your systems, you better make sure that you're positive about every detail of your work, or you'll immediately abandon your systems. Not at a 50% drawdown, but at the first 300 dollars loss.
If you're reckless and you don't look at them, on the other hand, you might get away with a lot more. I know this from personal experience. But it's hard to be reckless for a long time. It's hard to stay reckless once you've blown out your account because of your systems more than once.
Two year ago i had some profitable systems, I got lucky and I doubled my account every month, for 3 months in a row. Then, on the fourth month, something went wrong, I lost my confidence, tampered with the systems, and blew out my account in one month. It's not enough to have profitable systems. You have to know exactly what will happen at the worst moment and how you will react. It's much easier to underbet. But it's easier to underbet if you have a lot of money, which I don't have.
Been thinking about money management and risk management.
About the importance of the size of the drawdown and the frequency of it, both measured by profit factor and sharpe ratio (which I understand much less).
Given that no systems are infallible, risk/money management are very important.
Having profitable systems without having the proper risk and money management is like having a knife without knowing how to use, such as giving a knife to a child.
You're going to play with it until you get hurt. With the knife and with your systems.
Another part of risk and money management is having capital, because otherwise you'll be encouraged to ignore risk and money management, hoping to increase your capital faster.
And you can do it. You can do it once, twice and three months in a row, but sooner or later you'll blow out your account. Not necessarily because of the drawdown itself but because of your inability to mentally accept a drawdown you weren't prepared to see: you will start discretionary trading, trying to make your losses back, you will second-guess your systems, being afraid of future failures... you'll blow out your account, one way or another.
When your drawdown is as big as 50% of your account, unless your account is 2000 dollars, you will start second-guessing and doubting your systems for sure - even though they won't be necessarily failing.
That is why you have to know your systems and their performance: in order to not get scared when nothing unusual is happening.
When you'll incur a loss by your systems, you better make sure that you're positive about every detail of your work, or you'll immediately abandon your systems. Not at a 50% drawdown, but at the first 300 dollars loss.
If you're reckless and you don't look at them, on the other hand, you might get away with a lot more. I know this from personal experience. But it's hard to be reckless for a long time. It's hard to stay reckless once you've blown out your account because of your systems more than once.
Two year ago i had some profitable systems, I got lucky and I doubled my account every month, for 3 months in a row. Then, on the fourth month, something went wrong, I lost my confidence, tampered with the systems, and blew out my account in one month. It's not enough to have profitable systems. You have to know exactly what will happen at the worst moment and how you will react. It's much easier to underbet. But it's easier to underbet if you have a lot of money, which I don't have.
Yamato
Legendary member
- Messages
- 9,840
- Likes
- 246
I'd like to be under the sea
I'd like to be under the sea
In an octopus' garden in the shade
He'd let us in, knows where we've been
In his octopus' garden in the shade
I'd ask my friends to come and see
An octopus' garden with me
I'd like to be under the sea
In an octopus' garden in the shade.
We would be warm below the storm
In our little hideaway beneath the waves
Resting our head on the sea bed
In an octopus' garden near a cave
We would sing and dance around
because we know we can't be found...
I'd like to be under the sea
In an octopus' garden in the shade
He'd let us in, knows where we've been
In his octopus' garden in the shade
I'd ask my friends to come and see
An octopus' garden with me
I'd like to be under the sea
In an octopus' garden in the shade.
We would be warm below the storm
In our little hideaway beneath the waves
Resting our head on the sea bed
In an octopus' garden near a cave
We would sing and dance around
because we know we can't be found...
Last edited:
Yamato
Legendary member
- Messages
- 9,840
- Likes
- 246
stiffed by the grocer
Pretty depressed and frustrated about it. When everything seemed to go just fine. I just cannot have one totally good day anymore. Since many years ago.
I can't complain because if you're wrong it's really bad to accuse someone of cheating you - I don't do that stuff. What I'll do is not go there for a couple of weeks, so if he ripped me off, he'll imagine why. If he didn't rip me off, he'll just think I didn't need to buy anything.
He didn't give me a receipt and, according to my estimate, he cheated me out of 3 euros on 20 euros he had me pay. Here's what I bought:
1 yogurt with a price of 1.80 on it
1 wine-like drink with a tag of 2.60 on it
2 Ceres beers (to sleep, for emergencies, when I wake up at 5 AM and go back to sleep otherwise)
150 grams of salami
1 box of mozzarellas
Now, if the wine drink cost me 2.60, the beers must have cost less, somewhere around 2.20. But let's even say they cost 2.50. Let's even say that for all drinks and the yogurt I had to pay 10 dollars.
How can the salami cost 5 dollars? And the mozzarellas how can they cost another 5 dollars? Impossible.
But there were more problems today. Here's the major problems of my day so far:
1) Got on the cab and the taximeter was partly covered by the cord of his cb radio. I told him, and he got upset because he actually had the right fare on (1 rather than 2), and so for once I had to apologize, whereas usually when this happens it's because they're ripping you off. Then we made peace but it got me frustrated anyway.
2) Vito the Chimp was at the office, hyperactive as usual (asking stupid questions and interrupting my work), so it added to my stress.
3) I got criticized by the boss for writing to him on an email that a colleague was fooling us by postponing forever the documents we had requested. He said to use a more business-like language, without adding personal comments.
4) Stiffed by the grocer.
When I am king you will be first against the wall...
http://www.songfacts.com/detail.php?id=1541
http://www.greenplastic.com/lyrics/paranoidandroid.php
Pretty depressed and frustrated about it. When everything seemed to go just fine. I just cannot have one totally good day anymore. Since many years ago.
I can't complain because if you're wrong it's really bad to accuse someone of cheating you - I don't do that stuff. What I'll do is not go there for a couple of weeks, so if he ripped me off, he'll imagine why. If he didn't rip me off, he'll just think I didn't need to buy anything.
He didn't give me a receipt and, according to my estimate, he cheated me out of 3 euros on 20 euros he had me pay. Here's what I bought:
1 yogurt with a price of 1.80 on it
1 wine-like drink with a tag of 2.60 on it
2 Ceres beers (to sleep, for emergencies, when I wake up at 5 AM and go back to sleep otherwise)
150 grams of salami
1 box of mozzarellas
Now, if the wine drink cost me 2.60, the beers must have cost less, somewhere around 2.20. But let's even say they cost 2.50. Let's even say that for all drinks and the yogurt I had to pay 10 dollars.
How can the salami cost 5 dollars? And the mozzarellas how can they cost another 5 dollars? Impossible.
But there were more problems today. Here's the major problems of my day so far:
1) Got on the cab and the taximeter was partly covered by the cord of his cb radio. I told him, and he got upset because he actually had the right fare on (1 rather than 2), and so for once I had to apologize, whereas usually when this happens it's because they're ripping you off. Then we made peace but it got me frustrated anyway.
2) Vito the Chimp was at the office, hyperactive as usual (asking stupid questions and interrupting my work), so it added to my stress.
3) I got criticized by the boss for writing to him on an email that a colleague was fooling us by postponing forever the documents we had requested. He said to use a more business-like language, without adding personal comments.
4) Stiffed by the grocer.
When I am king you will be first against the wall...
http://www.songfacts.com/detail.php?id=1541
http://www.greenplastic.com/lyrics/paranoidandroid.php
Last edited:
- Status
Similar threads
- Locked
