Deadline June

[Adamus]

Re: TurningPoints Improvements Needed

This is a short sharp shock type of error. It's the same error that I've outlined before - the system hasn't taken into account that there's a downward trend, lower highs and lower lows.

 

[Adamus]

Re: This week's 7 Habits priority setting

Long Term Goals

(1) Achieve compounding account growth. This was "Make a million" but aiming for geometric increases in equity makes more sense than just "a million" which is pretty arbitrary.

(2) Build a trading platform that does all the good things I need that I know from the various different trading platforms I've used or evaluated, e.g. TradeStation, NinjaTrader, TradingBlox as an open-source collaborative project made up of trader-developers.

Medium Term Goals

(1) Find or invent, write, test and implement trading systems.

(2) Software for foundation of trading platform - name it, get it up and running without the database performance issue it currently has dealing with millions of trades.

(3) Work out a better way of describing my trading activities in public. I need more functionality than T2W gives, i.e. some privacy options, back-up options, journal 'dumping' to xml file, file upload area, tags to filter my posts by when searching etc

Short Term Goals

(1) Gap Filler system - I realised the acute volatility of 2008 / 2009 needs a completely different category of filter than any that I tried so far. Back to the drawing board.

(2) Other low hanging fruit in the systems ideas list I have: Joel Rensink's One Night Stand and First Strike systems.

(3) No further response from BT. Looks like they will hold me to the contract £30 per month for another 18 months. Still need to take my trading machine around to a friend's house who uses Virgin Broadband. This should demonstrate whether I can run IB TWS overnight without any disconnects, especially between 4AM and 5AM.

(4) Software - name it - haven't thought up anything good enough yet. Still to set up the project testing batch run and code a database performance test which fails if it falls below a desired level.

(5) Sofware 2: get Apache and Tomcat running together on VPS.

(6) VPS for NinjaTrader - find best hosting service.

(7) Write a little Java program that can convert FXCM historical data to NinjaTrader format.

 

[Yamato]

Re: Open Invitation to All Mechanical Traders

To all mechanical traders who are at the same point as me in their career - limited capital and limited time and potentially big starting-out risks: the most logical answer is to pool capital and system to reduce basic risk from drawdown in the initial phase of building up the capital and systems portfolio.

If my explanations in the preceding post make sense and you're interested, get in touch by pm and let's talk.

Very good idea, but it has to be done right. Personally I've been lucky this time, with the investors I found, but previously I've been ripped off. What gives you the guarantee that the other guy will not just take your systems and show you nothing in return, or show you a crappy system in return? That's what happened to me two years ago with some people I met on Elitetrader.com. They asked to see all my systems and then they said they were no good and disappeared. Pretty depressing.

Another problem is in how to "pool" your capital together. Do they wire it to your account or viceversa? What is the guarantee for anyone?

A better way in my opinion is what I am doing. I am basically renting my systems. I suggest ideas, and I get audited, and not only do i get to make a % of total profit, but I also learn to appraise my systems: which are good and which are not, and that is how I found out that not 100%, not 50%, but just 25% of my systems are good enough to be traded.

So, good point in doing it, but maybe you have enough capital to be the person running other people's systems. Or maybe viceversa. But it seems hard to me to do something where two people are providing the same ingredients to the recipe. Such as a situation where two people are providing systems, or two people are providing money, or even both things at the same time. Also because systems are not always compatible and usually they require an infrastructure that varies from one person to another (different broker, software, operating system, markets, data subscriptions, etc.).

 

[Adamus]

Re: Open Invitation to All Mechanical Traders

Very good idea, but it has to be done right. Personally I've been lucky this time, with the investors I found, but previously I've been ripped off. What gives you the guarantee that the other guy will not just take your systems and show you nothing in return, or show you a crappy system in return? That's what happened to me two years ago with some people I met on Elitetrader.com. They asked to see all my systems and then they said they were no good and disappeared. Pretty depressing.

Another problem is in how to "pool" your capital together. Do they wire it to your account or viceversa? What is the guarantee for anyone?

A better way in my opinion is what I am doing. I am basically renting my systems. I suggest ideas, and I get audited, and not only do i get to make a % of total profit, but I also learn to appraise my systems: which are good and which are not, and that is how I found out that not 100%, not 50%, but just 25% of my systems are good enough to be traded.

So, good point in doing it, but maybe you have enough capital to be the person running other people's systems. Or maybe viceversa. But it seems hard to me to do something where two people are providing the same ingredients to the recipe. Such as a situation where two people are providing systems, or two people are providing money, or even both things at the same time. Also because systems are not always compatible and usually they require an infrastructure that varies from one person to another (different broker, software, operating system, markets, data subscriptions, etc.).

I don't actually know what the deal is that you struck with your investors. Do you get a percentage of the profits annually? Can you re-invest the profit in the same account and build up a share?

Regarding two or more people operating as a unit, it would have to be done as a company or a fund with the legal framework that treats the people as investors and it would have to be run by a third party lawyer or accountant with power of attorney to make the payments. That's the easy bit.

As you say, it would be hard for 2 cooks to provide the same ingredients. But I figure if someone is willing to go to length of setting up and paying for the administrative costs, then they are unlikely to disappear with my code. I think I would be convinced to go into partnership with someone if I saw their backtest results and out-of-sample results and preferably some live results and broker statements, although still I would have to trust that the person hadn't cheated on the out-of-sample testing. Maybe a six month or year period of forward testing is required first.

Last but not least all parties involved have to be flexible enough to agree on a trading infrastructure too, as you also mentioned, because everyone will want to be able to verify the systems are trading correctly and that nobody is using the account to play the markets themselves.

All in all it's a massive investment of time and effort to put it in place, so I honestly don't think anyone is even going to ask me about it but I figured there's nothing to lose by asking.

 

[Yamato]

Re: Open Invitation to All Mechanical Traders

I don't actually know what the deal is that you struck with your investors. Do you get a percentage of the profits annually? Can you re-invest the profit in the same account and build up a share?

Regarding two or more people operating as a unit, it would have to be done as a company or a fund with the legal framework that treats the people as investors and it would have to be run by a third party lawyer or accountant with power of attorney to make the payments. That's the easy bit.

As you say, it would be hard for 2 cooks to provide the same ingredients. But I figure if someone is willing to go to length of setting up and paying for the administrative costs, then they are unlikely to disappear with my code. I think I would be convinced to go into partnership with someone if I saw their backtest results and out-of-sample results and preferably some live results and broker statements, although still I would have to trust that the person hadn't cheated on the out-of-sample testing. Maybe a six month or year period of forward testing is required first.

Last but not least all parties involved have to be flexible enough to agree on a trading infrastructure too, as you also mentioned, because everyone will want to be able to verify the systems are trading correctly and that nobody is using the account to play the markets themselves.

All in all it's a massive investment of time and effort to put it in place, so I honestly don't think anyone is even going to ask me about it but I figured there's nothing to lose by asking.

Yes, it was a very complicated deal to strike, and I was lucky, because the investors turned out to be a good match for me (but it was an unlikely thing to happen). I get a share of the profits, let's not say exactly how much for privacy purposes. Also, we recently made a deal whereby, if we're scaling up, I only get half of my share (on a "high water mark" monthly basis), and the other half of my share (together with their entire share) goes towards building up the "profit cushion" needed to withstand the potential drawdown. Then, if we'll stop building up the profit cushion, everyone will get their share of the profit. The money gone into the profit cushion will stay there until we're trading together. I will only get it at the end, when I retire (or if they retire). This complex arrangement was devised because I can't just be expected to go on for years without ever getting a dime, with the risk of never seeing a dime - in case something unexpected should happen, such as them dying, going into a coma, going crazy, or, more likely, getting into a fight with me.

I suppose you're being ironic when you say "that's the easy bit". A "fund", a "legal framework", a "lawyer/accountant"...it will be very unlikely that you'll get started with all that expensive infrastructure (we have none of that) without even showing each other your systems, and the minute you do that, as I said before, you risk getting screwed.

Overall, if you have the right financial and mental balance, it would be best to do it all alone. But I had neither. So it was easy for me to take the leap. After blowing out my account once too many (this time with borrowed money), I was desperate. But this time i was lucky. They didn't just take my systems and disappeared. However, the probability of that situation happening are 10% or less in my opinion. In all other cases you're either getting into a partnership with incompetent people, or with people who will screw you, one way or another.

Regarding your offer to all automated traders, a while ago I was thinking of proposing a similar thing to you, whereby I'd provide the systems and you would provide the capital: 40k pounds is not bad at all. But you were lucky because I met the investors first... yeah, "lucky" because I was a lousy money manager back then and maybe still am, and would have risked losing your money. For example, I started out by trading many crappy systems, but they said "first show us the results, then we'll increase the investment" and that saved us, because we only have increased it now, after I have taken as long as six months to figure out which ones are my good systems.

Personally, even though you have a good capital, I would suggest to you to look for investors to whom you'll offer your systems. But you need experts, not people who know nothing about trading or they won't teach you anything, and it will be like being alone, and having asked a loan from your bank.

 

[Adamus]

Re: TurningPoints Improvements Needed

Here's another potential area for improvement. The new bar is still only half way through so it might come back in the right direction but what I often see is a strong move against the trade trips the reversal switch and the retracement turns it into a whipsaw.

It's particularly hard psychologically since it was getting close to target - so maybe these losers rank higher in my memory than they should. Let's see how it develops.

... so - it just bounced around a hell of a lot. What can be learnt from it - I think the amount of bouncing around could be measured somehow - a different kind of measure of volatility, say the size of the candle wicks in relation to the whole candle, could be used to widen the stop a bit, might save a couple of trades from being caught by spikes. That verges on curve fitting though.

 

[Adamus]

Finding some way of keeping out of high volatility

The high volatility of 2008/2009 is destroying my backtest results.

Initially I ran backtests on a test period of 2005 to 2008 - short because of NinjaTrader limitations on my dev box, a memory issue.

One of the main problems in building a system with this test period and this strategy is that the losses in 2008 were huge. I found a combination of techniques that ironed out the huge losses but it turns out they were curve-fitted, surprisingly to me.

The reason why I curve-fit a solution despite being ultra-wary of doing so, I think, is because of the profit and loss stats - this is the first system I've coded where there are many little wins and only a few large losses. So in retrospect, it was easy to find ways to erase the few large losses without dampening the flow of little wins, but of course I didn't recognise the danger that the pattern was unreproducible.

I decided to take a different approach to it and to have a look at the raw stats, e.g. gap size, ATR, end-of-day profit on the Monday, and see how they relate.

I expect I'll just end up filtering out trades when the ATR goes over a certain limit, but I'll have to hardcode a specific ATR value for each seperate market, which I don't like doing for purist reasons. Maybe I'll find a relationship based on comparative values, who knows. The standard deviation of the daily ATR or something.

 

[Yamato]

Yeah, let me tell you my experience about 2008 and about curve fitting.

1) 2008: i recently wrote a series of posts on my journal where I appraise the ability of each one of my 14 "good" systems to make money or at least not lose it while its underlying is losing big, which can be defined as a study on the correlation of the system to its underlying. The CL systems can't quite make it: at best they break even (except for one, which is the best one) and usually they lose, even though much less. This is because the CL went from 130 to 30 in 2008, or something like that, and it was hard for most of my systems to be indifferent to that. All my good systems on the other futures instead have no problems making money whether their underlying futures go up or down. So: it can be done, and it can be done without any "combination of techniques", which risks curve-fitting.

Since a few days ago, I will keep telling myself and everyone to keep in mind and plot on a chart how the underlying behaves relative to the system trading it. This study is of major importance and with a very advantageous cost/benefit ratio. Sharpe ratio, profit factor, max drawdown... none of those performance parameters do it. You have to do it yourself. You want to come up with a system that performs better than its underlying - otherwise it's much easier to buy the future and keep it open, renewing it as soon as it expires. A very healthy system is one that makes money whether its underlying future is going up or down. An acceptable one is a system that makes as much, but loses less than its underlying. A lot more can be said, but I am throwing some ideas here and there and don't expect to be exhaustive.

2) Curve fitting: once again, the fight against curve-fitting is a fight to be fought with the out-of-sample and one which is as large as possible. I've gotten to a point where sometimes I am using an out-of-sample of 50% of the data I have, lasting as long as 5 years out of 10 years I have. My success rate now is around... 50% of the systems I create are good. Before I was using the out-of-sample, only 25% of the systems I created were good. I've been forward-testing for two years systems that I thought were excellent but which in real testing only piled up losses. This is mostly because I never used the out-of-sample methodology.

Creating, creating, creating... discarding, discarding, discarding
To make money you only need one good system. Discard, like I did, all the ones that don't work, even if it meant discarding 90% of them, and move forward only in the direction of where things work very well. Then either improve what works well (with the risk of ruining its edge), or, better, don't touch it, and move on to something else. From the start of my back-testing, in 2002, I have created at least 150 systems, of which only 74 will be forward-tested until now (I have now 71 with 3 more to automate). All the others have been dropped along the way. Only 12 are being traded live with real money. This means that less than 10% of my trading systems so far have produced real profits. You don't have to delete your work, but set it aside and archive it in a folder.

We all have good trading insights, ideas, systems. It's just a matter of getting rid of whatever else is bad. I had 10 good ideas out of 100. I didn't make any money until I got rid of those other 90 ideas, and this did not happen until the investors forced me to trade, day after day, week after week, month after month, the systems I had selected, including the bad ones: only then did I fully realize what worked and what didn't. And I realized how great was what I had, even though it was only 10% of all my work (this if I consider all the work I produced in the last 10 years).

 

[Adamus]

cheating with the out-of-sample

Thanks for the post which mostly confirms what I am thinking. However I'm not ready to give up this system just yet. I have cheated twice now on the out-of-sample - once at the beginning where I had built the raw Gap Filler system and didn't want to spend time refining it if that didn't work - and it did work albeit with large drawdowns; and once just now after I realised how and why my system was curve-fit. I'm going to try one more different approach - with the same test period and the same out-of-sample.

I know what you're going to say - I've seen what went wrong in the out-of-sample and I might just curve fit my system, also curve fitting it to the out-of-sample that I've already seen.

But I think the mechanism I am looking at to make the system survive the volatility is simple and robust and should have a reproducible effect in the future.

 

[Yamato]

Well, peaking once at the out-of-sample decreases the reliability of the out-of-sample methodology but it's still far better than not using it at all and using the whole sample to optimise your system. The more times you peak, the less your out-of-sample test is reliable (and the less it can be called "out-of-sample"). If you peak 20 times it's almost as if you never used the out-of-sample, but it's still better than not using it at all. Of course I would recommend not peaking even once, but peaking just twice is a good start.

 

[drolles]

Adamus,

Just a suggestion, i’ve noticed you mention a number of times that you have lost versions of NinjaScripts, you might want to look at Beanstalk (http://beanstalkapp.com/). I’d be happy to talk you though how I’ve set it up to work in the background and use it to store versions of NinjaScripts. I use it with Tortoise SVN as a front end service. If you use the introductory service it is free, so is TortoiseSVN (http://tortoisesvn.tigris.org/). They do have a storage limit and a user limit, but it might to the trick for you.

I think it will be a substantial help with your version control.

Cheers,

drolles

 

[Adamus]

ninjascript source control

Adamus,

Just a suggestion, i’ve noticed you mention a number of times that you have lost versions of NinjaScripts, you might want to look at Beanstalk (http://beanstalkapp.com/). I’d be happy to talk you though how I’ve set it up to work in the background and use it to store versions of NinjaScripts. I use it with Tortoise SVN as a front end service. If you use the introductory service it is free, so is TortoiseSVN (http://tortoisesvn.tigris.org/). They do have a storage limit and a user limit, but it might to the trick for you.

I think it will be a substantial help with your version control.

Cheers,

drolles

Thanks for the link. I'll check it out if I get hacked off with my current source control methodology - saving the ninjascript in the excel spreadsheet where I save the backtest results, a tab per backtest including settings, results, trades, equity curve and periods.

Back-ups are something extra to take care of but otherwise it's simple but effective.

In the long term and previously I use a home-grown java app, but it's currently on blocks in the garage at the moment after I ran into database performance issues that I couldn't solve immediately.

 

[Adamus]

Gap Filler system building using analysis rather than brute force

Rather than just wondering what a certain change in my system will do to the results and trying to find out by running a brute force optimisation for a parameter, I thought I'd try a more intelligent approach so I output the profit and the dailiy ATR for the simplest realistic system.

Enter 5 points on limit away from the Sunday Open on opposite side from Friday's close in direction of Friday's close.
Exit Monday at 21:00GMT+0.

The results are perhaps predictable. 2000 - 2008: the higher the ATR, the less profitable it is.

The first chart is the simple net count of profitable trades - losing trades where the ATR as a % of price is 0 - 1%, 1 - 2%, 2 - 3% etc. The second is the total profit.

I figure I should consult with some excel whizzkid because I'm doing this stuff by cutting and pasting formulae from cell to cell to build up a massive table of data, but there must be ways of coding a formula in one cell or running a built-in excel function.

 

[Adamus]

British Telecom, or the Dept of Circumlocution

I'm still trying to get somewhere with British Telecom. For every one email I get out of them, I have to send at least three.

Their support staff told me I could get around the problem using a VPN. I reminded them it was a problem that they cause.

They then told me, after I divulged that I actually make money from my activities using their broadband, that I should get a business broadband account because they were only residential support and had no capability to test the software I was using that had broadband problems. They also said in a tacit admission of guilt that "BT does shape internet traffic to ensure the vast majority of our customers can use the service for consumer related internet tasks." What happened to net neutrality? ****ers!

 

[drolles]

Re: ninjascript source control

Thanks for the link. I'll check it out if I get hacked off with my current source control methodology - saving the ninjascript in the excel spreadsheet where I save the backtest results, a tab per backtest including settings, results, trades, equity curve and periods.

Back-ups are something extra to take care of but otherwise it's simple but effective.

In the long term and previously I use a home-grown java app, but it's currently on blocks in the garage at the moment after I ran into database performance issues that I couldn't solve immediately.

Adamus,

Thanks for the reply.

Again, if you want to look at it, I’m happy to help. I can talk you though how I set it up.

For back-ups I use Carbonite (http://www.carbonite.co.uk/). This is a great app and very cheap for unlimited back-ups. It has some quirks (particularly around NT and file directories), but overall I’m very happy with it. By the same token you only really test a back-up service when you need to restore and I haven’t had to do that yet (touch wood).

Cheers,

Drolles

 

[Adamus]

Re: TurningPoints Improvements Needed

Here's another potential area for improvement. The new bar is still only half way through so it might come back in the right direction but what I often see is a strong move against the trade trips the reversal switch and the retracement turns it into a whipsaw.

It's particularly hard psychologically since it was getting close to target - so maybe these losers rank higher in my memory than they should. Let's see how it develops.

... so - it just bounced around a hell of a lot. What can be learnt from it - I think the amount of bouncing around could be measured somehow - a different kind of measure of volatility, say the size of the candle wicks in relation to the whole candle, could be used to widen the stop a bit, might save a couple of trades from being caught by spikes. That verges on curve fitting though.

More bouncing around - this time the GBP/AUD. The exotic pairs seem to do this a lot. This time it cost a few quid. What can be done about it - maybe check the level of 'bouncing around' and not trade if it's high.

 

[Adamus]

Re: This week's 7 Habits priority setting

Long Term Goals

(1) Achieve compounding account growth. This was "Make a million" but aiming for geometric increases in equity makes more sense than just "a million" which is pretty arbitrary.

(2) Build a trading platform that does all the good things I need that I know from the various different trading platforms I've used or evaluated, e.g. TradeStation, NinjaTrader, TradingBlox as an open-source collaborative project made up of trader-developers.

Bought myself a 7 Habits calendar which helps to do this goal setting and prioritization.

Just for the record and to finish my public goal setting, my goal number one has to be "Make 10 million".

Getting to 100K will be a major struggle.

Getting to 1 million will be easier, but a struggle still.

If I can get to 1 million from here using a well defined strategy, putting well designed systems in place with thoroughly worked-out risk management, then getting to 10 million is almost guaranteed, so my end goal here has to be to make 10 million pounds.

 

[sm1]

Adamus,
What software would you advise for walk-forward tests.
Thanks

I see a couple of explanations for this failure.

(1) I applied such whacky algorithms there was no way they were going to work out of sample

(2) a phase of market action happened in the last 4 months of the walk-forward that had never appeared in the optimisation window.

(3) 6 months is too long for a walk-forward - I should stick to 3 months (would have been nice). Pardo wrote his book in the early 90's, things have probably got more challenging since then.

(4) I have too many different components in the algorithm. It's a break-out system, so I have (a) a mid-point finder (b) a range finder for the limits either side of the mid-point (c) a filter (d) a stop distance calculator. Perhaps optimising against all of those is too much curve-fitting.

So what can a trader do?

(1) reduce the walk-forward window to 3 months. This probably reduces the chances of 'unseen' market types cropping up. It is probably more likely that the current market behaviour at the end of the optimisation window will continue through the walk-forward.

(2) stop optimising one or two of the algorithm components - i.e. lock down the stops to 2 times the average true range - or stick with one filter

 

[Adamus]

Adamus,
What software would you advise for walk-forward tests.
Thanks

Hi Sm1

there's no easy answer there. I only know Ninja Trader 7. I've got experience of old versions of Trade Station, I've heard from others about MetaTrader, Amibroker and Trading Blox, and I've looked at and wondered about others like NeoTicker.

To answer your question properly, you'd need to do a Which?-style magazine article with all the different features and prices and so on listed in a review of each platform.

NinjaTrader is probably the newest, so think lots of features AND lots of bugs, but it has some gaps such as not giving you portfolio analysis out-of-the-box. However some of its forward testing functionality is really good, and for that, it's free - although perhaps the others are too.

I think Trading Blox is the top of the range for portfolio analysis.

I guess Trade Station and Amibroker probably also rank quite well.

You might also think about the data question, where you're going to get it and how easy it is to import into your chosen platform.

 

[Adamus]

Your Trading Coach / YTC Price Action Trader

Has anybody bought YTC Price Action Trader?

I read his newsletter - and I only actually subscribe to 5 trading newsletters - and I always find something interesting in there so I figured his PDF ebook would be good value for money.

The only sticking point is that it's pretty much twice the price I'd be prepared to pay for it ($200) if it was a normal trading book in a bookstore. Plus of course if it's only an ebook, then it's probably not nearly as well edited as a real book.