Can mechanical forex systems work?

[rnicoll]

OK so there was a kick in the nuts - so it doesn't work as an automated system. It does however make a nice indicator for me to judge if we are in a trend or not.

Wait, did you try trading yesterday/today? I know this stuff is meant to be automated, but yikes the fundamentals are screaming "DO NOT TRADE" to me. I paper-traded my own today... err, I kept all the pieces?

I think I'm going to take my trending system and have versions that I apply. So I may think we have a bull run on the hourlies and so set that system on, or a bear run on the dailies and so set that system on.

Yes there will be a good bit of judgement call involved but I've driven myself mad trying to find the automated way to make thiese calls and just can't do it. I think my time would be better spent thinking about how the markets are currently behaving.


This may not however be such a bad thing. So I'm going to optimise the strategy I have on various timescales and then sit back and take a startegic view of the markets and let the system do the tactical stuff of actually buying and selling.

Sounds like exactly the route I'm on, too.

Oh, been reading Technical Anaylsis of the Financial Markets; while I'm still not 100% thrilled with the lack of statistical analysis of the correctness of each indicator/pattern/whatever, I think there's some promising stuff in there. In particular, I have this theory about using it to generate likely paths for prices to take, the human picks the one or two it thinks look most believable, and then the program auto-trades it. Thoughts?

 

[rnicoll]

Oh, beyond "it exploded, but then I expected that"; I was restructuring my code to prep for some more theories I wanted to test, and now it produces completely different (arguably better, but different) results, for code that's meant to perform identically.

And this is why version control is CRITICAL. Naturally, I have a previous version, I'll be doing a careful analysis tonight.

 

[rnicoll]

Been dabbling with my new strategy, "Will it blend?". This is the no-stops, no-profit target, no-leaving the market MACD contrarian trader. Several things leap out immediately as really positive:

1. When I optimise, as the data set increases the parameters tend towards just one value across all available currency pairs.
2. Even on the currency pairs that lose money, losses are significantly smaller than the profits on the profitable ones. Also, of the major pairs, all but USD/CHF and EUR/GBP are profitable most of the time.
3. NinjaTrader puts its exit efficiency as 75%. Compare to 55% for my previous one. Both still roll in at an entry efficiency of around 55%.

It's not by any means pefect; the underlying stats need clearing up, it would be good to calculate some of the parameters from the data rather than brute forcing them, and draw-down is a real issue. Still, seeing how this works has been a real eye-opener for how unstable my last attempt was, and just how neatly things go when you're on the right path.

Going to try it in shares, soon. Still not too comfortable with the difference in results from Sep-Dec last year to Jan-Mar this year, but working on it.

And now, sleep!

North5, how's things there?

 

[north5]

Well I'm not looking at any trading strategies at the moment. I'm working on the 'overlay'. Quite a few ideas so it's a case of bringing them together and giving them some time whilst I monitor them to see how different things affect it. Then when I am happy will bring the strategies into the overlay.

 

[rnicoll]

Well I'm not looking at any trading strategies at the moment. I'm working on the 'overlay'. Quite a few ideas so it's a case of bringing them together and giving them some time whilst I monitor them to see how different things affect it. Then when I am happy will bring the strategies into the overlay.

Interesting, interesting. My latest one does very well in non-trending markets, and doesn't lose too much in trending markets. I figured I'd find a counterpart for it, and run both at the same time...

Tried it on stocks, actually. Looked great until I realised the commission was about double and there's no way I can trade as much as I want to, directly. Bleh. Probably going to work up an index futures strategy at some point, though, that looked interesting...

 

[north5]

I think I now have the pieces of the puzzle I am looking for. Now I just need to assemble them into something that does what I am conceptualising at the moment. It's quite complicated to get my head around at the moment exactly how it fits together although the underlying idea is reasonably simple. It incorporates ideas from everything I have worked on so far so when I can fit it all together it should be a big step forward.

The basis of it I now have as custom indicator, but the decision making based on the various parts of the indicator needs thinking about how to automate. All in all it is going to be a LOT of Mt4 code I suspect, so may take some time to get together. As well as time to monitor and tailor as the market throws things up.


I'm going to call it system 42 I think. Hopefully it will be my answer but I'm still not sure I know what the question is!

 

[rnicoll]

I think I now have the pieces of the puzzle I am looking for. Now I just need to assemble them into something that does what I am conceptualising at the moment. It's quite complicated to get my head around at the moment exactly how it fits together although the underlying idea is reasonably simple. It incorporates ideas from everything I have worked on so far so when I can fit it all together it should be a big step forward.

The basis of it I now have as custom indicator, but the decision making based on the various parts of the indicator needs thinking about how to automate. All in all it is going to be a LOT of Mt4 code I suspect, so may take some time to get together. As well as time to monitor and tailor as the market throws things up.


I'm going to call it system 42 I think. Hopefully it will be my answer but I'm still not sure I know what the question is!

I'm currently up to my eyeballs in maths. Re-implementing MACD turns out to be harder than it looks, as you need to first implement EMA, and then... eh, getting there, anyway. Hoping to have my latest strategy as a runnable version in time for market open next week.

At some point probably going to completely turn things around and make a pattern-recognition version, based on candlesticks. Mostly because doing it right requires insane amounts of CPU time, and I want to have a demonstration of what fast looks like when people start fretting over C++ vs Java without first considering architecture.

 

[rnicoll]

Okaaaay. Just started my auto-trader and the market exploded. I swear that wasn't me!

Random thoughts while I'm thinking about it:

1. Anyone else think the ability to attach notes to graphs/trades would be fantastic? Any trading systems out there let you do this?
2. Realising just how incredibly useful it would be to get an eye-tracker on a professional trader, see what they're using to actually decide on trades. I've seen them used in game design for UI testing, basically to reveal where the user actually looks.

EMA code reworked to be more efficient, and work in constant memory to ensure stability. MACD should be trivial next step, now, and with a little luck I'll have my new strategy ready to roll on the paper trading for tomorrow. I'll post a copy of the code, excluding strategies, once it has stabilised out.

 

[rnicoll]

Today's explosion was the autotrader. Finished coding the new strategy, got all the pre-run tests set up and confirmed, set it running on a papertrading account. First trade was a bit bumpy but made a profit. Second I had to close out early to move it to another computer, but looked like it would have made a small profit (instead of a small loss) given a bit longer.

Set it up on the trading server, and went to bed. Overnight, it made 4 roundturns, and lost on 3/4 of them, quite heavily on 2. So I go to check it against the NinjaTrader predictions... and they're not even close (NT shows a significant gain overnight). Got a theory on what went wrong, but doesn't look like it's going live anytime soon.

 

[rnicoll]

WOAH!

When they say you can't trade news with FXCM, they aren't joking. Okay, Collective2 use FXCM for their Forex price feed. Normally, I get prices that are sometimes better, sometimes worse, generally a little worse but not really an issue. Had a trade open selling USD/CAD, there's some news I missed being scheduled, and at midday the whole thing explodes and hits my stop-loss, managing to leave the market at 1.06680. A bug in my Collective2 code turns this into a sell-to-open. Which trades at... 1.06615. That's a 65 pip spread!

Now, sure, I shouldn't have been in the market, my system should have coped better, the trading engine should have correctly reported the trade, not saying any of that isn't true... I'm just stunned by the spread here.

NinjaTrader backtest, on the other hand, shows the strategy neatly flipping the trade around moments before the market spikes, and making a gigantic profit.

Whoever said the hard part of strategy development was the strategy?

 

[rnicoll]

Tonight, we'll have a lesson on dangers of backtesting. On the left, the results fron NinjaTrader. On the right, the results from my paper trading account with Interactive Brokers.

Note the trades of the EURCAD at 15:41 and 15:42. NinjaTrader says 1.6058 at 15:41, 1.6042 at 15:42. Now, not saying that can't happen, but even if it does, it means my system's trying to trade when the market's moving too fast.

I'll be over here with the drawing board. Yes, again.

BTW, anyone actually still reading this thread? I'm terrified I'm posting to a lot of people who wandered away weeks ago...

Edit: Scratch all that. IB's timestamps are all an hour out. Going to work on better analysis tools for Monday rather than re-doing this by hand, though.

Edit2: Looks like all the timestamps got screwed up, but I can't tell if it's the trades in IB, or NinjaTrader, specifically. Have given up on this run as being useful, will re-do the tests on Monday with better recording, and see how that goes.

 

[Vrothdar]

*wanders around a bit, looks toward the sky and trips over this thread*

I'm still reading. I'm just taking it all in, trying to learn from your experiences and because of the (yet again) little amount of time I've been spending working on my own system(s) over the last week I've little to add myself.

:edit: The last but one post on your blog has made me really want to go back to university and do either a computer science type degree or a maths degree or actually finish the chemistry degree I was doing *sigh* I love science me.

 

[rnicoll]

*wanders around a bit, looks toward the sky and trips over this thread*

I'm still reading. I'm just taking it all in, trying to learn from your experiences and because of the (yet again) little amount of time I've been spending working on my own system(s) over the last week I've little to add myself.

:edit: The last but one post on your blog has made me really want to go back to university and do either a computer science type degree or a maths degree or actually finish the chemistry degree I was doing *sigh* I love science me.

As long as I'm not just consuming database space talking to myself I've edited the last entry to say this, but it turns out the timestamps on the IB account were all wrong. Still not sure why the system didn't work, but at least I haven't found a fundamental critical flaw yet. Going to write export to Excel code for my trading platform over the weekend so I produce spreadsheets to wave at people more easily.
Also need to start tracking things like session, and account, so I can share a single database between the live and paper trading accounts more effectively.

Also planning on seeing if I can find an effective strategy for E-mini S&P, too. Looks like it's got the volume, trading hours and commission benefits of currency trading, but without the headaches of finding somewhere that'll do ECN with a decent API for auto-trading. I've got a crazy little first attempt that makes about 20%/year before taking into account margin, but I'm not too comfortable with it, or particularly how it works.

Glad the deep maths/comp sci stuff isn't scaring people off too badly. I'm about to do another post in a similar vein, should be up by about 1am.

 

[avextrex]

Hi guys,

Any idea wether EURGBP will go further down?

 

[rnicoll]

Hi guys,

Any idea wether EURGBP will go further down?

About the only thing I can tell you about EURGBP is it trashes every strategy I've ever tried with it.

 

[rnicoll]

Apologies for the ugliness, I'll do a spreadsheet version later. Meantime, here's the trades it's done so far, since 10:15pm (UK time). They're showing a decent resemblence to what NinjaTrader reckons this time, but still aren't all that close, which is concerning.

+---------+---------------------+--------+-------------+---------------------+--------+------------+
| SYMBOL  | ENTRY_TIME          | ACTION | ENTRY_PRICE | EXIT_TIME           | ACTION | EXIT_PRICE |
+---------+---------------------+--------+-------------+---------------------+--------+------------+
| GBP.USD | 2008-08-10 23:45:00 | SELL   |      1.9144 | 2008-08-11 00:32:00 | BUY    |     1.9157 | 
| GBP.USD | 2008-08-11 00:48:00 | BUY    |      1.9129 | 2008-08-11 01:03:00 | SELL   |     1.9151 | 
| EUR.AUD | 2008-08-11 00:57:00 | SELL   |     1.68805 | 2008-08-11 01:03:00 | BUY    |     1.6872 | 
| EUR.CAD | 2008-08-11 00:55:00 | BUY    |       1.593 | 2008-08-11 01:04:00 | SELL   |     1.5954 | 
| EUR.USD | 2008-08-11 00:51:00 | BUY    |       1.494 | 2008-08-11 01:12:00 | SELL   |    1.49615 | 
+---------+---------------------+--------+-------------+---------------------+--------+------------+

Oh, and I make that 53 pips profit since market open 3 hours ago :-D

 

[Victor90]

Interesting! =) Keep posting! =D

 

[rnicoll]

Well that was a bit more successful. Full log of trades until IB shutdown at 4:55 am attached, but let me summarise with... 165.5 pips in 7 hours :clap:

I'll be paper trading this for a while still, expect regular spreadsheets as I work.

 

[Vrothdar]

Congratulations mate. Hope the rest of the forward testing goes as well.

 

[rnicoll]

Interestingly, it made some money during the day, but towards the end it lost something like 200 pips in a row, which rather impacted the overall profitability! Total profit by 19:06, 186 pips, minus commission of 18 pips for a total of 168 pips.