A small matter of programming

[rnicoll]

Having so far caused a long string of chaos across two threads and a blog, I figured I really ought to make a formal journal thread. For those of you who haven't seen my previous posts, you can expect a technology-heavy look at auto-trading using technical analysis.

The story so far:

I've been trying to trade by hand, on and off, for several years now. I tried spread betting for about a year, and made a decent profit for a while, but fundamentally I'm too nervous and twitchy. So, I decided to look at automated trading.

My first version was a seriously simple thing that used Stochastics to spot when the market was massively oversold/overbought. Second version used Bollinger Bands instead. Later revisions tuned the calculation of the bands, and worked on making them more statistically relevant to what I was testing for. I built a working trader against Interactive Broker's API, and set it running.

In short, that worked. In long, it requires monitoring of the market, and can easily be wrecked by a news event pushing the market outside the bounds expected by the trader. Acceptable, but not great.

The current plan involves using MACD and momentum to spot for traders getting too excited, and the market therefor likely to correct. I've been running backtests in NinjaTrader, which have been fantastically successful. For the last two weeks, I've been forward testing using Interactive Broker's paper-trading account, and that's not been so hot. As is, the code shows strange differences to using NinjaTrader to backtest, and the forward tests over the same time period. I'm so far unclear on whether the backtest or forward test is faulty.

However, the forward test results can be significantly improved if the amount of stake on each trade is tweaked depending on the success of the trades up to that point. Simply removing all trades with a stake below a certain level also improves the results noticably. Currently, I'm working on adding both of these into the auto-trader so it can do them live, rather than the current situation where they're calculated retroactively from the results.

 

[dcraig1]

The current plan involves using MACD and momentum to spot for traders getting too excited, and the market therefor likely to correct. I've been running backtests in NinjaTrader, which have been fantastically successful. For the last two weeks, I've been forward testing using Interactive Broker's paper-trading account, and that's not been so hot. As is, the code shows strange differences to using NinjaTrader to backtest, and the forward tests over the same time period. I'm so far unclear on whether the backtest or forward test is faulty.

That seems like a fundamental and really important issue to me. Do you think it is related to Ninja itself or to your own code ?

I'm developing my own testing/execution framework in Java an am always interested in this type of issue.

My approach is almost entirely event driven both for backtesting and live execution. Changing from one to the other then becomes just a matter of plugging in the different event sources at the bottom of the software stack. You then have as much as possible the same code paths for backtesting and live execution (give or take some order management in the live environment). I'm not familiar with Ninja, but this type of issue is very important.

PS There are a lot of other reasons to go for an pure event model including portfolio level testing and possibly optimization.

 

[rnicoll]

That seems like a fundamental and really important issue to me. Do you think it is related to Ninja itself or to your own code ?

You crazy kids and your scientific approach

Seriously though, yes it's extremely concerning, especially as the two strategies are as similar as I can get them in the two platforms. I think it's primarily down to how the market is sampled; a minute bar at 0 seconds past the minute is not the same as one at 5 seconds past, concerningly enough. However, without NinjaTrader's source code, it's impossible for me to tell exactly how it gets its data from IB, which is why I'm moving away from it...

I'm developing my own testing/execution framework in Java an am always interested in this type of issue.

My approach is almost entirely event driven both for backtesting and live execution. Changing from one to the other then becomes just a matter of plugging in the different event sources at the bottom of the software stack. You then have as much as possible the same code paths for backtesting and live execution (give or take some order management in the live environment). I'm not familiar with Ninja, but this type of issue is very important.

PS There are a lot of other reasons to go for an pure event model including portfolio level testing and possibly optimization.

Event driven, definitely. To date, I've been using 5 second bars (and order updates) as my driving event; that's changing across to incoming price ticks, which can then be filtered down. As stated elsewhere, at the moment prices come in, go to the instrument tracker for that instrument, and it then talks to the strategy. That's going to get flipped around so that it goes directly to the strategy. Means using meta-strategies for things like routing currency trades through a third currency instead of directly, but it should generally come out as a cleaner structure.

Right now, I have basic tick handling in, want to test it tonight for stability. Database writing is next, and is tricky because it needs to be done in a separate thread so that database slowdown can't jam up the incoming tick feed.

 

[dcraig1]

Have you considered ESP/CEP for event handling ? Esper - is free. I'm already way too far down the track to change horses, but if I was starting again, I'd have a good look at this.

 

[rnicoll]

Have you considered ESP/CEP for event handling ? Esper - is free. I'm already way too far down the track to change horses, but if I was starting again, I'd have a good look at this.

Trying shake a vision of jockey's horse-hopping mid-race.

Looked at it; first thought was "Well, it's not going to save much work, is it", followed by "...this time...". I can see it making strategy development a hell of a lot faster, actually...

Right now, I just want this accursed thing to work or not, so I can either get on with being fantastically rich, or with doing something else, but I'll keep it in mind if I'm going back to strategy development, thanks!

 

[rnicoll]

This morning's good news is that per-tick data from Interactive Brokers is a lot more reliable. Did stop randomly around 6am, but mostly it held together through the 5am reset, and I can easily accept that the interrupt was caused by network problems.

Tonight's job, then, is to adapt the strategy event system to smush tick prices into minute bars, and feed them through. Hopefully will have this running in time for a test overnight...

 

[rnicoll]

Not dead, but have a cold and generally am not so great. Also, tried trading by hand again (I blame the fever). Closed out for a £250 loss, and last time I check it was nicely hovering just in the profitable range. *sigh* I need to be able to sit through losses without fretting I'm doing the wrong thing, y'know?

 

[Arek Okrasa]

Hello Micoll

Could you tell me how auto-trader works? Is the software alright?
Also could you drop me a link with the web side where I can find auto-trader?

Thank you so much
Arek

Having so far caused a long string of chaos across two threads and a blog, I figured I really ought to make a formal journal thread. For those of you who haven't seen my previous posts, you can expect a technology-heavy look at auto-trading using technical analysis.

The story so far:

I've been trying to trade by hand, on and off, for several years now. I tried spread betting for about a year, and made a decent profit for a while, but fundamentally I'm too nervous and twitchy. So, I decided to look at automated trading.

My first version was a seriously simple thing that used Stochastics to spot when the market was massively oversold/overbought. Second version used Bollinger Bands instead. Later revisions tuned the calculation of the bands, and worked on making them more statistically relevant to what I was testing for. I built a working trader against Interactive Broker's API, and set it running.

In short, that worked. In long, it requires monitoring of the market, and can easily be wrecked by a news event pushing the market outside the bounds expected by the trader. Acceptable, but not great.

The current plan involves using MACD and momentum to spot for traders getting too excited, and the market therefor likely to correct. I've been running backtests in NinjaTrader, which have been fantastically successful. For the last two weeks, I've been forward testing using Interactive Broker's paper-trading account, and that's not been so hot. As is, the code shows strange differences to using NinjaTrader to backtest, and the forward tests over the same time period. I'm so far unclear on whether the backtest or forward test is faulty.

However, the forward test results can be significantly improved if the amount of stake on each trade is tweaked depending on the success of the trades up to that point. Simply removing all trades with a stake below a certain level also improves the results noticably. Currently, I'm working on adding both of these into the auto-trader so it can do them live, rather than the current situation where they're calculated retroactively from the results.

 

[rnicoll]

Can't brain today, I have the dumb.

Still tired and bleh from cold. Tried coding, wasted two hours mis-implementing some stuff that will need doing over later. Realised my entire threading model is wrong. Going to play computer games and eat sugary things instead, seems less challenging.

Arek, my reply is over in another of the threads you posted the message to.

 

[rnicoll]

Not dead, but have a cold and generally am not so great. Also, tried trading by hand again (I blame the fever). Closed out for a £250 loss, and last time I check it was nicely hovering just in the profitable range. *sigh* I need to be able to sit through losses without fretting I'm doing the wrong thing, y'know?

Reasons I think I'd be successful, if only I had a little more self confidence: now trading at a £1,150 profit, only two trading days later after I flee the market.

 

[Hotch]

now trading at a £1,150 profit, only two trading days later after I flee the market.

Huzzah!

 

[rnicoll]

I don't think I've ever been Huzzah!'d before. Not sure how to respond. £2,500 profit equivalent, though, 4 days after I fled the market.

Have completed basic adaption of the code to use ticks instead of bars as its data source. Tomorrow is sanity checks on this, and then leaving it running overnight to see how it recovers. The weekend should then see threading the remaining data through the framework, and with a little luck i'll have a full test version ready for Monday!

 

[rnicoll]

Test version running, continues to recover nicely from the 5am reset.

However, it also appears to get itself tied into knots, and can't properly track what orders are live. Eventually, this causes it to fail to place any new orders.

...and it still doesn't match NinjaTrader. While this seems to be caused by NinjaTrader doing something odd with the numbers, that's not actually so useful to know, as whatever it does it makes it more profitable...

 

[rnicoll]

NinjaTrader, it turns out:

1. Doesn't quite handle market open/close the market in the way I do. I'd argue I'm more accurate, but it's really up for debate.
2. Uses the mid-point as the trade price, give or take the pre-defined slippage. Which is great when the slippage reflects the market at all, which in frequently isn't true.

So, mystery solved, yay.

Getting the accursed thing to make actual money, not so much with the solved yet. Still, at least that's one problem down.

 

[rnicoll]

Getting there. Latest version appears to at least work-around most of the clunky weirdness that goes on, and that seems to really help profitability. For the twistedly curious, I present trades for this week as of 10:32 today.

Next, need to try getting through a week without any major bugs, and then we can talk about taking it live.

 

[rnicoll]

Oh that can't be good

"Following the announcements overnight, the Bank of England will be monitoring carefully the conditions in sterling money markets and will take appropriate actions if necessary to stabilise those markets."

- Bank of England|Publications|News|2008|Statement by the Bank of England, 15 September 2008


Soooo... in short, the money markets are badly screwed enough that be BoE is going to intervene openly, and is telling everyone they'll do this.

I think today may be a good day for me to find something else to trade!

 

[rnicoll]

So far, not so great for the auto-trader, at about 1.5% down on the day, which means about 8% down after margin.

On the other hand, we did lose a bank today, so maybe it's not the perfect day to judge on.

 

[rnicoll]

The week did not get better, now 15% down before margin: Trades as of 11:01 BST 19th September 2008.

Mostly, taking a break for a bit. Going to merge some of my code into JBookTrader, see if I have any more success working as part of a group. Once things stabilise, may try spread betting again.

 

[rnicoll]

Where's my Black Monday dataset?

So, one of the points made about the current financial markets is that quants didn't really plan for what if a highly unlikely event happens. Y'know, like the housing market dropping fast, the markets repeatedly moving several percent a day, that sort of thing. Which is odd, because a good grasp on statistics should have told them that given long enough, it was inevitable. I suppose, on the other hand, the year 2000 was inevitable and a bunch of developers didn't plan for that either.

Point is, in amongst my testing of my system on multi-month lumps of recent prices, why do I not have a Black Monday dataset covering the week before, and week after? Or a Dot-Com Bubble dataset centred around April 2000. Well, in my case, because I'm not aware of any good sources for this data; I've heard of places with some of the data I want, but mostly it's of mix quality, and really, truly, back testing needs per-tick and no-one seems to have that. Does anyone know where might have per-tick data for the Dot-Com boom, and any sort of useful data for Black Monday?

Second point is; why isn't "test your algorithm on the worst case scenario" a standard part of auto-trader design? Even if all it does is spot the volatility increase and exit the market fast, that's fine, but all auto-traders MUST have some way of going "This is over my head, I'm out of here". If they don't, you end up in messes like mine, where it lost 80% last week (on paper, thank goodness).

 

[rnicoll]

Going forward by going backward

Separate post because its a separate topic; I've brought the auto-trader up this week with my old strategy. My current theory on the strategy I've been testing for the last month is that the very high number of trades it executes was exaggerating a slight optimism in NinjaTrader's profit per trade (by under-estimating slippage). So, as is, that idea is back in the bin, and I'm seeing how my first plan works with the reworked architecture.