A small matter of programming

[MrGecko]

hi

just read the last couple of post on your journal, about backtesting on data...

... I must confess that i don't know how your backtesting is done, but, if you can import data from excel to backtest on in any way, some thing you might like to try is called "bootstrapping" (slightly different from the bootstrapping that goes on in FI pricing);

Say you have a series of 100 pieces of data (returns), and you are looking to backtest for a period of 20 days. A couple of ways to go about generating a data series is by:

* picking 20 of the 100 data points at random (i.e. each piece of data has probability 1/100 of being picked), string them together, and backtest your idea on this series; do that over and over until the cows come home, then average the results (bootstrapping is a little like a MonteCarlo simulation, except that is uses actual data, including the "outlying" days that MC's can miss

** Another way of doing a similar thing is to pick a starting data point and set this as "0". Then, draw a normal distrubition around "0" (i.e. with 0 as the mean) and pick your next data point, at random, from the distribution...

... I am not explaining it very well. Imagine your 100 pieces of data lined up vertically; set the middle point as "0" - and label prices up and down from 0 - 50 and 0 - -50 respectively. Then start at 0, draw up a normal distribution around it, and pick "1" - i.e. you try and "cluster" a series of 20 data points that were relatively close together in time - i mean, with the first method, you might get a series that starts with 19 oct 87 - big moves - followed by the last day of trading before christmas, with bugger all happening; it's not representative of realistic market conditions. Instead, you pick 19/10/87 as "0", then pick - at random - the next date's returns, which between 1 S.D. might be from 10th - 28th october; again, do this until the cows come home, and look for extremes / average results.


All of this you can do in excel, then (with any luck) run your backtesting software on the data - hope it's of some use to you.

You mention that you are using per tick data for your algorithm; can I ask why you elected for tick data instead of, say, 15 min intervals? The reason I ask is, at a tick by tick level, i assume the big boys will wipe the floor with you? I dont mean offence, but at that kind of granularity, massive processing speed, bare minimum of latency and so on are surely necessary to survive?

 

[rnicoll]

hi

just read the last couple of post on your journal, about backtesting on data...

... I must confess that i don't know how your backtesting is done, but, if you can import data from excel to backtest on in any way, some thing you might like to try is called "bootstrapping" (slightly different from the bootstrapping that goes on in FI pricing);

Ah, yes, bootstrapping. I did that back in university, had completely forgotten about it, good idea, thanks!

You mention that you are using per tick data for your algorithm; can I ask why you elected for tick data instead of, say, 15 min intervals? The reason I ask is, at a tick by tick level, i assume the big boys will wipe the floor with you? I dont mean offence, but at that kind of granularity, massive processing speed, bare minimum of latency and so on are surely necessary to survive?

In principal, I agree that if your algorithm is per-tick, the big boys will eat your account as a light snack. In practice bar data tends to have oddities caused by the sampling method used to convert the ticks into bars. For live trading, I'd prefer to do this sampling myself so I know what artefacts I'm going to be creating as side-effects. This is, in fact exactly what I do, to create minute bars.

For back testing, per-tick data is required to simulate the prices that orders will trade at, as accurately as possible. For example, moving from NinjaTrader's back testing, to forward testing with Interactive Broker's paper-trading account, I discovered that I get significant levels of slippage on stop-loss trades that just weren't shown on NinjaTrader because it assumed I could close at around 0.4%, while in reality 0.42-0.48% was more likely (0.52% in one case!).

 

[Adamus]

Hi, always interesting reading your posts!

Would you post a few pieces of info for your audience (or should that be for your spectators?):

- markets you are trading
- back-testing results & risk analysis figures if any
- historical data: from whom and how much did it cost

Good luck and don't stop now

 

[rnicoll]

Hi, always interesting reading your posts!

Would you post a few pieces of info for your audience (or should that be for your spectators?):

- markets you are trading

Right now, I'm doing paper-trading forward testing on AUD/USD, EUR/AUD, EUR/CAD, EUR/CHF, EUR/GBP, EUR/USD, GBP/USD, USD/CAD and USD/CHF.

- back-testing results & risk analysis figures if any

I've just broken my NinjaTrader install, which is all I had that could produce those _right now_, but I'll see if I can pull backtesting results from my own platform tomorrow. Some forward tests until then:

• Last week, using "Will it blend"
• This week, using "Catching daggers"

- historical data: from whom and how much did it cost

Good luck and don't stop now

Started with the year's back data you can get from Interactive Brokers, decided I wanted more, moved to IQFeed. Paying something like $65/month, but that includes exchange fees for NYSE data (and yes, having realised that, I grabbed T&S data for the entire Dow 30 from last week, which is VERY nice to have). That's about three years back data, which is plenty given I'm working at per-minute resolution.

 

[Adamus]

Very interesting. The reason I asked about the results stats is that I'm doing the same sort of thing myself and I'm interested to know what the results look like from your system which impressed you so much. I'm using EOD data and I don't think I've ever seen results that have really impressed me - but then using intra-day is probably a whole new territory. Plus, I'm looking for high frequency consistency that can be used to compound returns effectively. I'm getting close in my search but I'm not at the stage you're at. Plus I'm obsessed by slippage and commission obviously. What are you using in backtesting for that?

 

[rnicoll]

Very interesting. The reason I asked about the results stats is that I'm doing the same sort of thing myself and I'm interested to know what the results look like from your system which impressed you so much. I'm using EOD data and I don't think I've ever seen results that have really impressed me - but then using intra-day is probably a whole new territory. Plus, I'm looking for high frequency consistency that can be used to compound returns effectively. I'm getting close in my search but I'm not at the stage you're at. Plus I'm obsessed by slippage and commission obviously. What are you using in backtesting for that?

My general feeling is that the longer time scale, the more fundamentals matter, and therefore the less easy technical analysis is. Commission is calculated by NinjaTrader for me, that's not an issue (except that it "helpfully" doesn't do it by default). Slippage's a bigger issue, and is basically why I'm after per-tick data for use in my back testing, as it's the best way to be sure.

Remembered I had a second Windows box with NinjaTrader, so here's a bunch of results, plus the strategies used to generate them. All are done on per-minute data since the start of the year, with the default parameters included in the strategy code. "Mind the gap" is a bit odd, and you need to sure to give it market open/close times for the underlying market of the future it's used on, and ensure that "Exit on close" is unchecked, as it trades the market close/open gap.

If you get any of them to work in reality, do tell me

 

[Adamus]

I'd love to try them but I'm pretty wrapped up with what I've got already at the moment. But I like the idea of co-operating in principle. I'm sure you could benefit from a few of the result stats my setup churns out (he says proudly). My favourite is the 6 month / 12 month / 18 month profitability, which tells you what percentage of 6 month periods, 12 mth periods & 18 month periods were profitable over the complete back-test.

Re slippage, I just slap on $50 for slippage and commission to each trade. Unfortunately (or not perhaps) my brain has adjusted to see $-25 as not bad! Doh!

Are you live trading any of these yet? Perhaps you know already what your avg slippage is.

Although of course there are always black swans which come along to bump up the average - if you were long the SNP on 9-11 when the planes hit, you would have had $6000 slippage per contract.

I did see some bad slippage when I was trading 10 years ago, but that was a different age, using telephones and faxes.

 

[rnicoll]

Collective2

As the system I'm testing trades less frequently, meaning it's at least vaguely successful with Collective2, I've re-enabled the linkup. If you want to track how I'm doing (and keep in mind, its prices have high slippage), you can do so at:

Catching Daggers - C2 trading system which trades forex

 

[rnicoll]

I'd love to try them but I'm pretty wrapped up with what I've got already at the moment. But I like the idea of co-operating in principle.

Oh, it's not particularly meant as a request for help, although it's always greatly appreciated, more as a springboard for anyone else starting this to have some semi-successful examples.

I'm sure you could benefit from a few of the result stats my setup churns out (he says proudly). My favourite is the 6 month / 12 month / 18 month profitability, which tells you what percentage of 6 month periods, 12 mth periods & 18 month periods were profitable over the complete back-test.

Sounds interesting. I've got my own tools for producing stats, but knowing which stats are actually worth generating is definitely always useful

Re slippage, I just slap on $50 for slippage and commission to each trade. Unfortunately (or not perhaps) my brain has adjusted to see $-25 as not bad! Doh!

Are you live trading any of these yet? Perhaps you know already what your avg slippage is.

I'm doing forward testing using Interactive Broker's paper trading account. It uses the current market price to calculate the trade price, which gives a decent idea of slippage. Intruigingly, for "Will it blend", I get negative slippage because it's jumping in while the market's going against it, which is nice. Haven't calculated for "Catching daggers", which does a similar thing but tries much harder to find the exact turning point.

 

[Adamus]

big interruption there during the first half of the month! Must be a big filter sitting on your trade signal algorithm.

I should look at my back-test trades actually, probably doing the same.

 

[rnicoll]

big interruption there during the first half of the month! Must be a big filter sitting on your trade signal algorithm.

I should look at my back-test trades actually, probably doing the same.

Ah, no, that was me disconnecting the code from Collective2 while I fixed a bunch of problems. Like, losing $15k. Nasty bug that :-D

 

[rnicoll]

Wondering if I'm getting anywhere

Tried trading the output from my system by hand. Crunch, there goes another £200 down the drain.

Don't know how long I can keep doing this. My automated platform has successes here and there, but actually making money from it is apparently still a long way off, and my own trading hasn't gone well since April.

Pondering dropping currencies, as a lot of people do say they're too difficult to be worth it, but in favor of what?

Advice anyone?

 

[Adamus]

Giving up on a system because of one day's results is not exactly giving it a fair chance! If it's profitable in the long term, then you need to trade it long term and ride out the drawdowns in the meantime. Easier said than done, I guess.

 

[rnicoll]

Giving up on a system because of one day's results is not exactly giving it a fair chance! If it's profitable in the long term, then you need to trade it long term and ride out the drawdowns in the meantime. Easier said than done, I guess.

Yeah, I'm not very good at dealing with this psychologically, alas.


Anyway, got some backtesting done this evening. No real surprises, thank goodness, and there seems to be strong correlation between currencies that worked well yesterday, and work today. CHF crosses are always bad, CAD crosses always good, so far least, for example. PDFs attached for the curious. These are all early prototypes from the backtesting rig, and do not take into account commission either.

 

[rnicoll]

Tired...

So, my system makes 2.07% last week. So far this week, it's 2.18% down, in 13 hours. Got stopped out a bunch of times this morning, and BOOM there goes my profit. Fundamentally, a bounce system like this is always going to be risking a lot to make a little, and I wonder if I'm fighting the wrong battle here.

I could rework the order placement system to try getting more aggressively priced trades, but doing so might mean I miss too many perfectly good trades. Same goes for exit prices.

I could take the currency pairs that have been most hostile to my system (EUR/USD generally, AUD/USD and GBP/USD at the moment as well) and try trading them as breakout systems instead. I notice a lot of times when I get hit by stop-loss, there's two trades next to each other in related currency pairs (for example, buy EUR/GBP and then buy EUR/USD). I keep hearing how breakout systems don't work, though.

Oh, and just to make me really cheery, somewhere between here and Collective2 a close order on EUR/CAD got lost and is now showing 5% down on the day (fixed by hand). Oh, yes, and somehow it had a spread of 50-ish points earlier today on the same pair.

Just so tired of beating my head against this, you know?

 

[rnicoll]

Okay, rather calmer...

Looking over the last week, my strategy is right approximately 67% of the time, and the average losing trade is -0.19%. This means, to make a profit, profitable trades need to average about 0.13% a time.

Now, because it's a contrarian trader maximum likely profit can be calculated at time of entry, by looking at how far off "where it should be" the price is. Clearly, if this is less than 0.13% (which is particularly likely to be the case at times of high spread), it's safer not to trade. In fact, in extreme examples, it may even be more profitable for the strategy to operate in reverse.

So, this is going to be my next design change. I'll keep you all up to date, thanks for your patience through my panicking!

 

[Adamus]

Looking over the last week, my strategy is right approximately 67% of the time, and the average losing trade is -0.19%. This means, to make a profit, profitable trades need to average about 0.13% a time.

is that before or after commission and slippage? 67% is a good stat.

 

[rnicoll]

is that before or after commission and slippage? 67% is a good stat.

Before commission. Slippage is zero, as this is based on live, tradable prices. To be honest, while it sounds good, the amount that it loses on a bad trade is the real issue; 33% right but a higher profit per trade may actually work better, for example.

 

[rnicoll]

Just a quick update; been spread betting over some of the week. Gone mixedly, eventually gave up and pulled my cash out at about £4 down. Figured if I'm going to trade by hand, might as well do it properly, so have been dabbling with index futures and shares through Interactive Brokers. By the end of the day, slightly dizzy, and slightly less rich.

Learnt a lot, though. That while I can make money even in the current climate, the rate of change in the markets is far too high for me to deal with. I can't enter orders fast enough. Twice managed to screw up so badly I sold myself something.

In code terms, have been working on merging my stuff into JBookTrader, get a community working together on the codebase. More on that later. For now, I'm not dead, and haven't given up yet.

 

[rnicoll]

It compiles, ship it!

So... the credit crunch has caused my existing auto-trader to explode violently, showering virtual money all over the place. As such, I give you... the same trader but up-side down. So far today, 0.46% up before margin (after commission)!

I'll have a more detailed analysis of this later in the week.