Deadline June

[Adamus]

Dionysus,
v. interesting.

But optimise only if I really have to? I never wrote code for a system that worked straight away. Apart from some very long term trend following algorithms that had drawdowns lasting over a year.

are you saying, optimise and then do several different walkforwards over different date ranges showing distinct market conditions?

v. good point re parameter changes and effects on profits. i should make that into a rule.

 

[Adamus]

so what do you do with such parameters?

Another issue you have to be watchful of is the impact of a change in a parameter. Let's say you use a percentage change in your system. Let's also say that the percentage change setting of 12% gives you 75% profitable trades. If changing the setting to 11% then gives you 40% profitable trades, you are more than likely looking at curve fitting. A small change in the parameter giving a huge change in results should ring alarm bells.

Now that I recognise this problem when it comes up in an optimisation analysis, the question is, what do I do about it?

My first reaction is: don't try optimising that parameter any more.

But then I am still using a value for that parameter, whether I chose it via optimisation or just left it at the default or with the value I gave it or it acquired as I first wrote the code.

That strikes me as stupid - I would just be ignoring the problem but it wouldn't be going away.

So the next solution for dealing with the parameter is to surgically remove it from the code!

 

[Adamus]

terminology

I am about to launch myself into a new career as an algorithm trader.

mea dumbo

On another note, just realised that I misnamed my activities as 'algorithm trading' when I should be calling it program trading, or automated serial correlation arbitrage.

Algorithm trading seems to be an ill defined term that can include various other types of trading or just order execution.

I just used 'algorithm trader' since it seemed a suitable term to differentiate my technique from the traders out there reading charts and entering orders manually.

thought i wasn't so stupid but just realised the term is 'mechanical trader', rather than program trader. a bit like the name of the T2W forum...

And there are subgroups of mechanical traders.

People who run mechanical systems that they bought and that they don't know the mechanics of are "black box traders".

People who set up their mechanical systems to run without intervention are called "robo traders" - or rather, their programs are. Or just 'bots'.

But me, I'm just a plain vanilla mechanical trader, who gets his orders from his system everyday and enters them into the broker platform.

Whether this falls under the definition of 'automated serial correlation arbitrage' or not is a good question if you're feeling pedantic. Perhaps that only applies to robots, because they're 100% automated with just a start / stop button.

 

[Adamus]

next attempt

Market: British Pound futures

7 optimisations windows, 7 walk-forwards.

[FONT=Courier New]Optimisation            Walk-forward
Start       End         Start       End
2006-04-01  2008-03-31  2009-04-01  2008-06-30
2006-07-01  2008-06-30  2009-07-01  2008-09-30
2006-10-01  2008-09-30  2009-10-01  2008-12-31
2007-01-01  2008-12-31  2009-01-01  2009-03-31
2007-04-01  2009-03-31  2009-04-01  2009-06-30
2007-07-01  2009-06-30  2009-07-01  2009-09-30
2007-10-01  2009-09-30  2009-10-01  2009-12-31
[/FONT]

Here's what the BP future looks like in the first optimisation window. Goes up a lot.:-0 Goes up 0.2500 practically. That's $15,625. And irrelevant. :jester:

Plus there's a big gap in there in Nov 2007. I hope that's real and not the back-adjustment for my continuous contract going haywire.

 

[Adamus]

Minor delay caused by bug in CRB Data / Bar Chart software. Can't output continuous back-adjusted historical data series.

13 weeks and 4 days left.

 

[Adamus]

12 weeks 6 days left.

Plan for optimisation / walk-forward trading system discovery

(1) Optimise on a 2 year window

(2) no drawdowns for longer than 6 months

(3) no drawdowns bigger than $5K.

(4) more than 60 trades a year

(5) should make a profit both long and short

(6) only use stable parameters, i.e. parameters where small changes don't make big changes to the profits. Nice, rounded graph of equity against parameter value.

(7) if you don't find an optimisation that fits the rules, don't walk-forward or trade the market for that period.

(8) walk it forward 3 months

(9) build a collection of walk-forwards and collate the average drop in performance from the optimisation to the walk-forward. Do you still see positive expectancy? Enough to inspire real trading of an optimisation done on a window up to Feb 2010?

 

[Adamus]

got one

Found an optimisation that fits the rules.

The commission is the IB commission and I add slippage & spread to bring the transaction costs up to $30.70.

Total profit: $22000
Trades: 195
Max $ drawdown: $4070
Max len drawdown: 105 days
Max losng streak: 10
Win/unchgd/losing
weeks: 38/38/25


So the trading system:

Entries: break-outs from channel.

Midpoint of channel is linear regression of past 3 days.
Channel walls are 0.3 * 8 day ATR above and below.

Filter to allow trading only when a certain ATR-based fast / slow indicator combo is relatively low.

System can reverse.

Exits based on trailing stop 1.25 * 13 days ATR.

Just counting up my rules here: that's 8 variables

this is going to be grossly disappointing. I hadn't actually counted up my variables / parameters before, I'd always thought they made sense, so I could be happy with them.

This is the 'degrees of freedom' issue that I could never be bothered to look into, I just remember Pardo writing about 'not limiting your degrees of freedom'. :whistling

The walk-forward will speak the truth, no doubt......

 

[Adamus]

thought as much

and the walk-forward:

Hmm. Back to the drawing board.

 

[Adamus]

Better would be to optimize (if you really have to) over a small date range and then once you have the parameters you need, then run the system over a different date range pick lots of date ranges to cover different market conditions and see if your settings survive. You will quickly see if you have a system curve fitted for the range you optimized. This is a subtle but important difference.

Back at the drawing board, I'm going over what I know, mainly what I know doesn't work, and what people have said.

What you said, Dionysus, what is the difference between doing that, and optimising over the whole historical data (15 years)?

 

[Adamus]

Dire few days sorting out bugs in the software. CRB Data have a piece of software called "PowerGen" to create continuous back-adjusted contracts, but the date fields to begin and end don't work and I can't get past first level support to find out if this really is a bug. Plus some other faff kept me busy on the treadmill without getting me anywhere.

The forward-tests force me to abandon my current algorithm, and I'm looking around for something new to use.

I chose a simple MA trendfollower with RSI providing entry signals - and what an amazing performance across 25 markets!

It made 800K over 1,600 trades over 10 years - 35% winners / $7000 win - $3000 loss.

I thought I'd found that elusive algorithm that doesn't need optimising. Then I checked the equity curve (no, still haven't programmed in the Sharpe Ratio) and found that 100% of the profit came in 2008.

So I found the ultimate credit crunch algorithm :jester: - I think I'm going to do my testing on 1997 - 2007 to avoid this little siren song. Oh hold on, I can't generate the continuous contracts

 

[Adamus]

Finding sweet fanny adams in my search for systems.

The latest miserable equity curve. And that's optimisted.

Methinks I shall have mucho todo manana.

so the most lucrative systems seem to be the long term trend following but they require the most capital for any given risk of ruin. And probably more than I've got.

Long term systems are generally tradeable across all markets.

Medium term systems are perhaps more likely to be profitable across a particular sector or rather across chosen futures in that sectors. Does that make them inherently more risk-laden?

And a single market? Much easier to find a system that works, but that comes with a risk - obviously less data tested (1 mkt instead of 2,3,6 etc in back-testing)

 

[Adamus]

The latest miserable equity curve. And that's optimisted.

freudian slip :jester:

 

[Adamus]

counting down - 11 weeks and 3 days.

Is there no end to this madness?

Thought I'd found something useful, but further investigation revealed the extent of the drawdowns.

Just for the record, that's buying on dips below the 5 day MA when the RSI is above 50 (and vice-versa for selling) with a trailing stop 2 ATR below max profit achieved in position.

So what do I need? More trades? Maybe more trades with a profit target and a closer stop would help.

[FONT="Courier New"]
Total net profit	$26637.50000	Open position P/L	$   0.00000
Gross profit    	$95750.00000	Gross loss      	$-69112.50000

Total # of trades	      89	Percent profitable	      46%
Number winning trades	      41	Number losing trades	      48

Largest winning trade	$10087.50000	Largest losing trade	$-3100.00000
Average winning trade	$2335.36585	Average losing trade	$-1439.84375
Ratio avg win/avg loss	    1.62196	Avg trade(win & loss)	$ 299.29775

Max consec. winners	       4	Max consec. losers	       5
Avg # bars in winners	      14	Avg # bars in losers	       4

Max intraday drawdown	$-12850.00000		 
Profit factor   	    1.38542	Max # contracts held	       1
Account size required	$12850.00000	Return on account	     207%

[/FONT]

 

['Scope]

Cant you have a daily/weekly loss cut off?

EDIT: That was poor articulation.

I mean can't you program it to stop taking trades after £x losses in a x amount of days?

That would help with drawdowns no?

 

[Adamus]

I mean can't you program it to stop taking trades after £x losses in a x amount of days?

That would help with drawdowns no?

How do you turn it back on? If you wait until the system's making money again, then you've lost - or rather, lost the opportunity to make - that money.

I can see it might be a valid approach, but I never thought about doing this because it seemed like another way to curve fit. It's basically the same as having a filter built around some indicator from the market, except this time you don't know what the indicator is, it's just your equity you're looking at. It's a bit like making it a black box system.

I would play around with that if I'd ever heard of the approach being successful, but actually I haven't. Haven't heard of it being really evil either though.

 

[Adamus]

Banging my head against the wall - 10 weeks and 5 days to go

2 days off from the trading project for family.

CRB / Barchart.com are proving more and more useless at coming up with a fix for the continuous contract compilation problem I'm having with their PowerGen software. I can't believe I'm the only one having this issue, but still what can I do? Either I show them I'm not bluffing and swap data providers or I take the lame excuses and procrastination.

It's not really stopping my research, but it annoys me to be forced to make work-arounds the whole time.

However the research is not going well anyway. I fell back into an old time wasting habit stuck in front of TradeStation going round and round in circles chasing my tail testing little changes to inconsistent systems on far too much data and being hopelessly dissatisfied with any backtests.

I'm going to make myself a coffee and sit down and try to work out what sort of approach is going to give me the best chance of coming up with a tradeable system.

Don't hold your breath out there.

 

[Adamus]

How to use 10 weeks and 5 days profitably

Almost lost my project plan - so it's going on the web:

(1) Find system :jester:

(2) Have definite idea of slippage for each type of order in each market (stop, limit, mkt on close, mkt on open) by live trading.

(3) Write software module to record and display slippage on traded systems.

(4) Write software component to display roll-over alarms for markets

(5) Read risk disclosure and other IB documentation

(6) Write software module to update CRB historical data from TWS

At this point, only the first is absolute priority, the others can be prioritised once (1) is accomplished. (2) is dependent on (1) anyway, although (1) is somewhat dependent on (2) since it would be nice to know whether I am being too slack or too conservative with my slippage estimates. I have a fair few systems which would work great if the slippage was half the estimate I am using. So I have a nice circular dependency there. :whistling

 

[Adamus]

Great example of displacement behaviour

if at first you don't succeed, do something else instead

so now I've updated my project plan, perhaps I can get back to writing systems.

I have a simple approach, which hasn't been very successful so far:

(1) trawl around the internet looking for ideas for the basis of a system

(2) plug the system into TradeStation Easy Language and see what it looks like on the screen in terms of entries and exits on the historical data.

(3) tweak it with filters and alternative switching mechanisms to see how it behaves

(4) spend ages staring at a series of candles wondering what on earth characterises this particular stretch of history which makes the trades so bad

(5) confirm that it's a crap system that doesn't work reliably in any market with the kind of return I'm looking for

(6) go back to step (1)

There is of course a detour from step (4) which involves a clever bit of curve-fitting, some forward-testing, a slap to the forehead with "go straight to step (1)".

This is getting me nowhere. I can hear the silent masses of my audience telepathically reprimanding me "there is no Holy Grail, fool!" because I've set my sights too high.

So I need to re-evaluate: what is the minimum performance criteria I am prepared to trade for, that will get me £1K a month from £40K? That's a cool £12K p.a., which from £40K is 30% p.a.

And that's from mechanical trading on end-of-day data - without allowing for building up capital. Hmm, I may have to get a job.

 

[Adamus]

new theory

I'm currently looking into the possibility of trading multiple systems which only generate very few trades.

I have still got to do a lot of grunt work to optimise and walk-forward a statistically relevant number of these type of low frequency systems, but the idea stems from a system I am looking at right now which walked-forward successfully in 5 different markets.

Ironically I was looking for a high frequency system :jester:

I'm still using EOD data, despite the attraction of hourly or minute data, and I'm using a swing trading idea that puts limit orders above and below yesterday's close and exits any positions on the close. Plus a stop order to prevent run-away anthropogenic account balance change.

I filter it once with a moving avg to allow trading only in the direction of the trend, and then a second time with variations on falling ATR, or falling DMI, or falling CCI and such.

This is the sort of stats it produces:

_OutputTrdsShortMR  CRUDE OIL, BRENT-F01-Daily   06/30/1997 - 12/08/2009			 

	Performance Summary:  All Trades		 

Total net profit	$2000.00000	Open position P/L	$   0.00000
Gross profit    	$3200.00000	Gross loss      	$-1200.00000

Total # of trades	      19	Percent profitable	      63%
Number winning trades	      12	Number losing trades	       7

Largest winning trade	$1200.00000	Largest losing trade	$-410.00000
Average winning trade	$ 266.66667	Average losing trade	$-171.42857
Ratio avg win/avg loss	    1.55556	Avg trade(win & loss)	$ 105.26316

Max consec. winners	       4	Max consec. losers	       2
Avg # bars in winners	       0	Avg # bars in losers	       0

Max intraday drawdown	$-800.00000		 
Profit factor   	    2.66667	Max # contracts held	       1
Account size required	$ 800.00000	Return on account	     250%

Just to make my job harder, I am running seperate systems for long and short trading. i.e. I can optimise them seperately.

So over the weekend I'll do a massive walk-forward run with different markets and variations on the system.

 

[Yamato]

I've read the whole journal and I feel entitled to write one short post with my advice (advice from an unprofitable trader so far).

1) Deadline (of a few weeks) for everything to be ready to be traded with real money? Not a good idea in my opinion. Especially if no one is pointing a gun at you to start trading by then.

2) I sense you are expecting too much from your systems (a drawdown of a few months on a system is ok, and if you don't want a whole red year, then start testing with intraday data) and this leads to overoptimization, and this in turn leads to systems that don't work and get discarded. I'd rather find a good idea that works and with simple parameters produces no red years, set it aside and move on to the next idea, until you've built a few dozens different systems. By the way, if the same system works on different securities, then it's probably good. Personally I feel it's better to focus on quantity than quality, because that search for quality lead me to overoptimizing, producing worthless systems with a lot of parameters, and wasting a few years.

3) The fewer parameters you use, the more you can trust the system. In the same way, the fewer parameters you use, the more you can allow yourself to optimize them.

4) I am not sure if you mentioned something close to walk-forward optimization. In my experience it's best to optimize in the past 10 to 20 years or so, see if it works, and then use the system as it is. That is only in case you considered using an approach that every once in a while re-optimizes parameters. Personally I don't think it's a good approach.

5) After back-testing and optimizing and finishing a few dozens of simple systems (across as many securities as possible), I would advise to forward-test them (without real money), and keep those that work, or at least record the results for a few months, and at least not trade those that are not working.

6) Don't save money on buying data, in case you are. I found a few people (including myself) who were saving a few hundreds by not buying the needed (intraday) data for backtesting, and were being held back in their tests by that lack of data. Saving a few hundred dollars in this area will make you waste thousands in real trading, so it's not a good idea. So, I would buy all the data you will ever possibly need from the start. A reliable and cheap data provider is disktrading.com. That's what I used.