Deadline June

[Adamus]

More historical data complexities

Another reason why my progress with the mechanical trading slowed right down is that I discovered how NinjaTrader's historical data manager which collects tick, minute and daily data from Interactive Brokers, FXCM and DTC (well, I import that manually).

The ticks from Interactive Brokers arrive on my machine without a timestamp. NinjaTrader timestamps it with the current computer time and saves it to the database.

The ticks from FXCM arrive ready-timestamped with the time on the FXCM data server, i.e. the current New York time. NinjaTrader just changes the time on the tick to the timezone on my computer - i.e. London time.

That's all well and great but on my computer something is going wrong. During March due to the onset of Daylight Saving Time in both the US and UK but at different times, I realised that NinjaTrader wasn't applying the DST to the time change it applied to the FXCM data - for reasons unknown, NinjaTrader continued to adjust the NY timestamp by -5 hours regardless of the fact that it was being transmitted an hour earlier.

So that screwed up the FXCM data by an hour from March 13th. Maybe I could have lived with that, if everything had gone back to normal on March 28th when Daylight Saving kicked in in the UK. It would have been 2 weeks worth of data out by an hour - tolerable. But in a bizarre mangling of its timezone calculations, NinjaTrader decided that NY is now running 6 hours behind us and so the data continues to be out by an hour.

I posted a few charts of the FXCM data vs the IB data on the NinjaTrader forum to get their attention, and one of their support staff had a long close look at it, but was unable to get the same problem to occur on his computer and that was that. I have to re-import all my data. And of course the tick data from FXCM is gone from their servers after 30 days so if I want to save it, I have to export it and adjust it all manually and then re-import it.

This is the NinjaTrader support forum thread.

Hopefully NinjaTrader running on my new machine will be able to handle the DST transition properly. I think I don't have time to export, adjust and re-import the tick data so I can kiss it goodbye. I guess I'll archive it - I have it from June 2010 - what a shame, almost a year's worth for the 21 highest volume currency crosses.

I haven't even looked at the DTC data to see how NinjaTrader handled the manual import. Let's just hope it was OK since it would have been done all in one day every quarter, rather than each weekend.

It took me ages staring at the chart and writing down the times in NY, London and GMT during the days of March to work out which ticks should have what timestamp. Considering my academic qualifications in school maths were all graded A, I find it insane that I have such issues with what is simple arithmetic - should I subtract 1 or not. It seems someone at NinjaTrader has got the same issue.

Rant over.

 

[grimesd]

Hi Adamus,

So what's your situation right now? You have one or more automated trading systems running now, right? Are they proving to be profitable? What kind of % profit are you making on a monthly basis?

You're now talking about starting discretionary trading on top of the automated systems to cover day-to-day living expenses, right? Sounds very risky to me. What you probably need is a stable source of income, and trading is anything but stable.

What's wrong with taking a job, something not too stressful that'll get you out and take your mind off trading. Surely you don't want to be looking at charts and waiting in front of your computer on a day like today!

In short - I don't think discretionary trading is the right thing to do, unless you think your automated trading is going nowhere.

 

[Adamus]

Get a job!

h grimes,

good to see you're still around.

Realistically my "job" will be discretionary trading. It might not be something that is generally accepted by family, friends and acquaintances as a "stable job", but it is a job, especially judging from the amount of work involved.

As for risk and stability, you have a totally valid view point - it would be good to have a stable income and not to undertake anything too risky.

However there are several factors that I am taking into account, some of which are minor personal details I didn't think necessary or right to post here, and some of which I'll point out now since it seems a good place and time to justify myself further.

I tried to get the type of work I used to do, freelance Java programming, through January to March. I had some interest and I realised I could get that sort of work again if I wanted, but I have some more work to do to prepare my hosted server to run the systems, and to prepare my laptop which Ill now use as a trading development workstation. This work is almost done.

I thought about the situation though and put together my plans and my budget and my forecasts - I uploaded an excel spreadsheet of the forecasting mechanism I used for predicting the income from my mechanical trading.

Even with conservative estimates of my possible income from mechanical trading and discretionary trading and using funds that I will have access to at various stages over the next 2 years, it's clear that I can manage this situation without taking on a paid job.

At least the next 3 to 6 months will be very interesting and some of the work I am now able to undertake with NinjaTrader on my new laptop (I presume) will allow me to be more confident about the system I make and in fact to make systems quicker.

The discretionary trading despite being challenging should be something I can handle, given 3 to 4 months to learn and it will also pay dividends because the two types of trading should complement each other and give me advantages I'd otherwise never realise.

I think the bench mark 10,000 hours of screen time required to turn profitable in discretionary trading apply to me as well - but reduced in total by my trading experience gained already on the mechanical side. Reduced by how much though? Good question. I'm being optimistic and hoping that I'll get on top of it within 4 - perhaps maybe 6 months. We'll see.

To answer your question directly, I am running two systems live. One is undoubtedly profitable, the other is just profitable. If you had asked me a month back, I would have said it was unprofitable. I actually want to get my systems up to a level where I can compare live results to backtest results and also put them through the money management analysis I'm planning. That still requires work to be done - lots of work - and meanwhile theyre both running at a leverage level I estimated as best back when I created them.

Actually they're running at half that original leverage - I turned down my bet size in January.

 

[grimesd]

Yes I'm still around.

Like you I have two systems running at the moment and they are both profitable. Its just a matter of monitoring them and resetting every month. I also have a discretionary account that I use for hedging against my automated trades, if they are obviously on the wrong side of the market. I tried discretionary trading before switching to automated trading cos I was just no good at it (not because of the strategy but because of me). Maybe you'll succeed where I failed.

 

[Adamus]

Thanks

Yes I'm still around.

Like you I have two systems running at the moment and they are both profitable. Its just a matter of monitoring them and resetting every month. I also have a discretionary account that I use for hedging against my automated trades, if they are obviously on the wrong side of the market. I tried discretionary trading before switching to automated trading cos I was just no good at it (not because of the strategy but because of me). Maybe you'll succeed where I failed.

Bad luck, maybe you just didn't want to do it enough - I used to think discretionary trading was inferior to mechanical trading, but as I've been learning, I decided I was wrong. I don't mean inferior as in how much you can make - I think that's impossible to compare - but I mean I thought it was just plain not such a good idea.

Since I changed my mind about it and for the reasons I talked about here to do with income, I now really really want to succeed at discretionary trading.

I believe that the human brain wll never be bettered by a computer when it comes to trading one market. It's not like chess where IBM eventually beat Kasparov or Karpov or was it someone else...

 

[Adamus]

Those who crashed and burned in style.....


Rogues Gallery

 

[Adamus]

IB handed me 12 points slippage on the EUR.GBP

Still haven't found out what moved the markets, but it put EUR.GBP through my stop @8937 so forcefully that IB gave me most of the fill at 8949.

That spoils the zero slippage fills on 25 previous executions.

What was the news then? Did they discover that it was actually David Cameron who the US had killed and buried at sea off the coast of Pakistan? I'd have thought that would have the opposite effect.

 

[Adamus]

your semi-automated strategy

Yes I'm still around.

Like you I have two systems running at the moment and they are both profitable. Its just a matter of monitoring them and resetting every month. I also have a discretionary account that I use for hedging against my automated trades, if they are obviously on the wrong side of the market. I tried discretionary trading before switching to automated trading cos I was just no good at it (not because of the strategy but because of me). Maybe you'll succeed where I failed.

Just realised who you are (I always get everyone's online names cofused) - didn't you used to have a journal where you talked about your semi-automted stuff? Is that the one system - and you created another?

 

[Adamus]

Trading holidays

Just because it was really difficult to trade last week between Easter and the Royal Wedding, since I was stuck in a holiday cottage in darkest Wales where there is no broadband (although I found out there is massive broadboand for the military) I decided it's definitely worth checking whether a system is worth trading on a bank holiday or even over the holiday periods, e.g. Christmas / NY or Easter.

It was good not to trade last week - my systems had a signifcant down week. And my memories of Christmas and New Year are of wishing I wasn't trading.

 

[grimesd]

Yes that's me. I have a journal, but haven't bothered posting to it recently - nothing has changed and it would be a bit boring to continue it. Since adding a second strategy based loosely on the first, it's just been a matter of monitoring and doing the monthly resets. I have a full-time job so I can leave all my profits in my trading accounts. It's only tax planning that's going to cause concern soon

 

[meanreversion]

Yes I'm still around.

Like you I have two systems running at the moment and they are both profitable. Its just a matter of monitoring them and resetting every month. I also have a discretionary account that I use for hedging against my automated trades, if they are obviously on the wrong side of the market. I tried discretionary trading before switching to automated trading cos I was just no good at it (not because of the strategy but because of me). Maybe you'll succeed where I failed.

How can you know if your automated trade is "obviously on the wrong side of the market"? I run three systems and I am rubbish at second guessing them. I know this for a fact, because I've kept a record of all the instances where I've skipped signals and it has cost me dearly.

As Mark Douglas says, anything can happen (rule no. 1).

 

[grimesd]

How can you know if your automated trade is "obviously on the wrong side of the market"? I run three systems and I am rubbish at second guessing them. I know this for a fact, because I've kept a record of all the instances where I've skipped signals and it has cost me dearly.

As Mark Douglas says, anything can happen (rule no. 1).

I only hedge my systems on the odd occasion. Like today on EURJPY my systems were both short, so I entered a long in my SB account (which is looking good so far). My systems are still short EURJPY.

 

[Adamus]

Re: semi-automated strategy

I only hedge my systems on the odd occasion. Like today on EURJPY my systems were both short, so I entered a long in my SB account (which is looking good so far). My systems are still short EURJPY.

Grimes, what time frame do you trade on?

 

[Adamus]

Re: TurningPoints Improvements Needed

On CHF/JPY - to me, the most interesting minor pair, but this trade by the Turning Points system was all wrong. It didn't look particularly unlikely when it entered - I marked the target on there - but it seems to have overestimated the swing by about 50%, when I look at the last couple of days by eye.

The trade yesterday was much more on target so I need to check out why it changed. I expected the system would stay in synch for longer, but in this case it has got right out of whack after being relatively in synch.

The long entry ysterday was the tail end of a virtual trade that started last week which I only entered just yesterday. It was quickly reversed. Maybe there's something going on there in that reversal part of the system.

 

[grimesd]

Adamus, I look at the daily and H4 time-frames, but my systems trade from 2 & 15 Minutes.

As for the EURJPY - My systems are both still short and I'm still long. I'll exit my hedge when:
a) Price closes below support/above resistance.
or
b) One of the systems exits its position.

 

[Adamus]

Adamus, I look at the daily and H4 time-frames, but my systems trade from 2 & 15 Minutes.

As for the EURJPY - My systems are both still short and I'm still long. I'll exit my hedge when:
a) Price closes below support/above resistance.
or
b) One of the systems exits its position.

2 mins timeframe - presumably just for entries? That's really quick.

The reason I asked is because my systems trade on the 1 hour time frame and I don't think I'd have enough time to work out if I needed to put on hedges on all the markets I'm trading. I wouldn't want to do it anyway because I'm trying to make systems that need no outside intervention.

I remember your posts now, I believe you are of a different school of thought on that one.

 

[Adamus]

Re: IB handed me 12 points slippage on the EUR.GBP

More of the same and worse.

Filthy stinking ******* slippage - on the GBP/USD as well this time.

For such a supposedly liquid market, I believe something must have gone wrong at Interactive Brokers to produce such a gap on the charts.

I will check FXCM data as well.

 

[Adamus]

Re: IB handed me 12 points slippage on the EUR.GBP

More of the same and worse.

Filthy stinking ******* slippage - on the GBP/USD as well this time.

For such a supposedly liquid market, I believe something must have gone wrong at Interactive Brokers to produce such a gap on the charts.

I will check FXCM data as well.

FXCM isn't that much better. I think it's fair to say I might have got just as bad slippage.

This is more what I'd expect on CHF/JPY, not on cable.

So, 25 points slippage on my GBP/USD stop, and 11 points slippage on my EUR/GBP stop.

 

[Adamus]

Re: TurningPoints Improvements Needed

This Turning Points trade was a definite no-hoper.

The idea of the system is that it catches the swings early after identifying a turning point / swing high/low.

In this situation, 'TP' identified the swing low, but waited and waited and waited for 3 or 4 bars until giving a signal, by which time the swing was over and the new one had just started.

I see what has happened here. The first part of the signal - the swing low - kicked in but the second part of the signal - the MA cross-over - was lagging badly, for whatever reasons, I can look into that later. But what I should have done is to make sure that the signal doesn't trigger if the MA cross-over has taken so long that the market has moved right up the swing and starts turning again. That data is monitored by the system, it's just not applied to this part of it - stupid!

Of course there are many situations like this where the swing doesn't stop but goes on to give a storming winner. I could check back over backtest results to find big losses and I will probably see a lot of these situations causing them, but many of these big losses will not occur since the system will often reverse for a smaller loss.

I'll have to run the backtest with reversing turned off to find them all then.

 

[Adamus]

FMT system needs maintenance

This is my other system - just runs on GBP/USD between 6 and 7 every morning.

Today it went short at 6am and usually that would be it. Either it hits target or it gets stopped out.

But on the 6:15 bar (it runs on 15 min bars) it reversed and that cost me the trade.

I don't know how many times it reversed over the last x years but I want to test it with the ability to reverse disabled.