Deadline June

[Adamus]

I can see the book, although it looks like it could use some attention from a UI specialist.

So what do you do with it? It looks to me like you have to be fast.

 

[DionysusToast]

Actually - it won't help you much for Forex.

For fatter contracts try - ES, FESX
Thinner contracts try - NQ, YM, Bund

Fat & thin contracts work slightly differently in terms of how you use the DOM & the tape.

If you want some pointers - I can tell you what sort of things I use - but I would rather not discuss such things on the board. Hence the PM of a Skype id.

Cheers

DT

 

[Adamus]

Comparative Analysis of Trades using two different data providers

I'm concentrating on the GBP-USD from 2009-09-01 - 2010-06-30, using the same system.

This is the first run:

Series 1 Tenfore minute data
Series 2 Disktrading.com tick data

Start to Finish: 301.23 days
Time in market series 1: 24.91 days
Time in market series 2: 24.43 days
Trades in 1: 150
Trades in 2: 178
Trades from series 1 not corresponding: 71
Trades from series 2 not corresponding: 99
Trades overlapping in same direction: 75
Trades overlapping in same direction within 75% similar profit: 28
Both in market in same direction: 10.15 days
% simultaneous: 40.74
Trades against each other: 5
Series 1 profit: -1455.00
Series 2 profit: -2415.00
% similar profit: 165.98

 

[Adamus]

Series 1 Tenfore minute data
Series 2 FXCM minute data

Start to Finish: 300.94 days
Time in market series 1: 24.66 days
Time in market series 2: 31.52 days
Trades in 1: 150
Trades in 2: 188
Trades from series 1 not corresponding: 49
Trades from series 2 not corresponding: 87
Trades overlapping in same direction: 98
Trades overlapping in same direction within 75% similar profit: 39
Both in market in same direction: 13.82 days
% simultaneous: 43.84
Trades against each other: 3
Series 1 profit: -1455.00
Series 2 profit: -9170.00
% similar profit: 630.24

 

[Adamus]

In terms of simple profit difference, it's massive, but equity curves are so similar

Series 1: Tenfore data
Series 2: IB historical data

Start to Finish: 301.69 days
Time in market series 1: 24.91 days
Time in market series 2: 41.68 days
Trades in 1: 150
Trades in 2: 239
Trades from series 1 not corresponding: 26
Trades from series 2 not corresponding: 115
Trades overlapping in same direction: 123
Trades overlapping in same direction within 75% similar profit: 52
Both in market in same direction: 20.13 days
% simultaneous: 48.30
Trades against each other: 2
Series 1 profit: -1455.00
Series 2 profit: -9825.00
% similar profit: 14.81


Tenfore data equity curve on the left, FXCM, then IB historical (not from live feed), and DTC last

 

[Adamus]

The results with the Tenfore data correspond to the FXCM data better in terms of a similar time in the market, less trades not corresponding, more trades overlapping, more similar profits, number of days in the market at the same time, % similtaneous, fewer trade mismatches.

But the return on the FXCM is nothing like the return from Tenfore or DTC. It might just be random though. I haven't got the DTC equity curve yet, I'll upload that now.

 

[Yamato]

Seen from here, after a superficial analysis of your four posts above, if you have 3 sources for your data (fxcm, tenfore and disktrading.com), and 2 of them match more than the other, I would immediately discard the one that is the furthest, so fxcm, if I am not mistaken. I would discard it and never look back.

I can't say more because I would be too superficial to speak with so little knowledge about your analysis. However that principle in my opinion holds true: 3 diverse sources of data, 2 almost match and one doesn't... I'd discard the one that doesn't and never look back. Then I would maybe get IB's data, and see which one is closest to IB, between disktrading.com and tenfore, and I would discard the furthest one. You would probably end up keeping disktrading.com, like me, for both reliability and very limited costs.

I also gather that we haven't been doing exactly the same type of tests. I've been doing tests of back-testing vs forward-testing, whereas you've been doing back-testing vs back-testing (on the same period by different data vendors). I know it's close, in that we both tested different data (even though I did different data on different platforms), but it's not exactly the same, because you still don't know if your execution will match your back-testing. Of course, the common and understood assumption is that we're testing the same exact period and system, no matter what platform or data we are using.

In my opinion as soon as you establish which data provider is feasible both in terms of quality and costs, you should do this next step: verify if your forward-tested trades (day by day, live, on IB TWS), match the same trades made in back-tested mode on NinjaTrader (I think that's what you are using for your back-tests).

Since your live trading is on IB data, if you could find a way to also back-test on IB data, that would be ideal. And you should have embraced it from the start, without even considering any other source of data. Of course, if you want as many as 10 years of data, that is not possible because IB only allows you to download the most recent period, probably up to a couple of years. My choice was disktrading.com and tradestation for back-testing and IB TWS and excel for live execution.

Of course this final test I recommended has some validity only because I have already gathered one year of forward-tested data, which you might not have done yet, because you've started trading the systems only recently (and maybe you haven't even started recording their live trades, in forward-tested mode).

As far as my own tests, I've only made one of 10 I want to make, but so far I can conclude that the trades match closely enough to continue things as I've been doing them, and that I am executing the same systems I back-tested. But thanks to these tests, I've corrected a minor problem which was causing bigger differences than I have now. On tradestation I was excluding trades taking place on friday, because they lasted longer than 24 hours, and I was confusing them with trades taking place on holidays (I cannot exclude those on tradestation correctly) or before data holes. Now i corrected the code and am including those trades made on friday night and closed on monday morning. However the results were matching before as they are now, because despite not counting those trades on friday the systems performed similarly. Yes, there were a lot less trades and I couldn't understand why.

 

[Adamus]

NB I edited those last reports since you posted.

You're right - none of these tests help me until I've got some data from IB live. The IB historical data that they provide from their history farm is poor (I find their history farm incredibly slow and difficult to get a good connection to, and then they disconnect me due to their "pacing violations")

I'm currently forward testing with the system running since Monday on the IB paper trading account, which they assure me is the same feed as the live account. However even then the executions are simulated and that adds another factor to muddy the results, because in real life I may not get filled on my limits at the ends of bars and the slippage on the stops will be different.

I am seriously considering to run it live from next week too.

So in essence, the live feed trading will be the definitive result set to compare the other data against and I won't have any data on that until I've traded it. Perhaps end of September if I start trading it on the 1st.

 

[Adamus]

Actually the profit is a rubbish statistic to compare, especially when the final profit is close to zero.

I guess it would be a lot better to compare Total Wins + Total Losses.

I'll see if I can change the code and put the results in a table too.

 

[Adamus]

the low-down on IB forex live feed

From the horse's mouth:

You can find more information regarding our forex service on our website. Click on the Trading menu and select Product Listings. Then, refer to the Forex tab. more information is contained in the following link:

http://www.interactivebrokers.com/en/trading/pdfhighlights/PDF-Forex.php

Our own IDEAL PRO forex network is a combination of real time prices from 13 of the world's largest FX dealing banks. The market structure is like an ECN where you trade with other IB clients as well as with the liquidity providing banks.

You may also review a list of the participating interbank dealers from the TWS. Simply, right-click on the market data line of any IDEAL PRO forex pair and select Contract Info --> Details.

e.g. for EUR-GBP:

Exchange IDEAL, BARC2FX, BARCFX, BOAFX, CITIFX, CSFX, GSFX, HSBCFX, IBFX, IDEALFX, JPMCFX, LHMNFX, MSFX, NATIXISFX, NOM2FX, NOMFX, RBCFX, RBSFX, UBSFX

The tight spreads and substantial liquidity are a result of combining quotation streams from 13 of the world's largest foreign exchange dealers which constitute more than two-thirds1 of the market share in the global interbank market.

 

[Adamus]

As far as my own tests, I've only made one of 10 I want to make, but so far I can conclude that the trades match closely enough to continue things as I've been doing them, and that I am executing the same systems I back-tested. But thanks to these tests, I've corrected a minor problem which was causing bigger differences than I have now. On tradestation I was excluding trades taking place on friday, because they lasted longer than 24 hours, and I was confusing them with trades taking place on holidays (I cannot exclude those on tradestation correctly) or before data holes. Now i corrected the code and am including those trades made on friday night and closed on monday morning. However the results were matching before as they are now, because despite not counting those trades on friday the systems performed similarly. Yes, there were a lot less trades and I couldn't understand why.

I think a lot depends on the type of systems we are trading. I think the system I am testing must be really sensitive to small changes to the high and low of the bars between the data sets. The systems you use seem to be more neutral.

It's probably something I should bear in mind when invented more systems - there must be less risk that the backtesting results deviate from live trading results.

 

[Yamato]

I would say the most important factor might be the time-frame. I only use 15-minute timeframe on every system. And also my entry method in conjunction with that. I enter at specific times if the conditions are there. So no matter what platforms and data sets, the trade always enters at e.g. 10 PM CET or it doesn't enter at all. This itself excludes a lot of other potential differences and combinations.

 

[Adamus]

The Tenfore feed according to the IQ-Feed sales rep:

Tenfore includes about 30 conributors. You can get individual quotes from each contributor or a composite quote.

which probably makes it better than the IB live feed.

 

[Yamato]

Well, since IB data is impractical for long periods of time, if you need many years, your choice is now between Tenfore and disktrading.com.

 

[meanreversion]

Adam, the issue I'm facing at the moment is with the mandatory shut down each night of TWS. I can re-start it using TWSStart, but I seem to need to manually click on each market in Amibroker in order to force a backfill of data from TWS. Is this an issue you've come up against? (I appreciate you don't use Amibroker).

 

[Adamus]

Better analysis stats

I would say the most important factor might be the time-frame. I only use 15-minute timeframe on every system. And also my entry method in conjunction with that. I enter at specific times if the conditions are there. So no matter what platforms and data sets, the trade always enters at e.g. 10 PM CET or it doesn't enter at all. This itself excludes a lot of other potential differences and combinations.

That is probably a massive factor, I totally agree. I'm getting more informative stats by comparing the total profits + total losses.

Tenfore compared to:                     IB Historical  FXCM         Disktrading
Days start to Finish:                    301.69         300.94       301.23
Days in market Tenfore:                  24.91          24.66        24.91
Days in market other:                    41.68          31.52        24.43
Trades in 1:                             150            150          150
Trades in 2:                             239            188          178
Trades from series 1 not corresponding:  26             49           71
Trades from series 2 not corresponding:  115            87           99
Trades overlapping in same direction:    123            98           75
 .. overlap, same dir, & 75% sim profit: 52             39           28
Days both in market in same direction:   20.13          13.82        10.15
% time simultaneous positions:           48.30          43.84        40.74
Trades against each other:               2              3            5
Series 1 total profit + -losses:         47795.00       47795.00     47795.00
Series 2 total profit + -losses:         76895.00       60470.00     60645.00
% similar profit + -losses:              62.16          79.04        78.81

 

[Adamus]

I only have 1 year of IB historical data from their servers. It doesn't go back any further.

For Tenfore, I have 18 months of historical data from their servers. I haven't asked if they have more data available - that might be worth purchasing although I might be fantastically expensive.

The same for FXCM.

After doing this analysis, I can't honestly say I'm any wiser. I think I'll leave the final word until I've got a month of live trade data from the system on the live IB feed and then I'll run it again but only using a month. I guess I'll have about 30 trades in the sample.

I'll also have the IB live feed data collected by NinjaTrader as I keep it running on my machine 24x7 so in future I'll be able to use that in backtests and another analysis like this.

Intuitively I think the Tenfore data is the best, and it will be interesting to compare it with the IB live feed.

 

[Adamus]

Adam, the issue I'm facing at the moment is with the mandatory shut down each night of TWS. I can re-start it using TWSStart, but I seem to need to manually click on each market in Amibroker in order to force a backfill of data from TWS. Is this an issue you've come up against? (I appreciate you don't use Amibroker).

Hi MR

I do experience something like that with NinjaTrader - if IB TWS disconnects then I have to cycle through a chart for each forex pair or go to the Historical Data Manager and download the day's data.

I thought TWSStart had an option to keep TWS open by flipping the shutdown time from AM to PM and back again? That's what I do manually after someone told me about it.

 

[Yamato]

I would say, all things considered, disktrading.com is the way to go. You just said that you didn't ask about Tenfore pricing but it might be very expensive. IB is not a possibility... there are no other options. I would just go with disktrading.com and from now on exclude all other worries about data vendors and only focus on your systems. One type of vendor tries to grab you by the balls, obviously, such as Tenfore... (subscriptions, etc.) whereas disktrading.com gets the job done, gives you all the data you want for all the markets you want, and for very cheap. I would not have any doubts. Go to the vendor that sells data to everyone, don't go to Tenfore who seems to do it as a personal favor to you. They don't even have a page with prices on their web site I suppose (by how you described them). Trust me. There's many compatibilities issues that you will not have with the vendor who is used to selling to a lot of people (disktrading.com). I am not getting paid by them, so you should listen to me. Just as you picked IB because everyone has an account with them, you should pick disktrading.com. Of course, if you can spend 1000 dollars per symbol, then go for tickdata.com. Which I will never do unless I have a bank account with 20 millions dollars in it. Remember that discussion with that guy about disktrading.com we had months ago. He said there were many other alternatives and after several months of questions it ended up that it was all bull**** and it's either disktrading for 15 dollars per symbol (if you buy two symbols) or tickdata.com for 1500 dollars per symbol and you don't even know what's in the box.

 

[Adamus]

I'm certainly not going to stop using the disktrading data, that's for sure, but I'll always do a direct comparison of the trades of any new system against the Tenfore data for 2009 - present, and if they are reasonably similar then I'll do backtest over the whole disktrading data back to 2000.

I will also carry on using the Tenfore data despite the cost ($100 per month more or less) because I think it's probably closest to what the IB live feed is.

Obviously if Tenfore is just as different from the IB live feed as disktrading, then that's another situation, but I won't know that until September really.