Which data should I use for backtesting?

[david321]

Hi all.

I'm a bit confused here, please help me out

I have spent a long time backtesting some strategies using free Yahoo daily data. I have found some strategies that work well. To my surprise the strategies don't work well when I replace the Yahoo data with data from a trading platform like FXCM (the time period is the same). Now I'm wondering. Isn't it because the quotes from a trading platform doesn't follow the "official" quotes? Like the SP500 index and the SPX500 future? Does a trading platform have to follow the official quotes or can it adjust the quotes a bit depending how the future is traded on that particular platform?

I'm a bit confused. I thought that I could rely on the free Yahoo data when developing and testing strategies. Should I use data from a trading platform instead to see what a strategy will do "in real life"? If yes, which trading platform has quotes most similar to Yahoo's quotes?

Can you follow me?
I hope somebody out there can help me out

 

[Rhody Trader]

If you were talking an exchange-traded product like stocks or futures you shouldn't experience any real difference between Yahoo and any other source. Forex is different because there's no single pricing source.

What instrument(s) are you looking at?

 

[david321]

If you were talking an exchange-traded product like stocks or futures you shouldn't experience any real difference between Yahoo and any other source. Forex is different because there's no single pricing source.

What instrument(s) are you looking at?

I would like to trade the SP500. As far as I know then the only way is to trade SPX500 future index. Or are there any other ways to trade it?

My strategy might be very sensitive to minor quote differences because it opens and closes one trade within a day. That could be the reason why the result is so different depending on the data source...

 

[Rhody Trader]

The S&P futures will not have the same price as the S&P 500 index. They will move the same way intraday, but you don't want to use the raw prices of one to trade the other without adjusting for the spread.

 

[PollyM]

I would like to trade the SP500. As far as I know then the only way is to trade SPX500 future index. Or are there any other ways to trade it?

My strategy might be very sensitive to minor quote differences because it opens and closes one trade within a day. That could be the reason why the result is so different depending on the data source...

Could you run me through how you backtested an intraday strategy using daily data?

 

[david321]

The S&P futures will not have the same price as the S&P 500 index. They will move the same way intraday, but you don't want to use the raw prices of one to trade the other without adjusting for the spread.

I actually already adjusts for the spread. I use the raw data and after each trade I substract 0.1% commision. The 0.1% commision is calculated based on the spread beeing 1 point like at IGindex or Cityindex (1 point divided by approx. 1300). When doing that my strategy earns 200% during the last 3.5 years. When I use data from FXCM with no commision then the strategy only earns 40% in the same period.

I wonder if trading platforms like FXCM have hidden spreads build into the data? So when they advertise and saying "free trading with no spreads" or promise a very small spread then extra spread is added to the data. Like they receive your trading bet and make the same bet at another company with lower spreads thereby making money on the difference.

 

[david321]

Could you run me through how you backtested an intraday strategy using daily data?

Well you could e.g. use the open, close, high or low for the day. These numbers are intraday information

 

[PollyM]

Well you could e.g. use the open, close, high or low for the day. These numbers are intraday information

And is that what you have used?

 

[PollyM]

I actually already adjusts for the spread. I use the raw data and after each trade I substract 0.1% commision. The 0.1% commision is calculated based on the spread beeing 1 point like at IGindex or Cityindex (1 point divided by approx. 1300). When doing that my strategy earns 200% during the last 3.5 years. When I use data from FXCM with no commision then the strategy only earns 40% in the same period.

I wonder if trading platforms like FXCM have hidden spreads build into the data? So when they advertise and saying "free trading with no spreads" or promise a very small spread then extra spread is added to the data. Like they receive your trading bet and make the same bet at another company with lower spreads thereby making money on the difference.

Putting aside the trading platforms, you have at least tipped the balance into profit. This is an important milestone. Which backtesting software have you used or did you develop one yourself? Are there many variables that you need to input or is it a simple calculation? If you know the source codes for these systems then that should reveal why the differences occur.

 

[Rhody Trader]

I actually already adjusts for the spread. I use the raw data and after each trade I substract 0.1% commision. The 0.1% commision is calculated based on the spread beeing 1 point like at IGindex or Cityindex (1 point divided by approx. 1300).

I wasn't referring to the bid/ask spread. I was referring to the spread between the futures contract and the cash index.

 

[david321]

I wasn't referring to the bid/ask spread. I was referring to the spread between the futures contract and the cash index.

You're right. I didn't take into account the spread between the futures contract and the cash index. This can explain the lower strategy result because of extra trading commision.

 

[david321]

Putting aside the trading platforms, you have at least tipped the balance into profit. This is an important milestone. Which backtesting software have you used or did you develop one yourself? Are there many variables that you need to input or is it a simple calculation? If you know the source codes for these systems then that should reveal why the differences occur.

I have developed one myself in C Sharp. It's a simple calculation and it's more like a walk-forward test. I think the difference occur because of the hidden spread between the futures contract and the cash index. I haven't taken that into account. If I open and close a tradw within a day and extra spread is added compared to the cash index, then of couse the profit must be smaller. Now I have to find a spread betting company that offers low ask/bid spreads AND low hidden spreads. The market isn't that transparrent isn't it?!

 

[PollyM]

I have developed one myself in C Sharp. It's a simple calculation and it's more like a walk-forward test. I think the difference occur because of the hidden spread between the futures contract and the cash index. I haven't taken that into account. If I open and close a tradw within a day and extra spread is added compared to the cash index, then of couse the profit must be smaller. Now I have to find a spread betting company that offers low ask/bid spreads AND low hidden spreads. The market isn't that transparrent isn't it?!

As I mentioned earlier, it's commendable that you have developed a profit-making system. I understand you are still saying it's profitable, in which case, simply adjust your stake to suit your targets. A positive is still a positive, and vice versa.

If you're borderline then, yes, you may want to revisit some of your root assumptions and possibly re-model. But otherwise, I think you're doing well.

 

[david321]

As I mentioned earlier, it's commendable that you have developed a profit-making system. I understand you are still saying it's profitable, in which case, simply adjust your stake to suit your targets. A positive is still a positive, and vice versa.

If you're borderline then, yes, you may want to revisit some of your root assumptions and possibly re-model. But otherwise, I think you're doing well.

Great, that's nice to know. I think I'm finally ready to do some real trading. Thank you very much for your comments.

Thank you also Rhody Trader