Backtesting Software

[blackcab]

I've written a short paper concerning to this topic on my website:
http://www.zentrader.de/dataSimulation_e.pdf

Monte Carlo's great. You can do different types of out-of-sample testing than you mention in the PDF though that avoid the problems you mention - you can 'stripe' the in-sample and out-of-sample data based on the timeframe you're trading, e.g. if trading intraday, develop on alternate days and test out-of-sample on the other, alternate dates. That avoids the problem of changing market conditions or seasonality etc.

 

[Chorlton]

Chorlton,

I've written a short paper concerning to this topic on my website:
http://www.zentrader.de/dataSimulation_e.pdf

Only my two cents...

bye,
zentrader

Zentrader,

Nice paper!! Thanks for posting.........

 

[Ofir]

Yes, it can take money management, sizing, scaling in/out into account as necessary. Because it's open ended, and works by simulating a trader bar by bar as opposed to applying an equation to whole datasets, it can simulate arbitrary strategies, e.g. reserving Tuesday mornings for long-only trades at reduced exposure except where Monday's high was touched more than once and the previous week was low volatility, and the clocks went back within the last 10 days, and volume/range/close profile of the last 20 bars was x or y, etc., etc. You get the idea.

I'm looking at a futures contract not stocks so I'm not taking portfolios into account, but that would clearly be an issue if you were. If you're going technical and wanting to backtest mechanical systems, I'd tend towards a more powerful and flexible system that gives you the option to explore in depth, even if what you end up using is a simple, intuitive KISS-type system. At least you can test it properly. I'm not sure what the main options are there - Tradestation, Easy Language, MetaStock, maybe?

As a developer with more than 10 years of experience I would advise to use existing tools instead of developing your own. Try wealth-lab which gives you a development platform based on pascal. You can use this platform to write anything you need.
You should ask yourself what is the purpose of your development. Is it to write a trading system or reinventing the wheel.

 

[_coda]

As a developer with more than 10 years of experience I would advise to use existing tools instead of developing your own.

I am in two minds about this. Many here are techies, and I suspect enjoy building something that can actually be put to productive use. I am tinkering with a forex backtesting framework and am enjoying it immensely. At the same time I am looking at commercial backtesting systems.

1. Maybe there really is a silver bullet out there. If so, great! I'll take two.
2. They probably have some clever ideas that I can use.

Try wealth-lab which gives you a development platform based on pascal. You can use this platform to write anything you need.

Certainly looks interesting

From the wealth-lab FAQ:
Does Wealth-Lab support FOREX?
To the extent that FX markets trade like futures, it's possible to perform FOREX simulations in Wealth-Lab, however, no special support exists for FOREX.

Have to spend a bit of time looking at this

You should ask yourself what is the purpose of your development. Is it to write a trading system or reinventing the wheel.

The robustness of a home brew solution isn't (and doesn't have to be) anywhere near that of a commercial system. Using thumbsuck metrics I estimate that building a system for own use probably takes less than 5 - 10% of the effort of a building a commercial quality system. Assuming you aren't building a swiss army trading system but just a simple backtesting system without the eyecandy.

I have to admit that my main reason for doing my program is because it's fun. With luck it will lessen the learning curve as well.

 

[journeyforlife93]

OK, so what is the conclusion for creating a backtesting software that backtest each of own ideas? Anyone knows how to create the backtesting software step-by-step? Assuming that i am a computer-illeterate, where should i begin?

 

[LongVol]

To backtest relatively simple strategies (based on various technical indicators for example) the usual suspects would probably be a good choice (Ninjatrader, MetaTrader, etc). However if your strategy gets slightly more complex (pairs trading, futures-based strategies that need to include an intelligent roll-mechanism, etc) then these trading softwares will have trouble handling this.
While we started initially started out with these same platforms, we quickly migrated to the open-source version of AlgoTrader which allows a much easier handling of the mentioned situations while also allowing for less "off-the-shelf" strategies. Today we are using the commercial version of AlgoTrader (for backtesting as well as live-trading), and are fully satisfied with it.

On a side-note: don't get the "more complex strategies" wrong I fully agree with Alex, simple ideas are the most robust and thus often the best.

 

[AlgoBench]

@journeyforlife93 I think it very much depends on the complexity of your strategy. Whilst I certainly agree with keeping it simple, some simple strategies are built on fairly complex foundations (i.e. the underlying indicator). I think it is good to know the exact details of what a trading decision is based on. For less common ideas this may well mean coding it yourself, or being in an environment where you have considerable control at a low level.

______________________
www.AlgoBench.com

OK, so what is the conclusion for creating a backtesting software that backtest each of own ideas? Anyone knows how to create the backtesting software step-by-step? Assuming that i am a computer-illeterate, where should i begin?

 

[Solas0077]

any backtesting software with monte carlo analysis built-in?

 

[AlgoBench]

Yes, ALGOBENCH.COM uses a number of approaches to backtesting.

any backtesting software with monte carlo analysis built-in?

 

[ethanslad]

thanks guys, i will try it!!!

 

[numbertea]

Many backtesting programs I've tried usually give me much better results than I receive while actually trading. This problem inspired me to develope my own backtest, forward test system, into my automated trading program. One thing that definitely stands out to me is that one should always give one's self the worst possible spread while backtesting to keep the actual gains/ losses more realistic as to what really would have happened if you were trading. Forward testing validates this finding.

Cheers

 

[AlgoBench]

@numbertea.

Agreed.

At ALGOBENCH.COM we like to understand actual spreads quite carefully, as they also change throughout the day. Being as conservative, with your assumptions, as possible weeds out a lot of false strategies.

Many backtesting programs I've tried usually give me much better results than I receive while actually trading. This problem inspired me to develope my own backtest, forward test system, into my automated trading program. One thing that definitely stands out to me is that one should always give one's self the worst possible spread while backtesting to keep the actual gains/ losses more realistic as to what really would have happened if you were trading. Forward testing validates this finding.

Cheers

 

[Solas0077]

Yes, ALGOBENCH.COM uses a number of approaches to backtesting.

Thanks, I appreciate it.

 

[Solas0077]

Many backtesting programs I've tried usually give me much better results than I receive while actually trading. This problem inspired me to develope my own backtest, forward test system, into my automated trading program. One thing that definitely stands out to me is that one should always give one's self the worst possible spread while backtesting to keep the actual gains/ losses more realistic as to what really would have happened if you were trading. Forward testing validates this finding.

Cheers

In my experience the reason you get better backtesting results than in actual trading is not so much because of the backtesting platform although they all make assumptions but due to changing market conditions. You should never expect to get the same results in actual trading because when you backtest you have no competition to challenge your trades. I just purchased a license of price action lab to use to find patterns with high success rate and the backtest results I get in multicharts match withing 1% the results of this program but unless the patterns are chosen after some careful analysis they can fail in actual trading. I think monte carlo analysis is a good way to go about but one should not only rely on that. Forward testing is important as you said at least for a good number of trades depending on frequency.

 

[bbscon]

I highly recommend Confidant (iPad app) if you want the power of true custom backtesting with no programming. I find it easy to use and strong enough to create all sorts of combinations of conditions and triggers for my strategies.

 

[malaguti]

I highly recommend Confidant (iPad app) if you want the power of true custom backtesting with no programming. I find it easy to use and strong enough to create all sorts of combinations of conditions and triggers for my strategies.

thats only because you created it!