Trading Strategy Research

[MoneyWeaver]

Hey,

I am trying to gauge interest in investment modeling/backtesting/optimization in order to possibly create a product that can solve them for my own use and possibly others.

I would like to ask anyone that is interested a few questions about what you perceive as the biggest challenge you face in carrying out backtesting/stress testing/optimization of your trading ideas. I know that that there is a multitude of software that does this but as far as I can see most of the software on the market is either deficient or expensive (or both). Most of the software also does not focus solely on the area of trading strategy research.

Researching your trading ideas and strategies correctly is at best a meticulous and exhausting exercise requiring you to:

- Select appropriate modeling/testing software
- Learning how to use it
- Purchase extensive market data then cleaning and verifying the historical price data (ie garbage in/garbage out)
- Coding your strategy in computer executable form (usually requiring mastering complex proprietary or a general purpose programming languages like c++, python , R, MATLAB, TradeStation, MetaTrader etc)
- Executing the test
- Analyzing the performance metrics

All this takes time and expense if you want to achieve meaningful results. I'm sure most of us are aware of all the pitfalls around testing/optimization and the problems with simulating trading strategy results but I would be interested in knowing:

[1]Do you test your trading ideas/strategies?
[2]What makes you want to punch your computer screen while researching and testing your ideas?
[3]Would you be prepared to pay for research software that did not cost a fortune and would not require you to learn anything new?
[4]Do you expect the software to be available as a download to use on your PC or as software as a service via your PC, tablet/mobile phone?
[5]What are the biggest pains with trading strategy development/testing/stress testing/optimization etc?

I am trying to gauge the level of interest in this area and would like any feedback on the above or any other comments you might have that might be helpful. If your not interested in the subject then please ignore.

Cheers.

 

[viktor_k67]

[1]Do you test your trading ideas/strategies?

yes

[2]What makes you want to punch your computer screen while researching and testing your ideas?

nothing

[3]Would you be prepared to pay for research software that did not cost a fortune and would not require you to learn anything new?

depends on the functionality and simplicity.

[4]Do you expect the software to be available as a download to use on your PC or as software as a service via your PC, tablet/mobile phone?

Online - I set the criteria you run the test and you show me the result. The simpler - the better.

[5]What are the biggest pains with trading strategy development/testing/stress testing/optimization etc?

Run couple of million tests to find the good setting for criteria in a strategy.

 

[MoneyWeaver]

Base functionality might be strategy development, testing, optimization and performance metric reporting. I guess a trading strategy would be setup by configuring various strategy simulation parameters (eg trade management, stoploss/limit order formulas etc), selecting appropriate studies & indicators, entry/exit rules, portfolio selection, walk-forward test period etc No coding would be required but if further flexibility is required (ie creating your own indicators) then a visual programming editor (ie using drag/drop modules) could be used again without the need for traders to code up a particular strategy/indicator (not all strategies could perhaps be tested this way but most simple to reasonably complex strategies could be tested).

If you want to offshore the hassle of executing and managing data/testing/optimization and strategy reporting then you could setup the strategy on your PC/tablet or mobile device and send it to the cloud to run on a virtual R&D engine. On completion you would be emailed a detailed report including diagnostics (to confirm that the strategy is producing trading signals that behave as intended). The analysis would use either the recommended data vendor database or you could supply your own. As the focus would be on R&D and a quick iniitial delivery then I would not want to divert efforts into the area of trade execution etc (ie supplying a full blown trading platform). However an export facility to allow the import of trading strategies into popular trading platforms might be looked at a later date. The focus would therefore be to quickly and easily generate/test new trading ideas as well as providing a quick and easy way to develop/test/optimize a trading strategy.

Cheers

 

[Solas0077]

I use Price Action Lab and this software combines strategy discovery with backtesting on single security and portfolios, robustness evaluations and various other tests. It generates code for Multicharts 4 too. It does not require any coding form my part and this solved a few of the problems you mentioned.

 

[MoneyWeaver]

The market is full of algorithm development, backtesting, optimization software solutions some better than others including no coding solutions. Anyone that has found something that works for them and is flexible enough to cover all their R&D requirements do not need to look any further. I am building my own stack because this gives me greater control and with a little extra work can offer this to others if there is sufficient interest. Additional functionality normally only found in institutional grade applications could also be added in the future in order to differentiate the software from other solutions. However, in order to get something up and running the aim would be to provide easy way to quickly develop/test trading strategies without manual coding but be flexible enough to answer questions like:

• Has a trading strategy been profitable in the past?
• How robust is a trading strategy?
• What are the risk metrics associated with a trading strategy?
• What effect does differing money management strategies have on a trading strategy?
• How does a trading strategy perform in different markets?
• How does a trading strategy perform in a trending or ranging markets?
• How does a trading strategy perform over differing timeframes?
• What happens if a stock falls 20% in one day do these stocks have a strong chance (> 90%) of bouncing the following day?
• Can you gain from non-standard MACD parameter settings…
• The best buy/sell signals from indicators such as MACD, Moving Averages…
• The impact of stop losses on risk metrics...
• What happens with trailing stops…
• What happened as average true range calculations adapt to market volatility…
• When should you use profit targets and how to set them realistically…
• Do momentum strategies work with currencies?
• Do momentum strategies work for weekly stock returns?
• Does momentum trading work on a daily timeframe?
• Is pair trading a viable, technical strategy?
• Does one price jump lead to another in the same direction?
• Is there a correlation between volume and price direction?
• Are moving average trading rules effective in emerging markets?
• Are exponential moving averages superior to simple moving averages?
• Do reversal strategies work?
• Which technical indicators are the most effective?
• Do moving average crossover rules work with volatile stocks?
• What is the time window to act upon new signals?

Any software that may be delivered will almost certainly not be the answer for everyone but at the moment I am only trying to gauge interest.

Cheers

 

[AppleInvestor]

It's not the software that's deficient, it's the way people use the software and what they try to do with it. For the most part, people try to overoptimism and make the strategy reflect what they want to see, rather than being truthful with themselves, and allow the strategy to tell them objectively just how crappy it really is.

No one wants to see their strategy look crappy, yet 98% of all strategies are crappy and not robust. If traders would simply recognize that fact, then we would be concentrating on the method of testing rather than the tools.

 

[MoneyWeaver]

I agree that some of the problems are probably down to poor backtesting practice and misunderstanding of what backtesting can and cannot do. But software that makes it dfficult to implement complex strategies, or does not provide support for portfolio level testing, money management, slippage, out of sample testing etc also adds to these problems. A combination of poor software which also allows poor backtesting practice will lead to less than usefull results.