Statistical analysis, Backtests & automated trading : Which way to go?

[JossBeaumont]

Hi everybody,

I am learning futures trading and I have spent the last year reading, learning, paper trading, trying to set up a sound trading system and I will soon start to trade for real.
Till now I've been focusing mainly on tape reading, footprints, TA, Market and Volume Profile and other discretionnary techniques, but the more I learn the more I feel the need to be able to:

1) Conduct my own statistical analysis.
2) Perform extensive backtests by programming my trading ideas.
3) Possibly set up some basic automated trading algorithms.

I don't know any programming language yet, and to choose one is an important decision as I'll invest a fair amount of work and of course I don't want to waste time learning a programming language and/or new software(s) and after that changing to another one, etc...

So what do you think will be the more suitable for my needs?

I am currently using Sierra chart but I am considering moving to another platform (perhaps Tradestation or Market Delta?).
I am already familiar with Excel but I doubt if it is powerfull enough to conduct statistical intraday analysis and/or backtests on large samples.
I have also heard about Matlab... But is it really necessary to use a third party program for statistical analysis and backtests or should I use my trading platform directly?
What about programming language C++? Easylanguage? Another one? Some much options are available I don't know what direction I should take...

Thanks in advance for your help!

Jonathan

 

[Crassius]

I feel the need to be able to:

1) Conduct my own statistical analysis.
2) Perform extensive backtests by programming my trading ideas.
3) Possibly set up some basic automated trading algorithms.

I don't know any programming language yet, and to choose one is an important decision

So what do you think will be the more suitable for my needs?

Jonathan

We all must find our own path. If your path will be anything like mine, you will need to learn to program.

I could not have progressed very far with out taking up programming to work out ideas, build custom indicators and rules based systems that can be tested, measured, learned from, improved upon etc. Its just the way my brain works. One doesn't learn how to trade, program the method into a system and then sit back and trade that for the rest of your life. Its a never ending process for two reasons.

First, the market is always changing. For example, FX volatility is at a 5 year low as I write this. It has been declining for the last few months. I had to tweak my Volatility Breakout system to compensate for the changing conditions. If I couldn't do this, I would have systems that only make money in favorable conditions, but give it all back when conditions inevitably change.

Second, as I trade anything that is at least breakeven, the market will reveal a little ankle. Just a flash of skin that reveals an insight I would not have known had I not approached it in a systematic, measurable, repeatable way. Such insights lead to new indicators to drive new systems, or tweaks to existing ones.. Rinse and Repeat.

I trade multiple systems that diversify my risk by targeting different aspects of the same market. I have more ideas in notebooks than I have time to program.... and I am always learning more.

So what language shoudl you learn? Which also may mean, what platform?

Forget excel. Fine for some things but we have better tools available for this task.

TradeStation's code is very english like... In his many works, John Ehlers explains his ideas in math, which I can't understand, but then he also always gives the TradeStation code that I can easily follow step by step and see what his math is doing.

That's one plus for EasyLanguage. It easy to read.

However, I am going to suggest you take a serious look at NinjaTrader, which uses the very widely used and exquisitely documented C# programming language and the DotNet Framework. It is datafeed and broker agnostic which means you are not required to use a specific data provider like eSignal, and you don't have to trade through a specific broker to have access to the platform. You can use it free as long as you like for charting, backtesting and system development or learning how to do all the above with EOD data or in the market replay simulated environment.

NinjaTrader stock, futures and forex charting software and online trading platform.

If you want to actually place trades through NinjaTrader you need to pay.... 250% less than I paid for TradeStation way back in 1995.... $1000 for a lifetime license, or you can lease for $50 bucks a month (which is waste in the long run). No I don't work for them, just use their stuff, which isn't perfect, but its pretty damn good.

I would suggest you download for free, work through the tutorials NinjaTrader provides and get a couple good books on teaching yourself C#. There are tons of them. Read the reviews on Amazon and get more than one...

While there are many books on teaching yourself C#, there is even more learning and tutorial stuff for free on the web. When you want to so something you just Google it and an answer will pop up usually with a tutorial or two. Microsoft has the language very well documented with examples, and they even have self teaching material online... so do many many other people on the web, like this guy that has saved me countless time with his concise tutorials.

Dot Net Perls

If you work in C#, and not some very narrowly used platform specific script, you will always be able to find help...... and this can not be under valued by a new programmer.

If you have no background at all in programming, a book on programming logic and design can be a good investment. I have a couple by this author. They are not cheap but invaluable to someone like me that somehow stitches together code that works having taught himself.

Amazon.com: Joyce Farrell: Books, Biography, Blog, Audiobooks, Kindle

Another great thing about working in one of the worlds most popular languages and not some platform specific scripting language besides the wealth of documentation, are the vast libraries of reusable code that make up the DotNet Framework.

The DotNet part of all this is a collection of libraries of re-useable code so you don't have to build everything from scratch. Anything that you can dream up.... statistics, neural networks, vector math... whatever.... its already been done, and you can use the preexisting code in the libraries by making changes and additions to get exactly what you want. C# isn't the greatest for math... but it can and is used. It is also easy to transition to C++ from C# which you may never need to do.

 

[A Dashing Blade]

Advising someone with no programming knowledge to start with C# is . . . . rediculous.
Most advanced language I'd recommend would be Java (most Trading apps have a Java interface), easiest I'd recommend would be excel vba.

 

[Crassius]

Advising someone with no programming knowledge to start with C# is . . . . rediculous.

Most advanced language I'd recommend would be Java (most Trading apps have a Java interface), easiest I'd recommend would be excel vba.

The only thing ridiculous is your answer. Java is somehow much easier than C#?

C# is a straight forward language not at all difficult to learn and today is often one's first language.

 

[JossBeaumont]

Hi guys,

@ Crassius:

First of all thanks for your very detailed answer.

I could not have progressed very far with out taking up programming to work out ideas, build custom indicators and rules based systems that can be tested, measured, learned from, improved upon etc. Its just the way my brain works.

Mine too!

One doesn't learn how to trade, program the method into a system and then sit back and trade that for the rest of your life.

Forget about that. Never crossed my mind. I know it's hard work.

So now, I have narrowed my choices to

Tradestation, NinjaTrader or Sierra Chart

Those last days I've been more and more inclined to go with Tradestation: great platform, Market profile, a lot of resources. Of course I know the disadvantages of a prop language, but maybe I can deal with it.

At first glance Ninjatrader hasn't got any of the features I need (Market and Volume profile and Footprints). But still, after your advice I will download it and try it for a week or so, maybe I'll change my mind...

The DotNet part of all this is a collection of libraries of re-useable code so you don't have to build everything from scratch. Anything that you can dream up.... statistics, neural networks, vector math... whatever.... its already been done, and you can use the preexisting code in the libraries by making changes and additions to get exactly what you want.

I really like that! That's the most crucial point for me.
But isn't it the same with Easy Language? And C++ (Sierra Chart)?
I heard that, especially for Easy Language there is a very active community, a lot of resources (I just can't wait to read the books you mentionned), etc...

I already use Sierra Chart for a year as it has all the features I need so If I decide not to go with Tradestation, I'll consider Sierra Chart first.
But what you said about re-useable code, statistics, neural networks, vector math... Is this also valid for C++ or is this only for DotNet?


One last thing,
Everybody's talking about backesting and automated system but few about the statistical analysis that is necessary to set up a system before putting it to a backtest (or more likely, I don't know how to search properly for that on Google...).
Which software do people use for that? Do you use directly your trading platform? Is it suitable for this purpose?

Cheers


@ Dashing Blade

I have considered Java at the beginning but really I didn't find any platform that I like with it. Moreover, although the simplicity of the language is a plus, it's not my main criteria. I want more something that I can go with for the rest of my life even if I have to work harder at the beginning provided I don't have to change afterward.
About Excel: Is it possible to conduct extensive Intraday analysis with it? The maximum lines is 65000 or something like that... It seems very far from what is needed to make a statistical analysis on several years...

 

[Crassius]

First, TradeStation is a fine platform... I bought a license for it way back in 1995 before they changed the licensing to leases. You will have a fine platform should you go that route. It has been around a long time, and there is a big ecosystem that has built up over that many years.

NinjaTrader comes with a lot of built in functionality, but just like TradeStation, it is extended by third party vendors who build on the NinjaTrader foundation, as well as users who share their extensions in the NinjaTrader forum. So both platforms have what you are calling libraries.... I call it an ecosystem.

You can't judge what one or the other has just by looking at what is built in... you have to look at what is available in the ecosystem. I don't know if Market Profile is available from third party vendor or a clever user who shares it on either the Ninja Forum or BigMikes forum. Just because it isn't built in doesn't mean it is not available, or couldn't be built by you or others as long as the logic is not proprietary...

The libraries I am talking about are not trading specific. They support all manner of programming and make it possible to do anything one can do on a computer. Many are part of the DotNet framework. Many of them are free and open source, some you have to pay for. With them, you can accomplish anything you can dream up that is possible on a computer. You can't do that in EasyLanguage.

By example here is one I'd like to buy when I hit the lottery.... there are other free, not so slick similar libraries... This is a Ferrari.

Statistics Library Features - Features - .NET, C# (CSharp), VB.NET, F# Math, Statistics and Matrix Libraries

Both platforms take care of a lot of trading functionality for you out of the box. They connect to data feeds, draw charts, place trades and have built in indicators etc etc. The user then extends the platform with additional functionality..... maybe with an indicator, a complete system, or whatever they can dream up... including data manipulation, statistical analysis or pre-processing as a first step whose result gets used later.

TradeStation has a scripting language called EasyLanguage, that has it own syntax and pre-built functions. You can find the value of a moving average by making a function call. You can accomplish a lot with EasyLanguage, but it is specifically built and narrowly focused on trading tasks like finding the value of a moving average.

NinjaTrader has exactly the same setup.... It has a pre-built function to get the value of a moving average, for example, so you don't have to write tht logic from scratch. But, NinjaTrader uses C# and DotNet to write the functions.

That's a big difference....

If you want to do something beyond everyday trading logic such as a moving average, in Trade Station you would have to write that code in C#, C++ etc. on another (programming) platform, compile the working code into a dll, and then call the dll from inside EasyLanguage.

Since Ninja uses C# in all the places TradeStation uses its own EasyLanguage, you don't have to do some tasks in another language on another (programming) platform then call that code.

Your best bet is to have an open mind and try each platform.... work with them for a while and see what works best for. Like me, you may change from one to the other someday.

Ninja has the unlimited trial period so you can really try it before you have to invest in it.

I am not up on TradeStation's current offers, so maybe you know of a better deal.... but one deal I do know about is....you can open an FX account with IBFX, deposit $1000 and they will give you access to TradeStation with no lease payment or minimum trade quota. That seems to mean you never have to actually trade the $1000... just make the deposit and play with the platform for as long as you like.

One other drawback to TradeStationis it is NOT broker agnostic.... you must trade through TradeStation Securities. If you should invest a lot of time and effort into building things for Trade Station, and later decide to change brokers... you would have to change platforms. This is an old model, and newer software like NinjaTrader are broker agnostic.

Hope this is helpful to you and others who come upon the thread in the future.

 

[natureboy]

I say MultiCharts. They now have both C# .net version and an EasyLanguage version (they call their language PowerLanguage). Backtesting in some ways is better in MultiCharts - they offer more statistics.

For statistical research though - I'm beginning to use R-based platforms. Doing 2-4 variable searches to find the repeating statistical patterns. Right now i'm looking to use SciLab. MATLAB is attractive as there are already the data feed plugins and statistical tools... its expensive though. 10k USD to set up everything in MATLAB that I would want.

 

[A Dashing Blade]

C# . . . is often one's first language.

Any danger of some facts to back up this assertion?

 

[A Dashing Blade]

. . . @ Dashing Blade

I have considered Java at the beginning but really I didn't find any platform that I like with it. Moreover, although the simplicity of the language is a plus, it's not my main criteria. I want more something that I can go with for the rest of my life even if I have to work harder at the beginning provided I don't have to change afterward.
About Excel: Is it possible to conduct extensive Intraday analysis with it? The maximum lines is 65000 or something like that... It seems very far from what is needed to make a statistical analysis on several years...

1) Programming is programming. Know how to code in one language and (by and large) the concepts are easy to transfer to another language. It's more important imo to learn how to construct an algorithm

2) Excel and intra-day? Certainly if you use VBA, I do it myself (both professionally and at home) with Bund and Stoxx Index tick data.

 

[Crassius]

Any danger of some facts to back up this assertion?

When you don't don't know what you are talking about.... illustrated by your assertion the Java less advanced than C# and is your suggestion for a first language.... You not only make a fool of yourself, but you confuse the many people that will come onto this thread in the future.

 

[Anupriya]

Hello Jonathan,

Please see the response to your queries below:

1) Conduct my own statistical analysis.
Ans. Tradestation is a good platform to start on, but as pointed out by Crassius above, it would be better if you start with a platform where you can code in C++ or C

2) Perform extensive backtests by programming my trading ideas.
Ans. You can do basic statistical analysis in your trading platform itself (true for most of the platforms), but in case you want more rigor in your analysis, you might want to use R or R-Excel. Its good enough for most of the trading oriented statistical analysis and is open source! At this link http://www.r-tutor.com/ you will find tutorials for learning R along with data sets which you can use for practice.

3) Possibly set up some basic automated trading algorithms.
Ans. Are you considering HFT oriented strategies or just automating your low frequency trading strategies? The choice of the platform (as well as trading infrastructure) would heavily depend on that.

In case you are looking for a comprehensive training program on Algorithmic and Quantitative Trading, you might want to check QuantInsti's Executive Program in Algorithmic Trading. It is fairly comprehensive and practically inclined program for working professionals.

Best Regards,
Anupriya Gupta

 

[divisor]

I had a similar question. There is a yahoo lisp interfaces library here
https://github.com/pnathan/cl-yahoo-finance

Using this one can set up very quickly a simple piece of paper trading software. I would recommend you do this first. It's in lisp, but if you don't want to learn lisp you can look at the source code and rewrite the parts you want in whatever you want. Or write a wrapper around it.

As far as interfacing with your trading platform, I don't know anything about that, but the good news is that you can always write a wrapper around it and continue writing in a language you are proficient in.

C#/java/C++ are all industry standard, well-supported, and full-featured. You would do well to start there.

 

[BigPointValue]

You've received some excellent advice from both Crassius and Natureboy. I hope these additional observations help.

I started off in the same way as Crassius. I purchased TS2000i in 1998. It was a labour of love with data management being a huge and time wasting issue. Nowadays Tradestation is slicker and more corporate but with this evolution it has become less community friendly and very expensive, many have turned away from it as a result. But it is the individual trader standard (it's now miles ahead of Metastock it's traditional rival). AIQ which used to be an option is now a joke.

The major benefit of Tradestation as I see it is "EasyLanguage." My evaluation of it is slightly different from Crassius. In my experience you can go way beyond programming moving averages. You can also program good successful systems with it. I've programmed dozens of systems & several hundred indicators and functions using this language alone. The main limitations of the language are:
1. Few programmers outside the TS community know of it or use it (therefore specialist programming and libraries are limited to the TS community).
2. Tradestation occasionally change the language and this means you have to rework your code. If you have used code from a magazine e.g. Stocks and Commodities Magazine in the 90's or early 00's and the code is not adequately commented you could be in trouble. Using such code is a key learning tool. (Regarding the Ehlers code it is actually wrong in his book! so things like this can be frustrating - I got it from his website and even then had to rework it!)
3. Most importantly, complicated code (e.g. non linear maths) is not really amenable to Easylanguage: you will then need to use a TS specialist or expert math/stat programmer or both which means;
a. sharing all or part of your proprietary strategy with someone who is clued up about trading or
b. using Tradestations SDK which Tradestation do not properly support and which is highly abstruse and difficult to integrate. Last year I went through 3 top programmers (on oDesk and vWorker) before finding one who could link a specialised dll to tradestation and get it to work. A TS specialist could not get the DLL to work and charged me $250 for just trying!
The main disadvantage of the Tradestation program is.
1. Cost.
2. Time to learn (though there are numerous very useful videos and tutorials and you can attend seminars)

Other key advantages of Tradestation are:
1. Many books and magazines contain code written in some version of easylanguage
2. The learning materials for easylanguage are good
3. The language is specifically written for traders
4. As a non-programmer you will learn to program in a manner focussed to your needs.
5. You will find many programming examples on the TS forum (but also many broken, un-maintained links will frustrate you!)

So, to wrap up on Tradestation: It's a fantastic program, the helpful culture has changed and you are more on your own these days. It can be a great place to start.

Now to the recommendations of Crassius and Natureboy:

Both are correct in my opinion. Here are some additional thoughts:
MultiCharts admit that they are only EasyLanguage Compatible. BUT they are broker and feed independent unlike Tradestation (which is great)
NinjaTrader uses C# which has a steeper learning curve to Easylanguage and you don't get the help of all those books and articles when learning the code.

Overall: As a final solution I favour NinjaTrader but you must realise the path is a relatively long one. It will take you another couple of years to become proficient.

As an initial solution I suggest tradestation for your first year whilst you learn C# and brush up on your maths/algebra and statistics if you need to . You will also need to learn about Trading system development. Tushar Chande is a good and entertaining place to start, he is a generous writer (though I no longer use his systems the thought process is of the essence).

As to your statistics question: You don't need to do your stat's first, you do them at the end. The essential element is to decide which metrics you want. If these are provided by your program then great but I'd suggest programming them yourself. Thomas Stridsman used to be a great place to start but the Easylanguage code and the Excel code he uses has changed. The principles of system testing however remain the same.

1. Look for a stable system (no huge winners or losers).
2. Look for percentage winners/losers that you are psychologically comfortable with (especially if you intend to trade trends where you will have lots of small losers).
3. Optimise DON'T curve fit.
4. Once you have a basic good system improve it by looking for good Entry and Exit efficiency

BPV

 

[nelsonhimes]

It depends on the investment owner's mind whether the purpose is for lending the resource to someone else for economic purpose or not..

 

[dfenn]

Anyone on this thread had any experience with Amibroker for futures backtesting. I am in a similar situation to the guy who started this thread. I have some Easy Language experience, but Tradestation kicked me out due to an error one of their employees made and they didn't like my complaining!

 

[avano]

Anyone on this thread had any experience with Amibroker for futures backtesting. I am in a similar situation to the guy who started this thread. I have some Easy Language experience, but Tradestation kicked me out due to an error one of their employees made and they didn't like my complaining!

Amibroker is just fine and it has some nice features like WFO and portfolio backtesting. It is also cheaply priced for what it does but a little more difficult to learn than TS. I recently switched to full use of Amibroker because of the easy with which I can test the patterns I get from price action lab and the systems I develop. In the beginning you must be careful with the parameter settings for backtests because they are tricky. As far as live trading I plan to use the IB plug-in.

 

[dfenn]

Amibroker is just fine and it has some nice features like WFO and portfolio backtesting. It is also cheaply priced for what it does but a little more difficult to learn than TS. I recently switched to full use of Amibroker because of the easy with which I can test the patterns I get from price action lab and the systems I develop. In the beginning you must be careful with the parameter settings for backtests because they are tricky. As far as live trading I plan to use the IB plug-in.

Thanks!! I am considering it.

 

[numbertea]

I have settled on LightSpeed. Their API is native Cpp (MFC) and it works great for me so far. I am happier with Cpp over other languages because I don't have to make modules in a different languages. Cpp is very fast math and statistical decisions on the fly are part of my trading platform so I need that speed. Tick data on LightSpeed is great between 6 am and 5:05 pm, these are the hours I run my system.

Cheers