Why automation is a must

[NetTecture]

Instead of increasing the number of strategies and markets, which is a brute-force approach, one may also investigate more sophisticated strategies that have the potential for higher return/drawdown ratios. This would include strategies based on trading a cointegrated portfolio of assets (stocks, ETFs, etc.), as opposed to trading a single asset within each trading strategy. Cointegrated portfolios of 3 or more assets have better statistical properties for forecasting price changes, and so can be implemented in an automated system using a trading algorithm. Chan describes some of these algorithms in detail in the book I mentioned earlier. You may still want to implement multiple strategies, but if you're drawdowns are lower, you don't need to run hundreds of them just to stay in the green.

More complex strategies are IMHO as idiotic as it gets. It has significant disadvantages:

* It is harder to code and debug complex strategies that follow likely multiple signals.
* It is harder to backtest them properly.
* It is harder to analyzer their behavior.
* It makes it nearly impossible to decouple signal based risk parameters. Not that most people even have a more than baseline risk management anyway (thanks to things such as software being grossly not suitable for managing many strategies with nontrivial risk allocation).

There is an elegance in having strategies doing one signal ONLY and then combining them to instance portfolios outside of the strategy. But then, which commercial software has risk allocation structures that are more complex than a max position size, or even interfaces suitable to actually manage a large number of strategies. Heck, I have yet to even see the commercial product giving me a matrix of weeks vs. strategy results - most do not even have a concept of tracking logical trades, instead relying on some magic signal matching trying to come up with the individual trade pnl.

I generally prefer to keep things simple on the strategy level. Strategies are the least tested (in terms of code quality) part of any trading infrastructure. And given that I and some others use our own software from the ground up - we actually DO have the ability to trade a large number and keep them under control. One of the modules I work on at the moment, for example, deals with complex risk allocation strategies (and redirecting strategy trades to a simulator if risk can not be granted because an exposure is already too high in a particular market).

I definitely prefer that from having "one strategy to rule them all" that is generally worse from a code quality point of view.

 

[TickCOM]

Good to start with a sophisticated approach, and then simplify by removing criterion by criterion from the strategy. Then use same strategy on multiple non-correlated currency pairs. If same performance across the pairs is a good sing of a robust strategy.

Time has run from building your own development platform, not sustainable any more with new area of big data, cross market awareness - platforms are becoming too complex.

 

[NetTecture]

Time has run from building your own development platform, not sustainable any more with new area of big data, cross market awareness - platforms are becoming too complex.

That is ignorant. Totally. As in: EXACTLY today is the time. Why? Because every single commercial platform out there totally fails to utilize any of the elements you have named. Big data? None of the retail environments even utilize a low end databare properly. Big data - besides the fact that I fail to see where big data enters, and no, some terabyte of data re not big data - none of the commercial software out there uses any big data approach. Platforms becoming too complex? Yes - by focusing on easy of casual use, ignoring modern software development practices and generally being not for automated trading.

Here is an example. NinjaTrader 8. No proper database use to store backtest results, no excel integration for backtest analysis. UI based trading as primary focus - so not able to run headless on a server, no ability to review trades and strategies outside of being logged into the session, instead of using a client/server approach for actual trading. It is a great software that for example I use - for charting and coming up with ideas. Very limited risk control for strategies. No command line interface - which is ok. No way to correlate cash trades with a backtest (to find out how good or bad the backtest assumptions translate into similar executions). No lightweight strategy execution environment. Heck, you can not even group strategies. We run strategies in differnet environments - some vs. simulator, some are being tried in cash (on a separate account with very small size), some run real, with good size - and mixing them up in the UI really does not help getting an idea how the cash trading goes. And no, looking at the broker account may not help either - because there you only see an aggregate of the different strategies, which - if you have multiple strategies trading one instrument - can not always sensibly be deducted back to strategy instances.

Same with backtesting. NT can not handle backtests properly. No database of results. No distributed computing. Want to use 5 machines for optimizations - have fun managing the workload manually. And then somehow exporting results to excel. My own little server room currently uses 20 computers for optimizations and I only log into them once per month to check their system state - work is distributed automatically. If I refactor a strategy, I make a reference backtest, refactor, run another backtest and then compare results - code refactoring should not change them by a single element. How do you do that in NinjaTrader?

So, yes, you are right. it is no the time. If you accept working with software that is obviously not using your computer properly. But if you want something modern, there is NOTHING on the market, at least nothing that is widely advertised. Been there, looked at it, found and still find nothing I want to use.

 

[TickCOM]

If you have spend many years building your own development environment, most probably you wont find anything else you would like to use

Anyway, no platform is perfect, the most important is time to market for a strategy - you will always have to adapt to the limitations of a development platform. For example, one are obsessed about using tick data, but using range bars holding also the spread information gives equal reproducible results - its just another kind of strategy you develop, equally performing.

What matters is time to market - this is king. And your time to market has been how long ... decade or two ?

 

[NetTecture]

Anyway, no platform is perfect, the most important is time to market for a strategy - you will always have to adapt to the limitations of a development platform. For example, one are obsessed about using tick data, but using range bars holding also the spread information gives equal reproducible results - its just another kind of strategy you develop, equally performing.

And you are aware (or ignorant) of any commercial product using range bars WITH SPREAD during backtesting? Because here your argument falls apart

Btw., I use tick data because of a reason that is very simple: I expose tick data to the strategy and I expect backtest and real trading to be 100% identical from a strategy point of view (i.e. no different code paths because the strategy may rely on tick data but can not do it in backtest).

And yes, bars + spread are good - but then they are not supported by any platform I know of. Which is not exactly making this an argument towards the use of a commercial platform.

 

[TickCOM]

Assuming there exist a platform which support the spread on each bar close - to backtest using tick data is only relevant if you exit a trade on a profit level, ie not at a bar close.

The problem to solve is; if you have a profit taking, then you need to call the strategy at each tick ! If you have 20 strategies running on 20 different instruments, mean your platform will be very very busy.

A solution is to add the spread to the bar close, and not use profit taking outside a bar close. But using 1 minutes bars is too risky since lots can happen within a one minute bar. Which mean we need to use range bars, but traditional range bars trigger a train of bars on a move, or price jumps - and they do not provide the important information of sideways trading.

So the solution is the TR bars timeframe - which is a combination of using minute bars, and ranges bars, (combined bars chart) with the adjustment that only one range bar is printed on a move, not to have a train of range bars.

The above imply we need the tick data to make the range bars, but we do not trade/backtest on every tick.

Attached a M1R40 chart, 1 minute combined with 40 tick range bars - which also happens to hold the spread information at each closed bar. Blue candles are range up bars, yellow candles are range down bars. Green and red 1 minute bars.

Using the TR chart, you catch both the fast moves and the sideways moves, provide about 30% more information, and do not spoil the technical indicators on sudden moves.

 

[TickCOM]

As to your comment on Big Data, I do not mention such unless it exist. This is a screen shot of big data in a platform - 4/8/16/32/64/128 GB of in memory data, without the use of database or file system. Which is what I ment, development platforms have become too complex for privates to start making their own - in fact at a level that strategies cannot be deployed any more, but are run from within the development environment. I think a platform like Deltix do not deploy any more.

On a 21:9 DELL

 

[NVP]

Good,discussions here guys ....v interesting

Tickcom I've replied to you today.......pm

N

 

[counter_violent]

The automation guys only need to remember one thing. Turn off all bots around major news, and events, like the big boys do, thus ensuring a liquidity vacuum and prices doing varying degrees of spiking and flash crashing. Otherwise, you could be the victim

 

[NVP]

The automation guys only need to remember one thing. Turn off all bots around major news, and events, like the big boys do, thus ensuring a liquidity vacuum and prices doing varying degrees of spiking and flash crashing. Otherwise, you could be the victim

Agreed ......and always have a nightwatchmen for,the systems involved......just in case that black swan goes off on its head.....

 

[Solas0077]

Automation is dangerous during flash crashes and other extreme events.

 

[new_trader]

Automation is dangerous during flash crashes and other extreme events.

Automation is dangerous. That's all that needs to be said.

 

[IFeelFree]

More complex strategies are IMHO as idiotic as it gets. It has significant disadvantages:

* It is harder to code and debug complex strategies that follow likely multiple signals.
* It is harder to backtest them properly.
* It is harder to analyzer their behavior.
* It makes it nearly impossible to decouple signal based risk parameters. Not that most people even have a more than baseline risk management anyway (thanks to things such as software being grossly not suitable for managing many strategies with nontrivial risk allocation)...

So, if something is hard, we shouldn't do it?

One should only take on a coding task that one has the ability to accomplish. You code it, debug it, and backtest it very carefully and thoroughly. Also, for anything more than the most basic algorithmic trading, I would not use commercial software. I code it myself.

One way to get an edge in this market is to exploit any unusual skills that you may have. In my case, I have a graduate level math background and decades of programming experience. So, I'm a natural fit for algorithmic trading. Some of the strategies I'm using are complex, but I feel I understand the math very well and I'm able implement them successfully. "Complex" doesn't necessarily mean more and more of the same signals. It can mean a deeper understanding of the mathematics of time series analysis (autoregression, cointegration, Kalman filters, etc.), as applied to financial data analysis and forecasting. That's my approach.

 

[Pat494]

The sudden fall in the £ has probably unstuck some automated strategies imho.
It gave the system I am currently testing a nasty shock initially but it has partially recovered now.

 

[Pat494]

I have a few ideas to make a system. OK so I am hopeless at coding but I have found a website that will they say help. Here is it's address

mqlprogrammingservice

For $150 they say they will code up my ideas within 2 - 5 working days. It went in today so I am now sitting back with fingers crossed etc. I will let you know what happens. I don't even know if it will make a profit.

 

[NetTecture]

I have a few ideas to make a system. OK so I am hopeless at coding but I have found a website that will they say help. Here is it's address

mqlprogrammingservice

For $150 they say they will code up my ideas within 2 - 5 working days. It went in today so I am now sitting back with fingers crossed etc. I will let you know what happens. I don't even know if it will make a profit.

THat so much sounds like spam it is not even funny. All the best - you really got top of the line people working for you. I mean, why would anyone not work for 500 pound plus a day as freelancer when he can develop a full strategy for someone else for a pittance?

 

[Pat494]

THat so much sounds like spam it is not even funny. All the best - you really got top of the line people working for you. I mean, why would anyone not work for 500 pound plus a day as freelancer when he can develop a full strategy for someone else for a pittance?

If and I know it's a big if it is profitable I will have the last laugh at your post.
Anyone else with a big idea, I can only suggest you give it a try. Even if it fails.

 

[NVP]

semi automatic works..........but never let the robots loose on their own .....

 

[NVP]

So, if something is hard, we shouldn't do it?

One should only take on a coding task that one has the ability to accomplish. You code it, debug it, and backtest it very carefully and thoroughly. Also, for anything more than the most basic algorithmic trading, I would not use commercial software. I code it myself.

One way to get an edge in this market is to exploit any unusual skills that you may have. In my case, I have a graduate level math background and decades of programming experience. So, I'm a natural fit for algorithmic trading. Some of the strategies I'm using are complex, but I feel I understand the math very well and I'm able implement them successfully. "Complex" doesn't necessarily mean more and more of the same signals. It can mean a deeper understanding of the mathematics of time series analysis (autoregression, cointegration, Kalman filters, etc.), as applied to financial data analysis and forecasting. That's my approach.

I used to be able to balance a spoon on my nose ..........damn hard to program that into a forex system though

(sorry couldn't resist )

N

 

[Pat494]

So, if something is hard, we shouldn't do it?

One should only take on a coding task that one has the ability to accomplish. You code it, debug it, and backtest it very carefully and thoroughly. Also, for anything more than the most basic algorithmic trading, I would not use commercial software. I code it myself.

One way to get an edge in this market is to exploit any unusual skills that you may have. In my case, I have a graduate level math background and decades of programming experience. So, I'm a natural fit for algorithmic trading. Some of the strategies I'm using are complex, but I feel I understand the math very well and I'm able implement them successfully. "Complex" doesn't necessarily mean more and more of the same signals. It can mean a deeper understanding of the mathematics of time series analysis (autoregression, cointegration, Kalman filters, etc.), as applied to financial data analysis and forecasting. That's my approach.

You may be willing to sell one of your more profitable systems ?