Best Thread Algorithmic trading experiences

Hi,

I'm new to these boards and also new to trading. I've found lots of the information posted on the stickies really useful to compliment what I've read on technical analysis. I'm genuinely pleased to have stumbled upon this place.

Basically I came to the conclusion some time ago that people's behaviour in making trading decisions is the often the root cause of them losing money. So I thought I would build some software that took 'me' out of the equation and code up algorithms based purely on technical analysis. The system is currently working on UK stocks. Over the next 12 months I'm going to trade on paper and refine the algorithms until I find some strategies that are generally consistent. Once I'm confident with it, then I'll put it to work for real. Kind of my pension really.

The current strategies are all based around trends that occur over 2-3 days periods. This is putting me on paper in and out of the market quite frequently. I don't mind this if the 'little and often' culminates in a positive return but I get a sense that more lucrative strategies are longer term and are in the order of months.

Is this why people tend to enter positions with a view to a longer timeframe rather than day trade? I'd be interested in the reasons why people day-trade vs taking longer positions. I don't currently understand why people would choose one over another I suppose.

Also I tried to search for anything on algo trading on these forums and nothing popped up. Is what I'm doing quite marginal or is it just down to the fact that most experienced traders aren't software developers?

cheers

robster

e2a - typo in thread title - good intro from me eh................

robster970 said:

I'd be interested in the reasons why people day-trade vs taking longer positions. I don't currently understand why people would choose one over another I suppose.

Also I tried to search for anything on algo trading on these forums and nothing popped up. Is what I'm doing quite marginal or is it just down to the fact that most experienced traders aren't software developers?

cheers

robster

Click to expand...

I can only answer from a personal perspective, and most people here are probably going to disagree with me I'd say the choice of time frame traded is completely down to personal psychology, and that people tend to gravitate towards something they're comfortable with (or they fail).

There might be perfectly good logical reasons for trading longer term, but if it doesn't suit your personality you are going to struggle (even if trading an automated solution).

There's not much stuff on algorithmic trading at T2W (or indeed trading in general ), although its worth taking a look at the technical trading forum, there's a couple of good contributors there developing with tradestation (Charlton immediately springs to mind), probably the best places for information are the forums provided by charting application providers, I got a few ideas from the trade station and wealthlab forums, although its mostly noise to be honest, people faffing about with indicators and stuff

Just out of interest why UK stocks ?

GammaJammer said:

And also, searching T2W is unlikely to yield much on Algorithmic Trading as it's a little beyond the scope of most people here. Unfortunately it's not a search that yields much of genuine value on the web at large, as anyone with something decent to write about is more likely to keep it secret at the moment. So all info you're likely to get for free is probably going to be pretty vague.

That said, it is the future, and I for one am always happy to get involved in any promising threads on the subject.

GJ

Click to expand...

Thanks GammaJammer - I have indeed found that there is pratically nothing out there on the www about algo trading. I concluded early on that people aren't likely to reveal their edge if they have one. I don't think it needs to be really sophisticated either, just consistent. My current strategies are dropping the occassional clanger though (stuff like hitting a stop, selling then in the next analysis cycle re-buying again because it meets the basic entry conditions )

@Zupcon - thanks for the hint - I'll take a look at that this afternoon.

UK stocks? I'm British so picked a market that I'm familiar with and is also suffering from volatility - it's a good test of the algorithms - if they can handle bearish conditions then I would have though bullish would follow - or is this completely naive on my part???

My own two bits to toss in here is to make sure you spend a lot of time developing a real understanding of the risk side of things. A lot of very high profile algo traders (hedge funds, etc.) got themselves blown up because they failed in that category. Not that you're likely to trade in their fashion, but the principle stands.

Aside from that, I'll echo zupcon on the timeframes. Trade what makes sense for you. GJ is right about most retail traders being in short timeframes because of undercapitalization - and a simple lack of understanding that a large point/pip risk doesn't necessarily mean a large account % risk ("I could never risk 350 pips!").

robster970 said:

UK stocks? I'm British so picked a market that I'm familiar with and is also suffering from volatility - it's a good test of the algorithms - if they can handle bearish conditions then I would have though bullish would follow - or is this completely naive on my part???

Click to expand...

Actually, it might be. Bear markets tend to be more volatile than bull ones, especially in stocks. Whether that means you would expect better bull market performance from your system depends on whether it thrives on volatility (as some do) or whether it works better in calmer markets.

Rhody Trader said:

My own two bits to toss in here is to make sure you spend a lot of time developing a real understanding of the risk side of things.

Click to expand...

Any good stuff to read here John - just to get me going on the subject?

Can you explain a bit more about undercapitalisation? Sorry if this is really basic stuff but I am interested in running two sets of algorithms - one aimed at cycles < 5 days, another set > 3months and hence not understanding undercapitalisation sounds like it might be a problem for me to do this.

BTW - one of algorithms recognises noise and uses weighted acceleration of ROC in conjunction with moving averages - in that sense in calmer bull markets the noise disappears

robster970 said:

Any good stuff to read here John - just to get me going on the subject?

Can you explain a bit more about undercapitalisation? Sorry if this is really basic stuff but I am interested in running two sets of algorithms - one aimed at cycles < 5 days, another set > 3months and hence not understanding undercapitalisation sounds like it might be a problem for me to do this.

BTW - one of algorithms recognises noise and uses weighted acceleration of ROC in conjunction with moving averages - in that sense in calmer bull markets the noise disappears

Click to expand...

IMO Algo Trading and Automated Technical Trading are not the same thing. I believe the term "Algorithmic Trading" is used by brokers to execute customer orders in an optimal fashion.

Undercapitalization means to me that you have less capital that it is required to cover margin and intraday drawdown at some point and to keep your position risk low enough. There are some very good books on the subject I recommend that you read at least half a dozen before you start risking your hard earned money in the markets.

One that comes to mind is the new book by Michael Harris, "Profitability and Systematic Trading". He has a chapter on risk and money management and he is down to earth and practical.

Rhody Trader's book may have good information in it, I don't own a copy but I plan to order one soon.

intradaybill said:

IMO Algo Trading and Automated Technical Trading are not the same thing. I believe the term "Algorithmic Trading" is used by brokers to execute customer orders in an optimal fashion.

Click to expand...

Definitely fall into the latter camp at the moment. Thanks for the tips btw.

I think I'll just end up coding up what works and then deciding what it is afterwards. The order management side of things is something I've thought about. All in the goodness of time. Am currently reading the £10k wipeout thread.

i've been testing dozens of expert advisors. [ea]

the essential problem is there are three types of market london, usa and asia and two types of price ie trending and range. no 1 ea will cover 24hr trading or the basic 6 variations of a 24 hr market.

so you end up with specific ea for specific jobs that will only trade when the exact conditions are met. which might mean 1 trade a week or what have you.

another problem is the market is mainly computers these days ie 60% and on up which is changing the character of the market. These are programmed by maths dons from oxford and elsewhere who take the view price has no memory and so laugh at anyone who uses TA which is nothing but believing price has memory which they equate as 'believing in santa'. There are courses at Oxford University on trading against the chartists who are seen as 'predictable. A random market produces the appearance of patterns- like clouds in the sky can look like 'things'. People will see these random generated patterns and 'believe' this is due to 'price memory' and so will trade them with 'that belief.' So you can actually trade the chartists by saying if a random generate pattern emerges which is known to be believed by chartists trade it to take them out.

yes most trading errors are human errors. most people have a system that 'works' if they traded all the signals and traded them according to the system. which we don't because we get bored etc.

given the random nature of price with no memory that gives the 'appearance' of patterns trading with TA will always be as much luck as judgement or rather those who have the most discipline are likely to do better.

Big traders look at something called R2 to measure performance. an R2 of 1 is perfect [every trade a winner] and you should automate yourself out a job. most look at an r2 of 0.3 to 0.7 as 'normal'. but that is a big topic one can search for on the net.

brokers /sbs etc don't have to fix prices etc like some think. the randomness of price and human error gives them a good living.

automation isn't the big winner. its just frees you from sitting there.

the myth of big bucks in finance has been showed to be mainly based on fraud by a few?

My day job is running the development for a huge system used in the www across Europe. It handles about 4m transactions a day. I never ceased to be amazed at just how predictible it's usage is and how that usage varies in a really predictible way by day of week, time of day and through external events like Champion's League football.

I'm applying the same basis to the system I am coding. People and system buying/selling behaviours create repeated patterns that appear in the market. My system/code doesn't care what the commodity or price is. It's looking for the pattern only. Some patterns are more sophisticated than others and I'm expecting a varying degree of success.

FTSE market behaviour although random does exhibit patterns. It is not white noise. Mathmo's from Oxbridge may be correct about price memory but they probably forget to explain that it's behaviour is not random or pseudo-random either.

Maybe my project will show itself to be a waste of time. Maybe it won't. Money isn't my primary motivation here. It's the intellectual challenge of building the thing in the first place, the by-product of success being a degree of financial success. Hopefully.

most TA standard patterns are automated and most are free on the net along with their test results.

but if there are patterns that are derived from transactions then that would be more exclusive as its would be based on info most people would not directly have access to. that would be worth trying.

automation is the future and even if the system doesn't give great results it has the virtue of eliminating emotional mistakes. it can just sit there like a nodding donkey in a field bringing in whatever it brings in.

oiltanker said:

but if there are patterns that are derived from transactions then that would be more exclusive as its would be based on info most people would not directly have access to. that would be worth trying.

automation is the future and even if the system doesn't give great results it has the virtue of eliminating emotional mistakes. it can just sit there like a nodding donkey in a field bringing in whatever it brings in.

Click to expand...

That's kind of where I'm at with all this, come up with some mechanisms that aren't entirely based upon standard TA stuff (although of course it does measure averages, MACD, ROC etc) as well as the unemotional nodding donkey mentality to it.

I think over time as my knowledge and sophistication increases, it will be reflected in the code.

Thanks for all the support and tips so far.

what about neural networks or fourier transform?, these find patterns and estimate the cycle.....ta does work and can be modelled. Many do not use the correct ta for their analysis imo. But i agree with you that professionals laugh at retail chartists.

actually thinking about it you could possibly model a code based on the opposite of a chartist and probably expect a positive expectancy over time.

ta is only an estimation of cycle similar to 'form' in sports betting as we can only achieve an estimation and the best we can hope for. But you can model a code around this so long as you realise its only ever an estimation.

have you looked at statistical trading? or a mechanical system using fundamental data like earnings would possibly outperform a ta system?, what do you think?

Building a neural network. Easy to build. Training it though is another matter altogether. A friend of mine is basing his system on a neural net.

Using FFT's. I've coded FFT in the past for digital signal processing. This is ok for simple datasets.

Both FFT and Neural Nets require a lot of computational horsepower if the dataset is large and relatively noisy as is the case with price data.

I'm going to stick with what I've started based upon TA and other more exotic things I'll try and derive.

I know nothing about statistical trading. WRT fundamental's, I've decided to go down the TA route rather than the fundamental route. I get a sense that market behaviour has little to do with the company and rather more about the faddy nature of human behaviour.

yes i know what you mean and it works because trading is a mental game hence the fact we can see these behaviour patterns.

people knock gambling systems but there are sound stats in the betting field which can be realied upon where as trading is so much of an estimation imo....of course both can and are modelled successfully

by the way just some advice as i myself run my own codes......................

forward test for around 6 months with different settings and different versions
dont try to curve fit the code
code should produce posive expectancy over time
don't give up on the code if it has say 5 losing days, judge it at end of the month
if its ta system then dont trade it around news

you probably know all this but just in case it helps

my own code to date has produced 0.41% win rate with ave winner being $622 and ave loser being $410. Not much but it has alot of frequency.....I truely believe this is the best you can hope for in trading. i use fourier transform but my indicator reduces alot of the noise associated with this concept.

johnsue said:

by the way just some advice as i myself run my own codes......................

forward test for around 6 months with different settings and different versions
dont try to curve fit the code
code should produce posive expectancy over time
don't give up on the code if it has say 5 losing days, judge it at end of the month
if its ta system then dont trade it around news

you probably know all this but just in case it helps

Click to expand...

all advice is good advice right now

a) This is running for 12 months before I consider doing it with real money
b) Not doing any curve fitting.
c) Time is name of game - I know I won't get anything statistically meaningful in less than a month. Also bearish volatile nature of FTSE means I need to run this for longer anyway
d) ok - point taken
e) Why can't I leave it running through news as this will happen in real life and it needs to compensate for this.

How long is it taking for your FFT to run? What kind of dataset and size are you processing?

Hi, I am planning to go a similar direction as you are robster, i.e. writing my own automated trading system. I have a strong background in software and zero in finance So currently I'm just reading as much as I can. I would like to start paper trading soon to play with some algorithms, and I'm looking for some directions to get started. @robster if you don't mind me asking, which API/broker are you using and are you happy with it? Are you paying for a broker account for the 12 months in which you're just paper trading, or is there a free alternative? Thanks!

hansdampf said:

Hi, I am planning to go a similar direction as you are robster, i.e. writing my own automated trading system. I have a strong background in software and zero in finance So currently I'm just reading as much as I can. I would like to start paper trading soon to play with some algorithms, and I'm looking for some directions to get started. @robster if you don't mind me asking, which API/broker are you using and are you happy with it? Are you paying for a broker account for the 12 months in which you're just paper trading, or is there a free alternative? Thanks!

Click to expand...

The app is modular. There is a function called 'the fetch' where you basically pass stock codes into it. It currently sends a request off to yahoo finance. It's kind of like a REST interface, delayed by 15 mins but good enough to slurp data from for basic processing.

Once the application works to my satisfaction, all I will need to do is write a different interface function. The rest remains the same.

Have look here. If you've got a background in web services then it's simple.

stock yahoo data Ilmu Saham | ISX Stock Market Resource Center

Like most good webby stuff, it's free.

Happy new year

Rob,
I was interested to read your thread. After 6 years of writing algo trading apps for an investment bank & subsequently systematic hedge fund, I'm having a go at doing it myself. I've written an application that connects to various data providers, collects & stores prices and then generates a daily signal based on various technical & statistical indicators I've designed. The resulting trades are then transacted automatically. There are loads of things to think about and I'm happy to discuss in detail if you want to contact me, but here are a few "pointers":

- Always start with a hypothesis. The worst thing you can do is crunch some past data and try to produce a trading rule - that way you'll just end up overfitting your data and losing big time going forward. Start with an idea which includes a justification (e.g. a dummy example might be "I think the FTSE always goes up the day after it has fallen more than 100pts, as people think it has been oversold") and then analyse the past data to see if it fits.

- Understand risk. As a medium/long term trader I never trade a fixed percentage of my capital but instead vary my position based on volatility. Once my system has generated the appropriate signal for a contract, the final step is always to multiply by my own volatility adjustment factor, with the result being the position I want to hold. That way, I bet smaller amounts when volatility is higher (as an example, volatility of the ftse has been averaging about 3 times as much in Dec2008 compared to Dec2007, so for the same signal coming out of the model I would only take on a position 1/3 the size).

- Don't under-estimate technology risk. You can have the best model in the world but it means nothing if your program or hardware is unreliable as Murphy's law says you will suffer a failure just when it will hit you hardest. I've developed a whole series of tests for my code and whenever I make changes I always re-run the tests to make sure nothing has broken. I also host my code on a dedicated VPS as my home internet connection is unreliable.

Hope that gives you some things to think about, let me know if you want to discuss more.