Coding your own trading platform...Adding a database.

[goonior1]

Not sure if this is the right forum, but for you guys who code your own platform, did you implement a database for your historical stuff (quotedata, trades, executions, etc, etc) or just store things off as a textfile, .csv, etc?
What are the tradeoffs/advantages of having the database?

And, if you do, since I'm not too familiar with databases, any suggestions? (free is good).

I code in c#, if that matters and I've been investigating WPF. Although I don't have a ton of experience with it, data-binding to a database appears that it would get rid of quite a bit of drudgework.

Thanks for any input.

 

[gc1]

I would suggest either MySQL or SQLite, (the latter only if the need is simple and you don't need a client/server architecture). Both have many programming language interfaces including C#.

 

[goonior1]

Good call, I totally forgot about that one...We've used MySQL at work and it's worked pretty well. That's something I'll look in to.

Is it worth using the database for your tick data (not sure how performance/space issues compare to simply storing it as text), or do you just use it for other stuff?

Thanks,

 

[TrailingStop]

In my app I use SQLite to store general information of the securities. SQLite if OK for my needs since I do not have access from more than one app and one user at a time. On the other hand it was easy to integrate (I use it with ++) and has a good performance. I store quest in separate files. One file for each day and each security. For each security I create a new subdirectory in which the quotes files are stored. If I create the quotesfile I compress the contents to reduce the filesize. The access to the files is very fast since I know the security and the day for which I need the quotes. The decompress it and you are ready.

Daniel

 

[alexis_michel]

I would very much like to suggest keeping well away from pretty much every single database vendor other than microsoft itself. C# is a microsoft technology (language, whatever), and will work ORDERS OF MAGNITUDE better with another microsoft technology than with everything else (a fact of life with pretty much every software vendor). Maybe the bugs with other DBs will not be immediately evident, but believe me, they are there. And nasty ones at that.
I would go with MS SQL Server 2008 Compact edition (bundled for free with Visual Studio) or Express Editions (available to download for free also) and develop with it, and should you require something a bit more ... advanced, you can install with only one change of config a fully fledged MS SQL Server 2008 Standard edition, or even higher ended versions.
Nothing will give you better tools (Visual DB Management that any other vendor is still dreaming of), it IS an extremely powerful DB (Nasdaq adopted it a few years back for storing the gazillions trades that go through them everyday).
Do not believe all the hype about other DB, I have simply never worked with something that mature and that developer friendly.

On the other hand, if you don't need Relational data (typically, intraday ticks are not relational, for a back testing engine for example), it would also be a good idea to look at an Object oriented DB like Db4o. One line of code, and your object is stored. 2 lines of code and it is retrieved (as exactly the same object, no need for mapping code or all the plumbing around). I did evaluate it on a tick by tick database, and I could request for 6 months worth of intraday ticks (not bars) in less than a second, it pretty much blows anything else you could have (unless your pockets are deep enough for commercial time series database, and they will need to be very deep). It integrates perfectly with .net (unlike many so called cross platform stuff) and will keep you very happy. As a side bonus, the same as doing a Databinding of a grid in WPF with SQL Server, you can also databind that grid to an object (coming straight out of Db4o). And guess what, you modify the grid, your objects are modified also, ready to be re-commited to your DB.
Worth a serious look to say the least.
My 2 cents.