Backtesting with AmiBroker vs ProRealTime

PRTvsAB

So I finally bit the bullet and bought AmiBroker, which is comprehensive stand-alone software for charting and – more importantly for me -backtesting technical buy/sell systems.

Up until now I’ve been using ProRealTime.com‘s free Java-based charting software. The end-of-day data version is free, but their realtime intraday version is subscription based. It’s really nice considering the price, and allows you to program stock screeners and do limited backtesting using a BASIC-like scripting language. Other than being a little buggy sometimes, it’s a pleasure to use.

The major drawback to it is that you can’t backtest your trading systems across an entire universe of ticker symbols. You can only do one ticker at a time. So to get a decent overview of your system, you have to individually test each stock, record the results in a spreadsheet, and then add or average things up to get a sense of how one system might compare to another. But even testing 30-40 stocks can be quite laborious, especially if you’re also tweaking parameters in your system.

Enter AmiBroker. While there are web/cloud-based backtesting platforms available, those are mostly subscription-based. AmiBroker is the only software I know of that is standalone, will test a large batch of stocks at once, and costs a one-time fee. It’s not cheap ($279 at this writing) but I finally decided it was worth the ‘investment’.

I’m a Mac guy, so the fact that it’s PC-only is a drag. I had contemplated running it on a Windows virtual machine on my Mac, but fortunately my kids got a new Windows laptop for Christmas, and I’ve inherited their old one. So I’ve installed AmiBroker and am slowly getting the hang of it. It’s complicated, but having already learned ProRealTime, I’ve got a head start. Note the two platforms are unrelated except that they’re both charting software.

Tonight I was able to translate one of my systems from ProRealTime into AmiBroker’s scripting language, and for the first time ever was able to run a test over a large group of stocks (thousands) and over an extended period of time (Jan 1 2000 to the present). I picked my most winning-est system to test. And the results?

Kinda pathetic.

Why does the system look so good when picking supposedly random stocks on ProRealTime, but look so bad when using AmiBroker? No friggin’ idea. A little depressing, but I guess there’s room for improvement before I invest more money with that particular system.

P.S. The way the two platforms are configured, I will probably continue to use both of them. I’ll use PRT for checking my latest trade data and possibly screening, and AmiBroker for the actual backtesting. We’ll see if that changes going forward.

UPDATE 01/07/15: One thing I’ve found is that the free downloadable EOD data from yahoo finance has a few errors in it. Every once in awhile I lose my shirt or become an instant paper-millionaire due to glitches in the data. I’ve gotten into the habit of a) checking the best and worst trades of a backtest to make sure they don’t look suspicious, and b) not refreshing my data very often.

Wait, what? Not refresh the data?! Yes, really. If I’m testing 15 years of data, does it matter if my end date is 12/27/14 vs 01/07/15? No I don’t think so. And I’m not currently using AmiBroker as screener, so fresh data is not worth the hassle of cleaning it.

Also, AmiBroker allows your backtests to look into the future, which is a bad thing. And it’s quite easy to do it in a subtle way too, like for example accidentally using the Close price as a signal while buying on the Open price of the same day. So it’s important after designing a backtest system that you check to make sure the buy and sell prices are exactly what you expected them to be, and occur on the days your system indicates precisely. While there is a function to test for crystal-ball-reading, it’s not automatic.

Leave a Reply

Your email address will not be published. Required fields are marked *