Using Market Breadth to Gauge Market Health (part 2)

Are my results fishy? Do they not hold water?
Are my results fishy? Do they not hold water?

Welcome to part two of an ongoing series, where I look at different breadth indicators and their viability in describing market health. You can read – and should read! – the introductory post here:

Using Market Breadth to Gauge Market Health (part 1)

So last post, we (ok, “I”, since I’m doing all the work around here) established a baseline by using a moving average of SPY to tell us when to get in and out of the market. If the close had ten days above its moving average, we got in. If it had ten days below the moving average, we got out. This ten-day requirement is something for which I optimized, and it reduces whipsawing.

Rather than sticking with some tried-and-true values such as 120 or 200 days for our moving average period, we optimized the value for our in-sample period of 2010-2012 inclusive, and then saw how it fared in our out-of-sample period of 2013-2015. And for fun, we took a look at the period of 2000-2015, but knowing that using earlier data for an “OOS test” is sometimes frowned upon. A period of 85 days turned out to work best.

Moving on to our first breadth test:

One method for assessing market health I’ve seen around the web lately is the use of new highs and new lows. Here’s how you do it: count how many stocks in an index (I’m using the Russell 3000) are making new 3-month highs today, and also count how many are making new 3-month lows. We’ll use 60 trading days instead, since they’re about the same. You do that every day, and make sure you’re using historically accurate members of the index, so you don’t get any survivorship bias.

Then do some math:

hilo_breadth = new_high_count / (new_high_count + new_low_count) * 100

The “* 100” bit is just so I’m not dealing with a small number, and can think of it as a percentage instead.

I will continue to use the “ten days of signal” requirement, which standardizes these tests to the baseline, and also eliminates whipsawing. We need a threshold to tell us to trade or to get out. When our hilo_breadth is over the threshold for ten days, we get into the market. When it’s below for ten days, we get out. If it bounces around the threshold, we don’t change the status quo.

I first optimized for a single threshold for both entrance and exit**, checking values from 5 to 95 in increments of 5. Here’s a graph of the result, using CAR/MDD as the criteria:

type newhilo 3D 2010-2012 single thThe best value turned out to be 50. However I was skeptical about that value, because it’s so spiky. “Hey dufus, look to the left! What about that nice plateau at 5 and 10?” I appreciate you drawing my attention to that, but there’s a problem: at those values, with a 10-day requirement, no trading happens. We just don’t get values below 5 or 10 for that long in the in-sample period, even during the worst of the market moments. So the values are always above the threshold, and we’re just buyin’ and holdin’. Hah! Thought you had me, didn’t you?

Ok we’ll go with 50, but we’ll be casting a gimlet eye over our results. Next let’s see if there’s an exit value below 50 that improves the results more.

type newhilo 3D 2010-2012 th1 50As you can see, having an exit threshold of 45 improves results a little from the original 50. Let’s use those values: entrance at 50, exit at 45 (with 10-day signal). How does our in-sample equity curve look?

type newhilo 50,45 2010-2012

It’s behaving like we’d hope it would: getting out during the rough periods and getting back in afterward. It seems to enter and exit less than ideally though, as there are big drawdowns at both the start and end of each trade.

We had four trades during the in-sample period. Three were winners, one a loser. The average trade lasted 140 days, and our CAR/MDD was 0.51.

Let’s see how this held up in the out-of-sample (OOS) test:

type newhilo 50,45 2013-2015

The good news is that it didn’t fall apart. We captured the run of 2013. We unfortunately captured the dip of October 2014. In fact, we got out just before, and then got back in for the final meltdown. Doh! And more painfully, we got slammed with the Falling-Down-Stairs of August 2015. And then got out of the market, solidifying our losses.

For the OOS period, we had three trades, only one of which was a winner (the very long one). Average length of trade was 235 days. And our CAR/MDD was 0.48.

The bad news is: that’s worse than using a simple moving average! Our OOS period for the moving-average system gave us a CAR/MDD of 0.69, and the total profit for 2013-2015 was higher as well.

For fun (I have a weird definition of ‘fun’), let’s look at 2000-2015 with our new high/new low system:

type newhilo 50,45 2000-2015We biffed it pretty hard in the 2000 and 2008 bear markets. And total profit was sub-par compared to the simple MA system.

So it looks like using 60-day new highs and new lows as a breadth indicator works, sort of, but is inferior to other methods.

One thought occurs to me: perhaps a more useful method might be to look at the number of new highs as a percentage of the total, rather than comparing it to the new lows. Since we want to get out early, we want to know when stocks stop making new highs and are poised for a downturn. By the time they are making new lows, the worst is upon us. Or…and this is just crazy talk now…what if we use a lack of highs as an exit, and a lack of lows as an entrance? Hmm. Remind me to check this out if I forget. This could be a never-ending process…

Next post, we’ll look at yet another method.

** You may be wondering why I just don’t optimize separate entrance and exit thresholds freely. Rather than the method I used, which was to first determine a single threshold for both, then afterward optimize for a better exit below the entrance. I did at first try allowing free rein to entrance and exit values, but I realized that I was getting awkward values. For example, I might get an optimized value of 45 for entrance and but then 65 (higher than the entrance) for an exit. Which, when you think about it, is weird. Because you could enter and then already be set up for an exit. If it wasn’t for the ten day requirement, you’d get out the next day! And that was turning the trading system into a short-term system, with many trades lasting around ten days. They were triggered for an entrance, were already eligible for an exit, and then were just held until 10 days of exit signal were recorded. That’s a swing trade, which doesn’t tell us much about the overall market health.

Therefore I decided to optimize for a single threshold for entrances and exits first, and only then determine a threshold for exits, which had to be the same or lower. That way we know the threshold had to already be above the exit level when it first moved above the entrance level.

Using Market Breadth to Gauge Market Health (part 1)

Don't you get tired of looking at graphs? So here's a picture of a tree in Alaska that was felled by a beaver. Read whatever you want into it and the state of the markets today.
Don’t you get tired of looking at graphs? So here’s a picture of a tree in Alaska that was felled by a beaver. Read whatever you want into it and the state of the markets today.

We all want to know if it’s the right time to trade. And we’d also like to know which direction. So for at least the last two months like forever, investors have tried to come up with ways to judge how the market is doing.

One technique is to use market breadth as an indicator. There are a number of ways of doing this, but the basic idea is that you look at ALL the stocks in a particular market or index, count the ones that are meeting a particular criteria, and then use that count to determine how the market is doing.

For example, one might take a look at how many stocks in the S&P 500 are up today, versus how many are down. That’s a simple market-breadth measurement.

Simply dividing stocks into positive or negative for the day can be a little misleading though. The market might be very quiet, with more stocks negative than positive. But does that mean the market is in trouble? Not necessarily. What you really might want to look at is stocks that are very strongly negative or positive. And for that, you simply calculate diffusion.

Say you want to compare stocks that moved at least 4% in one direction or the other from the previous close. You count all the stocks that moved up more than 4%, and all that moved down more than -4%. You then calculate:

diffusion = totalup4 / (totalup4 + totaldown4)

Many breadth indicators use some threshold (like the 4%) to calculate breadth in this way. Pradeep Bonde at stockbee likes to compare stocks that are up or down 4% daily, up or down 30% over the last quarter, and some others. Brett Steenbarger at Trader Feed likes to compare the number of stocks making 3-month highs, vs the number making 3-month lows. There are a lot of ways you can measure the overall picture of the market. I set out to make my own comparisons, using a variety of metrics.

Some basics about my testing:

• I used AmiBroker for my software, and Norgate Premium Data for my numbers.

• I used historical constituents of the Russell 3000 index to test. No surivorship bias as a result.

• An analog for market health would be trading the SPY ETF when a ‘healthy’ signal was given, and exiting when an unhealthy signal was given.

• Account was started at $30,000, and the entire account was traded at once, and the resulting profit/loss compounded and traded in subsequent trades.

• I optimize over the period of 2010-2012. This was a useful time period, because it was an overall up market with some dramatic side turns down volatility avenue. I then will show how the results would have worked in the period of 2013-2015 (as of today’s date) and then, just for yucks, will show a graph of 2000-2015 for an even bigger, but perhaps less applicable picture.

• This is not meant to be a trading system per se, but an test of market-health indicators.

• These tests are probably best for determining longer-term trades. I have a short-term mean-reversion strategy for example that trades right through really bad times and still makes money. So using a longer-scale indicator like this would actually be detrimental to earnings for that type of system.

• I compare everything not with mere profit alone, but by using the CAR/MDD measurement (Compound Annual Return divided by the Maximum Drawdown). This rewards profits but penalizes big drawdowns.

• Other than this baseline test, I use diffusion for everything else. Some like to simply subtract the negatives from the positive and use that number. You could have half the market up tremendously, the other half down tremendously, and come up with a ‘zero’ for your result. I’d much rather know the percentage of positive big movers to the total of the big movers in both directions.

So first we need to establish a baseline. Many investors use a moving average of the market to determine whether it’s a good or bad time to be ‘long’. If the close is above the average, start shoveling money! If it’s not, put on your crash helmet and bar the door. Is that the best strategy? I sure hope not, because otherwise this will be a lot of words about nothing! Although it would still be good to know, right?

Investors will commonly use a 120-day moving average, and sometimes a 200-day average, and for the real go-getters, a 40-day moving average. But which is best?

So the first test is to see what results we get when entering the market the day after the close is above the MA of a certain period, and exiting after it closes below the MA. I tested MA periods from 5 to 300 days, in 5-day increments.


The best result was with a 45 day period, but it’s a little suspicious. When you see a sharp peak in results like that, usually your results will fall apart in out-of-sample testing. That’s because you just happened to land on a random patch of goodness, which you will never ever see again.

When looking at this graph, we can say that shorter rather than longer periods are better. There’s more of a plateau in the 80’s range, which might make for a more stable MA period. So let’s pick 85 days.

Now the thing about using a moving average to determine when to trade is that, if your prices are trading in a narrow range, you can get a lot of whipsawing as the close pops over or under the MA. So next I wanted to reduce the whipsawing a little, by requiring that the market have a few days over or under before entering/exiting the trade. Therefore to enter or exit a trade, I tested how many days the close should be above its moving average before entering, and using that same number of closes below the MA to signal an exit. If there are not x days of signal in a row, then the status quo is maintained. I tested from 1 to 10 days, and got the following optimization chart:


type ma 3D, 2010-2012 flg

As you can see, the 1-day requirement held its own. However longer periods of ‘signal’ come in stronger, with 10 days of signals before entering being the best of all. So moving forward, I will require all my entrances/exits to have 10 days of signals before acting. If I didn’t, this would make for some tedious testing, and the results might be spurious. Would trading one breadth indicator with 7 days of signals be better than trading a different indicator with 5 days? Seems like splitting hairs while angels dance on heads of pins.

Ok, so the big reveal: our* in-sample testing had a CAR/MDD of 0.43,  with four trades averaging 123 days each, and a 50% win rate. Here’s an equity chart for your viewing pleasure:

MA 85, 2010-2012
MA 85, 2010-2012

How did we do in our out-of-sample period?

type ma, 85, flg 10 2013-2015

We did just fine. The CAR/MDD was 0.69, with 3 trades averaging 211 days each, with two wins and one loss. The losing trade was held for a mere 27 days. Gains were higher in this period, and we can see that the doldrums of 2015 are reflected in this graph. Ah, if we could have just gotten out earlier in the Dip of October 2014. And yet we managed to miss the Falling-Down-Stairs August 2015 event. Win some, lose some.

And for chuckles and grins, here’s how our 85-day MA/10-days of signals system worked over a much longer period.


The dot-com crash was still painful, but we didn’t lose our shirt in 2008. Just our left sock and perhaps our gold Rolex.

Next post, we look at some more sophisticated market-timing indicators. But are they better? Stay tuned…


*I say “our” because, well, we’re all in this together, right? I’m not using the Royal ‘Our’ and I don’t have a team of minions that I’m speaking for.


DTO: Shorting Oil For Fun and Profit

The black bands are what I used to test my proposed trailing stop, so that I knew the stop wouldn’t be triggered by previous consolidation periods.

Much of what I write about on this blog concerns short-term trading systems, or “swing trading”. That’s not because my portfolio consists mostly of swing trades…it’s doesn’t! Most of my money is tied up in longer-term plays. It’s just that, well, they’re not that interesting to talk about. You do your research, you buy the stock, and then you check in every once and awhile to see if it would like a sandwich or a cold beer. Which does not make for riveting reading.

But I just completed a trade that was a hybrid duration. Longer than a typical swing trade, but not a truly long-term proposition. That’s because I was “shorting oil”.

Back in December, there was a lot of talk about oil prices continuing to fall. This isn’t news to anyone who has been semi-conscious for the past few months. But at the time, those “in the know” were saying silly comments such as “oil won’t fall below $70/barrel” etc. Usually when people start saying something could never happen, those comments turn around and mug them in a dark alley. And meanwhile, the Saudis sounded very much like they were keen on having prices fall through the floor. I have my own geopolitical theories about that, but the ‘why’ isn’t all that important. When it comes to trading, ultimately the price is king. Suffice it to say, I thought it was a good bet that oil might fall to $50 or even – gasp! – $40.

Since I’m not currently prepared to trade options or short stocks, I looked around for a ‘reverse’ oil ETF. Basically this was a security that moved up when the underlying commodity moved down, and vice versa. I wanted to ‘short’ oil without all that pesky margin requirement for short selling.

So I found the PowerShares DB Crude Oil Double Short Exchange Traded Note (say that three times fast), ticker symbol DTO. Why did I pick the “double short” version? Because it uses leverage to double the inverse movement compared to oil. When I calculated the volatility of the “single short” version, it wasn’t enough to justify the amount of capital I needed to invest. In other words, the risk was too small to justify me tying up my money. I needed a higher potential return for the amount of capital I was willing to invest, and the double version fit the bill.

So how to go about this trade? I figured I’d be in it a few weeks or a few months at the most. I knew that this trade did not have the potential to last years and years, because oil would eventually head back up. So a trailing stop was in order, but a tighter one than what I would use for a long-term stock trade.

My first thought was “hmm, how about 2 X the Average True Range below the highest low of the trade?” The 14-period ATR at the time was 2.8681, so twice that is $5.74, rounding up. That was my proposed trailing stop. For each trading day that had a new higher low, my stop would move up so that it was $5.74 below that low. If a subsequent low was lower, the stop doesn’t move. The trailing stop only goes one direction…up. When the price finally closes below that trailing stop amount, I sell the next day.

Note that this stop, like for most of my trades, is not a ‘hard’ stop. I don’t act on intraday highs and lows, but only on the close of each day. If the close is below the stop, I sell the next day at the opening bell.

I did some quick calculations to see how the stop would have fared during earlier consolidation periods in the recent run up. I wanted my stop to be loose enough to make it through these consolidation periods without “stopping out”, but not so loose that I gave back too much profit when the tide turned. You can see the bands I tested on the first image above. The stop I had picked gave just enough wiggle room.

So how did I do? Pretty well, thank you! I did give back a sizable chunk of money in the past couple of days. But even so, I bought this at $68.87 on December 11th, and sold it this morning for $95.30. That’s gain of 38% in about seven weeks.

The pink line is where my trailing stop ended up, which was then hit on Friday’s close.

I must admit, it’s been perversely fun to root for oil to keep falling. Money in my gas tank and money in my portfolio at the same time! But it did also suck that the rest of the market (and the rest of my portfolio) has been less than happy with the falling price of oil. Perhaps oil is finally slowing its descent, and maybe the markets will stabilize.