Best of Frank: The Efficient Market Theory Hoax
[I'm off this week, enjoying the waning days of summer and catching up on a few things I promised to finish before September. In the meantime, please enjoy this example of that great American summer tradition, reruns. This post originally ran May 26, 2009.]
You may have heard of something called the Efficient Market Theory. If you did, it was almost certainly in a negative context, some writer or blogger excoriating those egghead finance professors for confusing the world with 
their crazy and dismal theories. This is a rant mostly heard from the advocates of investing in individual stocks, but is also found occasionally in the arguments of those in favor of active funds over passive (index) funds and market timing over passive asset allocation.
Apparently, this poisonous heresy has been spread by overly educated academics near and wide for decades. They convince their innocent students that it is categorically impossible to make money picking stocks, that anybody who does anything other than buy an index fund is a fool. It’s a viewpoint that is not just wrong, it’s dismally pessimistic and, let’s face it, simply un-American.
Funny thing, though. I actually took finance courses from those egghead professors in college and B-school. I don’t remember an “Efficient Market Theory” being discussed. I still have my old textbooks. No index entry for it. It does appear in the index for Burton Malkiel’s A Random Walk Down Wall Street
Now, I do remember that in some of those finance classes I took there was a lot of talk about the Efficient Markets Hypothesis, or EMH. That’s in the index of most of the old textbooks. It may sound trivial, but the difference between a hypothesis and a theory is, at least in this case, quite significant.
EMH was conceived as a null hypothesis in the 1950s and 60s. On the slim chance that you are unfamiliar with the term, I will summarize. A null hypothesis is a reasonable, obvious, and often naive interpretation of data to explain what is going on. You invent it as the alternative to the new clever theory you are trying to test. “The Earth is flat” and “heavier objects fall faster than lighter ones” are examples of great null hypotheses from history. A key thing to remember about null hypotheses is that they do not need to make sense in the big picture, they exist only as simple explanations that can be disproven to justify more complex ones. The Earth is flat has a lot of issues as a theory, e.g. what happens at the edge and how the objects in the sky work, but it is largely consistent with ordinary daily experience.
EMH states that stock prices reflect all available information at any given point in time. Stock prices instantly change to reflect new information as it arrives and those changes will by definition be unpredictable and random, because anything that could have been anticipated would have already been baked into the previous price. As a result, you cannot make money picking stocks.
It’s not clear that when EMH was born any serious researcher believed it as a theory of how the real world worked. It was thought up as a straw man against which it could be proved that you could indeed make money picking stocks, particularly with some “technical” and “chartist” theories that were then popular and, with the advent of computers, could for the first time be tested methodically.
But when the professors used the computers to look at the data, EMH turned out to be very very hard to defeat with statistical significance. In fact, it would be decades before it would be done conclusively.
This simple empirical observation, that stock prices appear to be unpredictable and random and that it is (almost) impossible to demonstrate any way in which they are not random, had far-reaching and profound implications. The most important one with regard to our understanding of finance is that you can model the movement of stock prices as if they were random walks. Just about all the useful stuff to come out of 20th Century financial theory was based on this trick, including our understanding of how to diversify portfolios and value options and other derivatives.
But as useful as EMH is as an assumption, it is lousy as a grand theory of how the stock market, or other financial markets, work. Like the Flat Earth Theory, it collapses in on itself if you think about it too carefully. Any explanation of how EMH could be true has to start with lots of clever stock traders that collectively find the perfect, all information reflected, true price. As new information arrives, those traders instantly move the price appropriately. And how does that dynamic work? How do those traders wind up with the right price? There must be an economic reward given to the smart traders for getting the price right and a penalty for getting it wrong. In other words, it must be that some traders make money picking stocks, which violates EMH.
So even though they may appear to be random, for some traders some of the time, stock price movements are not actually random but are at least mildly predictable. To use a possibly strained sports analogy, the flurry of gestures the catcher makes tells the pitcher what to throw, but are presumably meaningless to the runner on second. To the runner, even though he knows the signals are meaningful to the pitcher, they can best be considered random noise.
Thoughtful people who understand the stock market don’t say that it can’t be beaten, they say it is very hard. So hard, in fact, that it is unwise for all but a few to even try. This got simplified by less thoughtful people as a belief that the market was perfectly efficient and that all market outperformance, even by Warren Buffet, was due to random chance.
The difference between stock price movements being actually random and might-as-well-be random may seem academic, but is not. Firstly, the arguments made by advocates of stock picking against the Efficient Market Theory should impress no one. Just because it makes no sense that beating the market is fundamentally impossible, it is not true that anybody can beat the market.
Secondly, while the evidence around EMH makes a good argument that an individual investor should not expect to successfully pick stocks, it does not follow that there do not exist professionals who can beat the market. Arguing, as many do, that investing in an active fund is pointless because all fund managers are the equivalent of monkeys throwing darts is specious. (On the other hand, the analogous argument that an individual investor is unlikely to be able to separate the monkeys from the geniuses, i.e. that picking mutual funds is no easier than picking stocks, has more than a little merit.)
And finally, it is worth observing that for many advocates of passive investing, their belief in the efficiency of markets is (appropriately) shallow and limited. It is quite common, for example, and I think John Bogle subscribes to this, to believe in equity index funds but to advocate very large allocations to equity as an asset class, in some cases approaching 100%. This is not consistent with a belief in efficient financial markets.
Allocating most of your money to stocks is itself an active decision that only makes sense if you believe that stocks as an asset class are cheaper than they should be. If you thought the markets were perfect then you would assume that all asset classes, stocks, bonds, real estate, gold, etc., were priced such that on a risk-adjusted basis they all had the same expected future return. Putting all or nearly all of your money in one class would then make no more sense than putting it all into a single stock. You would get no improvement in risk-adjusted expected return, since that is impossible, but would greatly increase the risk you were taking on.
That’s not an unreasonable argument, and one that is perfectly consistent with Efficient Market Theory as I understand it. And yet I’ve never heard it anywhere. Is it possible that this great and sinister theory isn’t quite as widespread as its detractors claim?
Uncategorized | Frank Curmudgeon | 
August 26, 2009 9:01 am
5 Comments
Other Links to this Post
RSS feed for comments on this post. TrackBack URI
Subscribe
By Ron, August 26, 2009 @ 12:17 pm
Burton Malkiel actually admitted in an interview that he believed markets were efficient over the long run, but “they do go crazy from time to time.”
http://www.pbs.org/wsw/tvprogram/malkiel_interview.html
He seems to believe the market is efficient … except when it’s not.
Story: one afternoon at a prestigious university, an economics professor who firmly believed in EMT walked down a path with one of his graduate students, when they both saw a $100 bill lying on the ground. The student started to pick it up but his professor stopped him. “Don’t worry with it,” the professor sighed, “if it REALLY were a $100 bill, someone else would have already picked it up.”
By Jim, August 26, 2009 @ 2:19 pm
The efficient market hypothesis only says that, after expenses, your expected performance using only publicly available information will be less than that of the market on a risk-adjusted basis. This is very difficult to disprove statistically. A few people will outperform consistently for a long period of time because they combine occasional good luck with discipline, but others who are just as intelligent and disciplined will under-perform. It is therefore a superior strategy for the investor who does not have access to special information to focus his energy on the discipline side (e.g., mechanical re-balancing to maintain an diversified asset allocation) while the people chasing the market buy high and sells low.
By Jim, August 26, 2009 @ 2:37 pm
Also, the EMH doesn’t say that markets are rational. Markets are frequently irrational, which makes predicting their behavior difficult even when you do have superior knowledge, and they can stay irrational longer than you can remain solvent. Furthermore, everyone else in the market is trying to behave rationally, so why would you expect to do better?
A consequence of the EMH is that the only consistent way to make money picking stocks is to sell your advice to other investors and hope they don’t keep close tabs on your performance.
By Retirement Savior, August 29, 2009 @ 11:33 pm
I would view the EMH and the MPT as just supporting the idea that diversification is the only free lunch than an investor can get. So beyond 9-10% nominal returns and around 11% standard deviation for a portfolio, the expertise and work involved will have to increase to gain any outperformance.
By mc, November 13, 2010 @ 10:24 pm
A good definition of “null hypothesis” is that it’s something you know is too simple, and you want to use it as a starting point, to measure how much the real world differs from it.