Friday, April 9, 2010

The Efficient Market Hypothesis

The stock-market is a fascinating beast. It's the largest casino in the world, better compared to a huge online game for adults. If you are able to predict the future price of a stock you are in (big) business. When it rises, you buy that stock and sell later. When it drops you short-sell that stock (basically selling it now before you own it and paying for it at a later time when the price is presumably lower). But the view held by most academics is that markets are efficient, that is, unpredictable. Imagine there is some knowledge out there in the world that makes the price of a stock predictable, then the first person who knows about it will "gamble it away". It takes only a few people (or even one) to remove the predictable pattern (if it were still predictable, gamble some more until it is no longer predictable). And there is little delay in this process too (since potentially millions of dollars are involved investors will act very fast). And so the hypothesis is that the market is a random walk: utterly unpredictable.

So what are all these thousands of investors wasting their time on? A huge paradox is presenting itself here. An army of investors are presumably making money on the market every day, while an army of academics is claiming they can't. What's going on?

Hypothesis 1: the investors are seeing patterns where there are none. They believe they are beating the market but in reality they don't. Perhaps they gamble on more risky stocks which have a higher average return. It is well know that humans tend to see patterns in data where there are none (it can't be coincidence that I met my old friend in Lissabon during the summer). We hear about the successful investors who have survived but they represent 50% of the population. The other half can be found in the gutter.

Hypothesis 2: Any obvious patterns are absent, but there are hidden patterns that are not public on which you can make money. It is a well documented fact that once a pattern is made public, it will instantly disappear because investors will start using it. But it's rather stupid to post your successful trick to make money on the wall (unless you are an academic). So, we must assume investors are using their own secret rules to trade. Some figured out you should trade on the scale of seconds or less, others use complicated rules of thumb at the scale of days/months etc. The mere fact that publicized patterns disappear tells us that before they were made public they were still predictable.

To me, the markets represent an interesting collective artificial intelligence that determines the true value of stocks very efficiently. Markets have even been used to predict other facts. If you want to know the answer to an arbitrary question (i.e. who will be the next president) start a market and let people bet on it. The collective wisdom of the masses supersedes the wisdom of any knowledgeable individual. We should probably be thinking about how to use this idea for better purposes.


  1. I read this study once about people who start to do day-trading. The study claimed 90% of them goes out of business within a year because they run out of money. The remaining 10% makes a hell of a lot of money though (they essentially make the money the others lost). The study claimed that the 90% did not understand the concept of spread (at least not well enough), which gets them into trouble.

    I think investors also create a lot of predictable patterns themselves. For instance, in technical analysis, stock prices are claimed to fluctuate between support en resistance lines. Whenever a stock hits a support line, investors start buying the stock because they think it hit its lowest point, thereby producing the predicted pattern. It's a perfect example of a self-forfilling prophecy...

  2. Interesting! I've once heard a similar concept called "Human Computation" by Prof. Luis Von Ahn at CMU. He introduces a concept that utilizes collection of human ability for recognizing characters to solve massive OCR problems. Here is a link to one of his presentation.