Blog #56 Why We Persist in the Prediction Illusion?

The last few blogs sports a central theme: stock market prediction is a loser’s game for 99% of humanity (the other 1% are the odd investment geniuses we now know but wished we had known much earlier).

If market timing, momentum trading, contrarian trading, technical analysis etc. are detrimental to our wealth, then why do so many still swear by these activities? I believe that the reasons lie beyond finance and has a lot to do with psychology.  In particular, understanding how our beliefs inform or misinform us is the first crucial step in improving the way we manage our personal finance and investments.

At the risk of over-simplifying a huge literature on this subject, here are three main reasons why we often become victims of the loser’s game:

#1 We are pattern-seekers.
The “we” here refers to practically all of us. This is because the normal human brain is hard-wired to look for patterns, whether they are cloud patterns or patterns in stock prices. But as I’ve mentioned many times, stock prices are actually pretty random (which is why they are often described as ‘random walks’). We don’t see them as random because we look with the benefit of hindsight. But hindsight is a poor guide to the future when the underlying object of prediction has a high noise-to-signal ratio as is the case with the prices of financial securities.

#2 We have a poor understanding of probability.
You throw a fair coin six times and you get HHHHHT. We have trouble making the correct sense out of such situations. Some of us will think that the next throw will have a more than 50% chance of landing on a head (after all, there seems to be ‘momentum’ in the coin tosses so far). Others will disagree and bet that the next throw will land on a tail (reasoning that a tail is long ‘overdue’). Both reasoning would be wrong in the current context; heads and tails both have an equal chance of appearing in any given coin toss. We laugh at the folks who fail this simple probability test, yet we fall into a similar trap when we try to second guess future stock price changes based solely on how they have moved in the past.

#3 We regularly get our timing of thoughts and events mixed up.
Have you ever felt as though you predicted exactly when the traffic light is going to turn green, or sensed that the doorbell is about to ring? Did you think you had a sixth sense or a god-given clairvoyance for getting these predictions right? Or did you attribute them to pure coincidence?

Recent psychology research [1] shows that people who think they are “special” are also more likely to think that they have some sort of clairvoyance. The research was designed as follows. First, the subjects of the experiment were asked questions to assess whether they think they have special abilities to tell the future. Next, these subjects were asked to play a game in which they were asked to quickly predict which of 5 white squares was about to turn red. Importantly, the square that turned red from one trial to another was chosen randomly. Therefore, it was impossible to correctly predict the red square with a higher probability than 1 in 5 or 20% (this random aspect of the experiment was known only to the researchers).

Subjects could either indicate that they didn’t have time to finish making a prediction before the red square shows up, or they can state that they did make their prediction before this event and predicted either correctly or incorrectly. If subjects got the timing of their thoughts and the event mixed up, they might think they made the right prediction before the red square turns up more often than chance would explain. Let’s call them the mis-timers. As it turns out, there were indeed many mis-timers. More interestingly, mis-timers were more likely to have delusional beliefs in their “special abilities”.  The study therefore hints at a basic deficit in the brain machinery that plays to folks with the flawed belief that somehow, they are blessed with the superhuman ability to tell the future.

Notes
[1] See Adam Bear, Rebecca Fortgang, and Michael Bronstein (2017), “Mistiming of thought and perception predicts delutionality”, Proceedings of the National Academy of Sciences.  Here is the article link.

 

Advertisements

Blog #55 Let’s Go Fishing

Unless you’ve been living on another planet, you would have come across experts musing about the potential of technology to revolutionize finance.  To be sure, some of the new technologies are indeed game changers – stuff like e-payments, block-chain and cyber-security. These advances are important, but they are kind of mundane. Investors are more excited about the prospect of software so powerful that can comb through tons of financial data, pick up interesting patterns and make accurate predictions about future asset returns. If and when that happens, we will surely have arrived at the gilded age of robo-assisted predictions.

But this is a big “if” and this blog explains why. To do that, we’ll go for a “fishing trip”, where the “fishes” we’re after are a handful of influential predictors of stock returns, out of a sea of possible predictors.

Suppose you want to predict tomorrow or next month’s stock price using a bunch of predictors. There are a multitude of possible predictors for the stock market including things like interest rates, bond yield spreads, inflation expectations, industrial output and GDP numbers and so forth.

So you can see that the first challenge of making good predictions is how to separate good predictors from bad ones.  This turns out to be a daunting challenge even with modern technology.

Let’s say you want to fish out 13 of the best predictors. By “best”, I mean having the best linear fit (something like Y = aX1 + bX2 + cX3 + etc), which is called a linear regression model. Here the X’s are the set of predictors, and the “a’s”, “b’s” and “c’s” are sensitivity coefficients that measure how a change in a particular X changes Y. For stock market predictions, Y is naturally the rate of return on a stock or a stock index.

I need to emphasize that your model must show that it has a good in-sample fit (i.e., based on past data) before we can even talk about prediction. Statisticians measure a model’s goodness-of-fit using a number called adjusted R-square. Never mind how adjusted R-square is computed (Excel can easily do it). All you need to know is that th adjusted R-square ranges from zero (the model doesn’t explain stock returns at all ) to one (a perfect fit).  From my research experience using annual stock returns, models with adjusted R-squares of more than 0.3 are rare, which is a reminder than stock returns are pretty random or not systematically explained by observed fundamentals.

Going back to fitting problem, here’s the key question: how many regression models must you run to find the best 13 predictors out of 100?

Hold your breath.

Answer: 7,110,542,499,799,200

A one followed by 15 zeroes is called 7 quadrillion. So this number is about 7 quadrillion.  How big is that? For comparison, there 100 billion stars in our galaxy. Hence, if this number represents another galaxy, that galaxy has 71,105 times more stars than ours. The number gets even bigger if we start out with 1,000 potential predictors instead of just 100, so large in fact that it takes a computer which can perform 10 million calculations a second more than 22 years to finish the task! For all practical purposes, this type of search problem is intractable. Computer scientist call such problems NP-complete (where NP stands for non-deterministic polynomial time). NP-complete problems are those that are too hard for today’s computers and possibly those in the foreseeable future.

It gets worse. So far, we’ve only been concerned with finding the best k out of m predictors using linear regression based on past data. Even if you manage somehow to find the best k predictors, it doesn’t mean that you’ve found a perfect crystal ball for prediction. Why? Because the world keeps changing. As they say, “past results do not guarantee good future performance”. Hence, your best prediction model using past future may churn out lousy predictions for next month’s stock returns. You need keep learning and updating your model to cope with new market conditions. That is a tough call.  It is hard enough to find the best fit model using past data. To have the best forward-looking model at all times is pure fantasy even for the most sophisticated machine learning software available today. The best we can do with large problems like these is to use heuristics or shortcuts but the problem with shortcuts is that you can never be sure whether the resulting ad-hoc model is robust and accurate.

So, the next time you hear someone claiming to have a state-of-the art robo-advisor that can accurately predict stock returns, you may want to ask if he knows what a quadrillion is 🙂

 

Blog #54 Is Machine Learning the Holy Grail of Financial Prediction?

Since we are in the era of ‘big data’ and fast computers, you may wonder whether machine learning has conquered finance, more specifically whether the computer can now replace humans in making accurate predictions of asset prices. If so, more power to robo-advisers!  If not, we’ve got to downplay the hype.

There has indeed been big leaps in in machine learning in specific domains (by the way, I will use machine learning interchangeably with artificial intelligence or AI). In 2017, the world recently witnessed one particularly impressive feat when Deepmind’s AlphaGo program beat the world’s champion Lee Se-Dol in the complex game, Go. Deepmind went on to develop an even smarter version of the program called Zero that can learn on its own, given the basic rules. After three days of self-play, Zero was smart enough to defeat the version of itself that beat Lee Se-dol, winning handily 100 games to nil. We are told that there is more to come!

Despite this impressive feat, it is strange that we don’t hear much about the success of AI in finance. I can think of two reasons why this is. One, the tech geniuses are keeping their secrets close to their chest for obvious reasons. Two, maybe there isn’t much success to brag about in the first place. It’s possible that even today’s clever machine learning technologies are not clever enough to be able to predict stock and bond prices accurately. To understand why this might be the case, let’s consider the limits of AI according to some of the world’s top AI experts.

Dr. Dave Ferrucci, co-founder and chief executive of Elemental Cognition and a former AI expert at IBM stresses that machine learning is simply a statistical technique for finding patterns in large amounts of data. He adds that “having a computer spew out an answer is not sufficient in the long term. You want to say, ‘here’s why‘ “.  But understanding the “whys” of financial asset prices is much harder than understanding the rules of a board game like Go. Despite its complexity, Go, like all board games, is actually quite easy for computers to figure out because the rules are finite – there is no hidden information and importantly, no element of luck (unlike finance). This means that researchers applying AI to Go can have access to a perfect simulation of the game. They can program their software to run millions of tests and be sure it’s not missing anything. Not many fields meet these criteria. Key examples that do include language translation, speech recognition and image recognition. Finance isn’t one of them.

It is not difficult to see why.  As Stanford University’s Andrew Ng, one of the founders of Google Brain, Google’s deep learning project, emphasizes, AI works only for problems where clear inputs can be mapped or linked to a clear output. This criterion limits the applicability of AI to problems involving categorizations such as the fields mentioned above.  Finance is orders of magnitude more complex than these problems. Take stock market predictions for example. It is incredibly hard to pin down the most important inputs or causal variables that drive stock prices. First, there are too many of variables to consider (GDP, inflation, interest rates, oil prices, exchange rates, and most slippery of all, market sentiments). As Warren Buffet once said: “investment is a game of a million inferences.”

Secondly, the financial environment is a dynamic one. Variables that are highly influential for stock prices during high inflation periods may be less so during low inflation periods and vice versa. Trying to “catch” the best set of predictors is like trying to catch a butterfly; the more you chase it, the more it will elude you.

Third, even if you think you can nail down have the relevant predictors variables, how do you put them together in a model to generate your predictions. A model is an equation that relates what you want to predict (say, stock returns) to a set of predictor variables. Coming up with an accurate prediction model is far from trivial. You can ask any finance professor; I guarantee you will walk away unsatisfied with the answer (I know; I was a finance professor myself).

What about computer scientists and mathematicians, the people most closely associated with pushing the limits of AI? Swetava Ganguli and Jared Dunnmon are two ‘quants’ at Stanford University who has done research in using AI to predict bond prices. They ask: how good different machine-learning techniques are in predicting the future price of corporate bonds? They use standard “shallow learning” techniques as well as more exotic neural-network techniques to answer this question.

Before I summarize their findings, some background on bonds. Corporate bonds are IOUs issued by firms or to raise money for the firm’s business. In return, bond investors receive interest and at the bond’s maturity, the bond’s principal value. As every bond investor knows, bond prices are interest sensitive. Moreover, they have to contend with default risk. The higher the default risk, the lower is the bond price. Yet, compared to stocks, bonds are relatively simple instruments and bond prices are generally less volatile.

Given these features of bonds, you would think that predicting bond prices is a piece of cake for machine learning programs. This is not what the researchers found! I won’t go into the details of their study (see the reference below).  They found that while the best predictions came from the neural networks, these fancy techniques took several hours to work their magic – too slow to be useful for actual bond traders. Interestingly, simpler such as linear regressions aren’t so bad in terms of both accuracy and speed. This is a slap in the face for AI. At the same time, it also affirms that simpler methods may be more robust (read: less bad) for highly complex problems, a fact that data scientists have known for some time. That may be the reason why there are so few stories about the success of AI in financial predictions.

Reference:
Swetava Ganguli and Jared Dunnmon (2017), “Machine learning for better models for predicting bond prices“, arXiv:1705.01143 (link)

Blog #53 Let’s Talk About Momentum Trading

I tested a popular contrarian trading strategy in blog #50 using Shiller’s PE ratio and found it wanting. The flip side of contrarian is momentum trading, which involves buying stocks after they have risen and selling stocks after they have fallen. So, momentum is about “going with the flow”, instead of against it. Since the contrarian strategy doesn’t seem to deliver the goods, you may wonder whether momentum trading will do the trick. This blog will attempt to answer this question with data.

As before, my data is from Robert Shiller’s website and consists of (a) the price levels of the S&P 500 index from Jan 1881 through Dec 2017 and (b) the index’s cyclical-adjusted PE ratio (CAPE) computed from its constituent stocks.

I will present two versions of the momentum strategy: a basic version and a ‘crash proof’ version. I will call these versions, MOM1 and MOM2 respectively.

MOM1 works as follows: starting with a capital of $1, I buy the index if the CAPE in each of the previous twelve months is above 15  (call this trading signal CAPE >15). Otherwise, I do nothing. If the index is bought, it is held until a sell signal emerges i.e., when CAPE < 15. To keep things simple, I assume that trading cost is zero and that cash earns nothing. I repeat this strategy each month to the end of the sample period (Dec 2017). I then compute the terminal value (TV) by compounding the monthly returns and compare it with the terminal value of the buy-and-hold (BH) strategy. The strategy with the higher TV is the winning strategy.

The performance of MOM1 is shown in the first line of the table below.

Momentum_resullts
MOM1’s terminal value is shockingly low – just $2.58, compared with $430.43 for the buy-and-hold strategy. In fact, it is worse than the contrarian strategy discussed in blog #50. The BH strategy is always invested, while MOM1 is out of the market about 56% of the time. This can be seen from the following graph where “1” indicates being is in the market and “0” indicates being out of the market. The high outage of MOM1 is what causes its terminal value to be so embarrasingly low.

Momentum_chart
Can we salvage the momentum strategy by tweaking it? The answer of course, you can!  After all, all trading rules are arbitrary. As a researcher, I am trained to be skeptical of data mining as this is called. Data mining is useless in data where there are no repeatable patterns to aid predictions, as is the case with stock prices which behave close to random walks. Nevertheless, to satisfy skeptics, I will relent and subject another version of the momentum strategy to the test. I will call this version, MOM2.

MOM2 works like MOM1 except that it suspends buy trades following major stock market crashes. MOM2 draws on behavioral finance research. This research shows that investors tend to freak out after a big drop in stock prices. Daniel Kahneman (of Thinking Fast and Slow fame) calls this fear, loss aversion. The bigger the drop, the more it sticks in the mind, and the more loss-averse investors become. Hence, MOM2 stops buying after major crashes even if the trading signal screams ‘buy’.

It is tedious to split hairs over the definition of a crash. So, in the interest of simplicity, I suspended buy trades in just three crashes in the sample period. They are: the October 1929 crash which led to the Great Depression, the Internet bubble crash in 2001 and most recently, the 2008 crash triggered by the US sub-prime crisis. For each of these crashes, no buy trades were allowed until the S&P 500 index regained its pre-crash level. After that, trading follows MOM1.

As expected, MOM2 improves on MOM1 (see second row of table above). Alas, this is small comfort as MOM2’s terminal value of $4.59 is barely one-tenth that of the buy-and-hold strategy!

The above results come as a surprise to me, as I am sure it is to you.  But they seem robust. They are also in agreement with other evidence presented earlier, notably the tendency for investors to chase ‘hot’ markets to their detriment (see blog #46). So, unless your brilliance leads you to the discovery of a wonder trading rule, “neither a contrarian nor momentum trader be” seems to be a sensible (if boring) strategy to follow in your quest for investment wealth.

Blog #52 Does Goggle Hold Your Ticket to Riches?

Google’s search engine has replaced the encyclopedias that I grew up with in the 1970’s. Nowadays, one goes straight to Google for answers on just about anything, including of course news about stock market and what other investors are thinking about the stock market.  You can probably see where I’m heading – Google search is literally ‘big data’ at your finger-tips, perhaps the ultimate place to become insanely rich by finding exploitable connections between financial variables. So, how good is Goggle as a source of information for predicting stock prices?  This is what I want to talk about today.  I happened to have done some serious research on this topic which I will share with you in a moment. Let let me first say a few words about data mining to set the stage.

A couple of years ago, two researchers at the University of Bristol combed a database containing 133 different variables, looking for pairs of variables that are ‘statistically significant’. Being scientists, they set the standards for significance quite high, so that the chances of getting flukes were just 1 in 100.

What they found stunned them. Out of almost 8,800 possible pairings, more than 3,000 were deemed ‘significant’. This is like catching 3,000 tunas with one fling of the fishing net from waters with a random assortment of 8,800 different types of fishes. A fisherman would be ecstatic with this result. But to the statisticians, it was too good to be true. Indeed, when they looked more closely at the 3,000 ‘significant’ variable pairs, most were junk.

 

Here’s another story of such ‘voodo correlations’. A few years ago, psychology Professor Edward Vul at the UC San Diego stumbled on a bizarre study that claimed to show a link between brain activity and the speed at which people walk. Curious, he and his colleagues investigated. What they found was shocking. The authors of the study had simply fished out from a random set of data, patterns that happen to fit their pet theory and then claimed that the results were ‘statistically significant’. This is a classic case of cherry picking – keeping what you like to see by throwing away what you don’t like to see! Thankfully, the majority of scientific studies uphold much higher standards. But there is broader lesson about pattern seeking and it is that our brains are incredibly clever in inventing stories to ‘fit the facts’, often downplaying the role of chance in the unfolding of events. The desire for coherence in noisy situations can be a problem for example, when you put your hard-earned money in the stock market based on skimpy evidence. I’m the first to admit that I have zero ability to predict the gyrations of stock market, which is why I choose to be long-term, buy-and-hold investor.

Before signing off, I promised to share with you my own research on predicting the stock market with Google search. I have two versions of the paper, one for an academic audience and the other, written for the general public. You can read the latter paper here.

 

 

 

 

 

Blog #51 Can You Always Trust What You See?

Confidence leads to trades. If you think you are endowed with a superhuman ability to forecast stock prices, you will most likely want to monetize your talent through picking stocks, timing the market or using charts to divine where stock prices are heading. While some investors do seem to have this gift, research conclusively shows that the vast majority of us sadly do not. So why do so many ordinary folks persist in thinking that they can pick stocks or predict stock market patterns?

As I briefly explained in my previous blogs, it is human nature to want to predict partly because we hate ambiguities and crave for certainty. The urge to sieve out patterns probably has to do with the size of our brains. The fact that we have the largest cerebral cortex among all mammals keeps our ‘smart’ brains busy looking for systematic patterns to exploit. Used in the right context, pattern-seeking can be advantageous to survival. Unfortunately, we often carry this ambition too far, fooling us into seeing repeatable patterns when none exists. We fall into this trap when we try to predict abstract quantities like the prices of financial assets like stocks, currencies and commodities.

Here is an amusing story of pattern seeking that, at first sight, has no connection to finance.

The story began in 1976 when the Viking 1 mission returned photos of the Martian surface. The image of a rocky face in the Cydonia region captured the public eye.NASA1.jpg

 

 

 

 

 

 

 

 

 

When NASA released the photograph almost a week later, they described it as a “huge rock formation in the center [of the photo], which resembled a human head.” Was it a trick of light and shadows, or a remnant of an ancient civilization?

Although NASA scientists quickly determined that the face was created by tricks of light and shadows, the public did not buy that idea. Instead, people seized on the more imaginative story that what they saw is the remnant of an alien civilization, suggesting that other rocky outcroppings in the area may be a crumbling extraterrestrial city.

Since 1976, the ‘face’ has appeared in a number of popular culture references, citing it as an indication of life on Mars at some stage of the planet’s history. Books, movies and television all took part in speculation.

The public’s speculation wasn’t settled until 22 years when higher resolution images were taken by the Mars Reconnaissance Orbiter and the European Space Agency’s Mars Express. Those images revealed that the original analysis of the Martian ‘face’ was correct: what the pubic wanted to believe was just an optical illusion caused by light hitting the surface and wells of the rocks at specific angles.

Here are images of the ‘face’ as disappearing over time as revealed by higher resolution images.

NASA2.jpg

You may laugh at this story and the gullibility of those who insisted that the face of Mars belonged to a giant human or alien.  But when we insist that past stock prices reveal trends for the future, aren’t we just as gullible? Some lessons in life are hard to learn.

 

 

Blog #50 Timing the Market vs Time in the Market

Its time to put the Shiller PE ratio to the test.  As mentioned in my previous blog, Shiller calculates a special PE ratio which he calls P/E10. He defines this ratio as price divided by the average of ten years of earnings (a moving average) and adjusted for inflation. Price refers to the index level of the S&P 500 index, and earnings are those of the 500 constituent stocks.

The PE15 trading rule is a contrarian strategy that buys the index when the PE is below 15 and is out of the market when the PE is above 15. It is an active strategy compared to the buy-and-hold (BH) strategy. In this blog, I will compare the profitability of the PE15 strategy with that of the passive BH strategy. The winner is the strategy that produces the highest compounded or terminal value, starting with an investment of $1.

My data is obtained from Professor Shiller’s website. The full dataset runs from Jan 1881 to Jan 2018, a total of 137 years. While a long dataset is a treasure trove for researchers, no real investors have such long time horizons. Therefore, in addition to the full dataset, I also show results based on a more recent period, from 1970 to 2018.

I implement the PE15 trading rule as follows: every month, I check whether P/E10 was above or below 15 for each of the last twelve months. Denote these signals as >15 and <15 respectively. If the signal for a month is < 15, I buy the index, and hold it until a >15 signal appears. When this happens, I cash out of the index. For simplicity, I assume that cash earns a zero rate of interest and exclude dividends from the return calculations. I also ignore trading costs (this will actually skew my results in favor of the PE15 strategy).

Here are the results.

Table1.jpg

Over the full sample period, one dollar increased to $430.43 for the buy-and-hold (BH) strategy but grew to only $229.98 for the PE15 strategy.  As the second row shows, the PE15 strategy is also inferior to the BH strategy in the more recent period. The last column (% Out) shows the percentage of months when the investor is out of the market. For the BH strategy, this is zero. For the PE15 strategy, the investor is 47.5% out of the market over the full sample period and a whopping 60% in the more recent period. Clearly, the investor following the PE15 trading rule paid a high price for being out of the market for so long.

Okay, the PE15 trading rule didn’t quite work out.  Maybe, it is too premature to say that the market is overvalued when its PE is above 15. As the famous economist, John Maynard Keynes once said, “markets can remain irrational longer than you can remain solvent“. So, let’s raise the bar to a PE of 20.

Here are the results of the new test.

Table2.jpg

Its the same story – BH beats PE20 hands down. Notice that missing out just 20.7% of number of months in the full sample period was enough to bring the terminal value down from $430 to $363, a difference of 16%.

Conclusion: Shiller has given us a rich dataset and a nice chart. But no, 15 is not a magical number for stock traders. And neither is 20.