Blog #54 Is Machine Learning the Holy Grail of Financial Prediction?

Since we are in the era of ‘big data’ and fast computers, you may wonder whether machine learning has conquered finance, more specifically whether the computer can now replace humans in making accurate predictions of asset prices. If so, more power to robo-advisers!  If not, we’ve got to downplay the hype.

There has indeed been big leaps in in machine learning in specific domains (by the way, I will use machine learning interchangeably with artificial intelligence or AI). In 2017, the world recently witnessed one particularly impressive feat when Deepmind’s AlphaGo program beat the world’s champion Lee Se-Dol in the complex game, Go. Deepmind went on to develop an even smarter version of the program called Zero that can learn on its own, given the basic rules. After three days of self-play, Zero was smart enough to defeat the version of itself that beat Lee Se-dol, winning handily 100 games to nil. We are told that there is more to come!

Despite this impressive feat, it is strange that we don’t hear much about the success of AI in finance. I can think of two reasons why this is. One, the tech geniuses are keeping their secrets close to their chest for obvious reasons. Two, maybe there isn’t much success to brag about in the first place. It’s possible that even today’s clever machine learning technologies are not clever enough to be able to predict stock and bond prices accurately. To understand why this might be the case, let’s consider the limits of AI according to some of the world’s top AI experts.

Dr. Dave Ferrucci, co-founder and chief executive of Elemental Cognition and a former AI expert at IBM stresses that machine learning is simply a statistical technique for finding patterns in large amounts of data. He adds that “having a computer spew out an answer is not sufficient in the long term. You want to say, ‘here’s why‘ “.  But understanding the “whys” of financial asset prices is much harder than understanding the rules of a board game like Go. Despite its complexity, Go, like all board games, is actually quite easy for computers to figure out because the rules are finite – there is no hidden information and importantly, no element of luck (unlike finance). This means that researchers applying AI to Go can have access to a perfect simulation of the game. They can program their software to run millions of tests and be sure it’s not missing anything. Not many fields meet these criteria. Key examples that do include language translation, speech recognition and image recognition. Finance isn’t one of them.

It is not difficult to see why.  As Stanford University’s Andrew Ng, one of the founders of Google Brain, Google’s deep learning project, emphasizes, AI works only for problems where clear inputs can be mapped or linked to a clear output. This criterion limits the applicability of AI to problems involving categorizations such as the fields mentioned above.  Finance is orders of magnitude more complex than these problems. Take stock market predictions for example. It is incredibly hard to pin down the most important inputs or causal variables that drive stock prices. First, there are too many of variables to consider (GDP, inflation, interest rates, oil prices, exchange rates, and most slippery of all, market sentiments). As Warren Buffet once said: “investment is a game of a million inferences.”

Secondly, the financial environment is a dynamic one. Variables that are highly influential for stock prices during high inflation periods may be less so during low inflation periods and vice versa. Trying to “catch” the best set of predictors is like trying to catch a butterfly; the more you chase it, the more it will elude you.

Third, even if you think you can nail down have the relevant predictors variables, how do you put them together in a model to generate your predictions. A model is an equation that relates what you want to predict (say, stock returns) to a set of predictor variables. Coming up with an accurate prediction model is far from trivial. You can ask any finance professor; I guarantee you will walk away unsatisfied with the answer (I know; I was a finance professor myself).

What about computer scientists and mathematicians, the people most closely associated with pushing the limits of AI? Swetava Ganguli and Jared Dunnmon are two ‘quants’ at Stanford University who has done research in using AI to predict bond prices. They ask: how good different machine-learning techniques are in predicting the future price of corporate bonds? They use standard “shallow learning” techniques as well as more exotic neural-network techniques to answer this question.

Before I summarize their findings, some background on bonds. Corporate bonds are IOUs issued by firms or to raise money for the firm’s business. In return, bond investors receive interest and at the bond’s maturity, the bond’s principal value. As every bond investor knows, bond prices are interest sensitive. Moreover, they have to contend with default risk. The higher the default risk, the lower is the bond price. Yet, compared to stocks, bonds are relatively simple instruments and bond prices are generally less volatile.

Given these features of bonds, you would think that predicting bond prices is a piece of cake for machine learning programs. This is not what the researchers found! I won’t go into the details of their study (see the reference below).  They found that while the best predictions came from the neural networks, these fancy techniques took several hours to work their magic – too slow to be useful for actual bond traders. Interestingly, simpler such as linear regressions aren’t so bad in terms of both accuracy and speed. This is a slap in the face for AI. At the same time, it also affirms that simpler methods may be more robust (read: less bad) for highly complex problems, a fact that data scientists have known for some time. That may be the reason why there are so few stories about the success of AI in financial predictions.

Reference:
Swetava Ganguli and Jared Dunnmon (2017), “Machine learning for better models for predicting bond prices“, arXiv:1705.01143 (link)

Advertisements

Blog #53 Let’s Talk About Momentum Trading

I tested a popular contrarian trading strategy in blog #50 using Shiller’s PE ratio and found it wanting. The flip side of contrarian is momentum trading, which involves buying stocks after they have risen and selling stocks after they have fallen. So, momentum is about “going with the flow”, instead of against it. Since the contrarian strategy doesn’t seem to deliver the goods, you may wonder whether momentum trading will do the trick. This blog will attempt to answer this question with data.

As before, my data is from Robert Shiller’s website and consists of (a) the price levels of the S&P 500 index from Jan 1881 through Dec 2017 and (b) the index’s cyclical-adjusted PE ratio (CAPE) computed from its constituent stocks.

I will present two versions of the momentum strategy: a basic version and a ‘crash proof’ version. I will call these versions, MOM1 and MOM2 respectively.

MOM1 works as follows: starting with a capital of $1, I buy the index if the CAPE in each of the previous twelve months is above 15  (call this trading signal CAPE >15). Otherwise, I do nothing. If the index is bought, it is held until a sell signal emerges i.e., when CAPE < 15. To keep things simple, I assume that trading cost is zero and that cash earns nothing. I repeat this strategy each month to the end of the sample period (Dec 2017). I then compute the terminal value (TV) by compounding the monthly returns and compare it with the terminal value of the buy-and-hold (BH) strategy. The strategy with the higher TV is the winning strategy.

The performance of MOM1 is shown in the first line of the table below.

Momentum_resullts
MOM1’s terminal value is shockingly low – just $2.58, compared with $430.43 for the buy-and-hold strategy. In fact, it is worse than the contrarian strategy discussed in blog #50. The BH strategy is always invested, while MOM1 is out of the market about 56% of the time. This can be seen from the following graph where “1” indicates being is in the market and “0” indicates being out of the market. The high outage of MOM1 is what causes its terminal value to be so embarrasingly low.

Momentum_chart
Can we salvage the momentum strategy by tweaking it? The answer of course, you can!  After all, all trading rules are arbitrary. As a researcher, I am trained to be skeptical of data mining as this is called. Data mining is useless in data where there are no repeatable patterns to aid predictions, as is the case with stock prices which behave close to random walks. Nevertheless, to satisfy skeptics, I will relent and subject another version of the momentum strategy to the test. I will call this version, MOM2.

MOM2 works like MOM1 except that it suspends buy trades following major stock market crashes. MOM2 draws on behavioral finance research. This research shows that investors tend to freak out after a big drop in stock prices. Daniel Kahneman (of Thinking Fast and Slow fame) calls this fear, loss aversion. The bigger the drop, the more it sticks in the mind, and the more loss-averse investors become. Hence, MOM2 stops buying after major crashes even if the trading signal screams ‘buy’.

It is tedious to split hairs over the definition of a crash. So, in the interest of simplicity, I suspended buy trades in just three crashes in the sample period. They are: the October 1929 crash which led to the Great Depression, the Internet bubble crash in 2001 and most recently, the 2008 crash triggered by the US sub-prime crisis. For each of these crashes, no buy trades were allowed until the S&P 500 index regained its pre-crash level. After that, trading follows MOM1.

As expected, MOM2 improves on MOM1 (see second row of table above). Alas, this is small comfort as MOM2’s terminal value of $4.59 is barely one-tenth that of the buy-and-hold strategy!

The above results come as a surprise to me, as I am sure it is to you.  But they seem robust. They are also in agreement with other evidence presented earlier, notably the tendency for investors to chase ‘hot’ markets to their detriment (see blog #46). So, unless your brilliance leads you to the discovery of a wonder trading rule, “neither a contrarian nor momentum trader be” seems to be a sensible (if boring) strategy to follow in your quest for investment wealth.

Blog #52 Does Goggle Hold Your Ticket to Riches?

Google’s search engine has replaced the encyclopedias that I grew up with in the 1970’s. Nowadays, one goes straight to Google for answers on just about anything, including of course news about stock market and what other investors are thinking about the stock market.  You can probably see where I’m heading – Google search is literally ‘big data’ at your finger-tips, perhaps the ultimate place to become insanely rich by finding exploitable connections between financial variables. So, how good is Goggle as a source of information for predicting stock prices?  This is what I want to talk about today.  I happened to have done some serious research on this topic which I will share with you in a moment. Let let me first say a few words about data mining to set the stage.

A couple of years ago, two researchers at the University of Bristol combed a database containing 133 different variables, looking for pairs of variables that are ‘statistically significant’. Being scientists, they set the standards for significance quite high, so that the chances of getting flukes were just 1 in 100.

What they found stunned them. Out of almost 8,800 possible pairings, more than 3,000 were deemed ‘significant’. This is like catching 3,000 tunas with one fling of the fishing net from waters with a random assortment of 8,800 different types of fishes. A fisherman would be ecstatic with this result. But to the statisticians, it was too good to be true. Indeed, when they looked more closely at the 3,000 ‘significant’ variable pairs, most were junk.

 

Here’s another story of such ‘voodo correlations’. A few years ago, psychology Professor Edward Vul at the UC San Diego stumbled on a bizarre study that claimed to show a link between brain activity and the speed at which people walk. Curious, he and his colleagues investigated. What they found was shocking. The authors of the study had simply fished out from a random set of data, patterns that happen to fit their pet theory and then claimed that the results were ‘statistically significant’. This is a classic case of cherry picking – keeping what you like to see by throwing away what you don’t like to see! Thankfully, the majority of scientific studies uphold much higher standards. But there is broader lesson about pattern seeking and it is that our brains are incredibly clever in inventing stories to ‘fit the facts’, often downplaying the role of chance in the unfolding of events. The desire for coherence in noisy situations can be a problem for example, when you put your hard-earned money in the stock market based on skimpy evidence. I’m the first to admit that I have zero ability to predict the gyrations of stock market, which is why I choose to be long-term, buy-and-hold investor.

Before signing off, I promised to share with you my own research on predicting the stock market with Google search. I have two versions of the paper, one for an academic audience and the other, written for the general public. You can read the latter paper here.

 

 

 

 

 

Blog #51 Can You Always Trust What You See?

Confidence leads to trades. If you think you are endowed with a superhuman ability to forecast stock prices, you will most likely want to monetize your talent through picking stocks, timing the market or using charts to divine where stock prices are heading. While some investors do seem to have this gift, research conclusively shows that the vast majority of us sadly do not. So why do so many ordinary folks persist in thinking that they can pick stocks or predict stock market patterns?

As I briefly explained in my previous blogs, it is human nature to want to predict partly because we hate ambiguities and crave for certainty. The urge to sieve out patterns probably has to do with the size of our brains. The fact that we have the largest cerebral cortex among all mammals keeps our ‘smart’ brains busy looking for systematic patterns to exploit. Used in the right context, pattern-seeking can be advantageous to survival. Unfortunately, we often carry this ambition too far, fooling us into seeing repeatable patterns when none exists. We fall into this trap when we try to predict abstract quantities like the prices of financial assets like stocks, currencies and commodities.

Here is an amusing story of pattern seeking that, at first sight, has no connection to finance.

The story began in 1976 when the Viking 1 mission returned photos of the Martian surface. The image of a rocky face in the Cydonia region captured the public eye.NASA1.jpg

 

 

 

 

 

 

 

 

 

When NASA released the photograph almost a week later, they described it as a “huge rock formation in the center [of the photo], which resembled a human head.” Was it a trick of light and shadows, or a remnant of an ancient civilization?

Although NASA scientists quickly determined that the face was created by tricks of light and shadows, the public did not buy that idea. Instead, people seized on the more imaginative story that what they saw is the remnant of an alien civilization, suggesting that other rocky outcroppings in the area may be a crumbling extraterrestrial city.

Since 1976, the ‘face’ has appeared in a number of popular culture references, citing it as an indication of life on Mars at some stage of the planet’s history. Books, movies and television all took part in speculation.

The public’s speculation wasn’t settled until 22 years when higher resolution images were taken by the Mars Reconnaissance Orbiter and the European Space Agency’s Mars Express. Those images revealed that the original analysis of the Martian ‘face’ was correct: what the pubic wanted to believe was just an optical illusion caused by light hitting the surface and wells of the rocks at specific angles.

Here are images of the ‘face’ as disappearing over time as revealed by higher resolution images.

NASA2.jpg

You may laugh at this story and the gullibility of those who insisted that the face of Mars belonged to a giant human or alien.  But when we insist that past stock prices reveal trends for the future, aren’t we just as gullible? Some lessons in life are hard to learn.

 

 

Blog #50 Timing the Market vs Time in the Market

Its time to put the Shiller PE ratio to the test.  As mentioned in my previous blog, Shiller calculates a special PE ratio which he calls P/E10. He defines this ratio as price divided by the average of ten years of earnings (a moving average) and adjusted for inflation. Price refers to the index level of the S&P 500 index, and earnings are those of the 500 constituent stocks.

The PE15 trading rule is a contrarian strategy that buys the index when the PE is below 15 and is out of the market when the PE is above 15. It is an active strategy compared to the buy-and-hold (BH) strategy. In this blog, I will compare the profitability of the PE15 strategy with that of the passive BH strategy. The winner is the strategy that produces the highest compounded or terminal value, starting with an investment of $1.

My data is obtained from Professor Shiller’s website. The full dataset runs from Jan 1881 to Jan 2018, a total of 137 years. While a long dataset is a treasure trove for researchers, no real investors have such long time horizons. Therefore, in addition to the full dataset, I also show results based on a more recent period, from 1970 to 2018.

I implement the PE15 trading rule as follows: every month, I check whether P/E10 was above or below 15 for each of the last twelve months. Denote these signals as >15 and <15 respectively. If the signal for a month is < 15, I buy the index, and hold it until a >15 signal appears. When this happens, I cash out of the index. For simplicity, I assume that cash earns a zero rate of interest and exclude dividends from the return calculations. I also ignore trading costs (this will actually skew my results in favor of the PE15 strategy).

Here are the results.

Table1.jpg

Over the full sample period, one dollar increased to $430.43 for the buy-and-hold (BH) strategy but grew to only $229.98 for the PE15 strategy.  As the second row shows, the PE15 strategy is also inferior to the BH strategy in the more recent period. The last column (% Out) shows the percentage of months when the investor is out of the market. For the BH strategy, this is zero. For the PE15 strategy, the investor is 47.5% out of the market over the full sample period and a whopping 60% in the more recent period. Clearly, the investor following the PE15 trading rule paid a high price for being out of the market for so long.

Okay, the PE15 trading rule didn’t quite work out.  Maybe, it is too premature to say that the market is overvalued when its PE is above 15. As the famous economist, John Maynard Keynes once said, “markets can remain irrational longer than you can remain solvent“. So, let’s raise the bar to a PE of 20.

Here are the results of the new test.

Table2.jpg

Its the same story – BH beats PE20 hands down. Notice that missing out just 20.7% of number of months in the full sample period was enough to bring the terminal value down from $430 to $363, a difference of 16%.

Conclusion: Shiller has given us a rich dataset and a nice chart. But no, 15 is not a magical number for stock traders. And neither is 20.

 

 

 

 

 

 

Blog #49 Is 15 a Magical Number?

Remember this guy?

Shiller

He is Robert Shiller, Yale economist, author of the bestseller, Irrational Exuberance and co-winner of the 2013 Economics Nobel Prize (see blog 27).  Shiller has spent a big part of career studying episodes of insanity in the stock market, especially periods when stock prices went into orbit, reaching levels that were way above its long-term price-to-earnings (PE ratio).

Take a look at the following chart and you will see what he means.

Shiller Graph

This chart is downloaded from Shiller’s website just this week. The jagged line is the PE ratio of the S&P 500 index from Jan 1871 to Jan 2018. The S&P 500 is the barometer of the overall US stock market which comprises thousands of firms, many too small  for institutional investors to bother.

Shiller calculates this PE ratio is a special way. He divides each month’s P by an average of the previous 10 year’s earnings of the 500 firms in the S&P 500 index. This is to account for the fact that earnings are cyclical. The 10-year average is smoother and arguably more accurate measure of ‘normal’ earnings than earnings in any particular year. Because earnings 10 years ago are not the same as current year earnings due to inflation, he also adjusts for that. The result is P/E10 shown on the vertical axis.

The historical average P/E10 ratio is about 15 (the moving average version of this mean moves around over time, but not much relative to stock prices). The consensus among market observers is that 15 is the market’s fundamental value, like an anchor that stock prices eventually return to if they go astray.

What catches your eye when you look at Shiller’s chart?  If you take 15 has the ‘rational’ PE ratio of the stock market, then clearly, there are many periods of insanity in the stock market. Shiller himself identifies 1901, 1929, 1966, 1981, 2000 as key periods when stocks were grossly overpriced. But what goes up must come down. Big crashes following exuberant periods are evident from the chart. The opposite is also true: the market always rebound after periods when stocks were oversold.

Being a behavioral economists, Shiller argues that these big swings largely reflect’animal spirits’ – investor emotions gone overdrive, leading to extreme overoptimism and pessimism. You can tap further into Shiller’s thinking in his popular book, Animal Spirits: How Human Psychology Drives the Economy and Why it Matters, (2009, with fellow Nobel laureate George Akerlof).

Shiller’s chart also leads to a tantalizing thought: can one profit by selling stocks when the market’s PE ratio is above 15 and buy when the PE ratio below 15?  Let’s call this the PE15 trading rule. This question brings me back to my recent theme, which is that many investors think there are profitable trading rules for predicting stock prices.  Is PE15 a trading rule that investors should follow to avoid market excesses or hunt for bargains?

I will deal with this practical question in my next blog 🙂

 

 

 

Blog #48 A Random Walk on the Stock Market

Did you wonder how the ‘fake’ stock prices in the previous blog were generated?  Statisticians have long noticed that stock prices bump around in a way that can be described as a random walk.

Random walk is the technical name given to a statistical process where the object fluctuates in a totally unpredictable manner, much like the results from tossing a fair coin (the chance of each throw landing on a head (H) or tail (T) being equal to 0.5). As anyone who has thrown coins in their spare time knows, occasionally one sees ‘nice’ sequences like HHHH or TTTT and we are tempted to think that the coin has remembers the past and repeats it.  Yet, the chance of a next coin toss giving a H or T remains at 50-50 (otherwise, yours is a biased coin).

Now, swap heads and tails for ‘up’ and ‘down’ for stock prices and the same cautionary tale applies. Thus, the fact that a stock’s price increased 4 days in a row is no guarantee that it will rise again on the 5th day (unless insiders in the firm have been accumulating the stock ahead of some unreleased good news).

The ‘fake’ stock prices in my previous blog were created using excel according to a certain specified random walk. You can download the excel program below to a picture of the randomly generated prices. The chart is dynamic in that every time you press F9 on your pc or Function F9 on your laptop, a new random price series will appear. Try it!  You  will see that these ‘fake’ prices have jagged ‘hills’, ‘slopes’ and ‘valleys’ that look pretty much like the real thing, except they were simulated.

Generating random walk for stock prices (excel)