Curiosity is what drives research. When we research, we question, explore and discover knowledge that makes us gain precious insights. This is why, scientists, investors and executives put so much emphasis on it: because research is fundamental for informed decision-making. Most importantly, we don’t want to get stuck with wrong decisions because we use shortcuts, unrealistic assumptions or gut-feelings when we face complexity. We want to see the whole picture, weigh tradeoffs and apply the scientific method constantly.
In this paper, we outline our framework to make this process scalable and to instantly make sense of massive amounts of data. As a recon drone mapping the surface of uncharted territory, our approach helps us shed new light on complexity, guide our decisions and develop investment insights. What does the right call look like?
When Failure is Not an Option
Those who have been in the markets during the last couple of decades know the story of Long-Term Capital Management (LTCM).
Founded in 1994 by John Meriwether – a famed Salomon Brothers bond trader – and run by a pool of finance veterans, professors, PhDs and two Nobel Prize winners, it has been one of the biggest Wall Street hedge funds with almost $126 billion in assets under management.
Its core investment strategy was fairly simple: use quantitative models – like the Black & Scholes formula – to spot price anomalies and invest accordingly, waiting for securities to converge to their fair values in the long term.
In the beginning, this strategy worked exceptionally well, recording net yearly returns of about 40%. However, its success did not last long.
As many of us will remember or have studied in textbooks, by the end of August 1998 Russia defaulted on some of its bonds and strongly devalued its currency – the ruble – creating a financial earthquake and panic throughout the market.
Eventually, LTCM’s academic risk management was blown away like a house of cards. Their portfolios were highly leveraged and unstable, over-exposed to a sudden increase in market volatility. As a result, in just one month the fund lost about 50% of its value and required a massive bailout, which ultimately led to its liquidation. From then, we all learned a very important lesson: instability and incorrect assumptions are costly friends.
Indeed, financial markets do not lend themselves to be easily understood. They are complex systems that have to be thoroughly investigated. Yet, they are one of the most exciting and challenging environments where we have assisted quantitative techniques being applied to decision-making processes.
However, although tempting, quantitative tools are not a one-stop solution to solve problems. Rather, they seem to be useful only after we have understood the reasons why something happens by performing thorough research. In this sense, the scientific method offers us a time-tested protocol to see the big picture and build investment insights – a clue on how to make the right call.
Start integrating AI into your investment process with the most advanced no-code solution.
Book A Custom Demo
Building a Model is Like Drawing a Map
The first attempts to draw maps date to around 600 BC when the Greek Anaximander drew a map of the world assuming it was cylindrical. A few centuries later, Romans were already expert cartographers, meaning that they already used a modern system of projections, latitudes, and longitudes.
The first world map appeared around the 2nd century AD when Roman cartographer Claudius Ptolemaeus depicted the Old World that spanned from about Spain to the Korean peninsula, as shown in Exhibit 1. Ironically, maps have taught us a lot about investing. Indeed, whether they have been used to guide navigators or for military purposes, maps have always been used as an essential tool to understand, orient and make decisions.
This is because they are useful for all the information that they filter out. Indeed, even if they offer us a stylized picture of reality, they help us deal with complexity and go straight to what matters.
For example, if we were to look at the New York City map, the Empire State Building would appear as a tiny rectangular mark in the middle, with no doors, no windows, and no antenna. However, this would be sufficient to get the right directions and go visit it at the corner between 5th Avenue and 34th Street.
To an extent, something similar happens when we apply the scientific method to investment analysis. We want to explore data – in all its depths – to really understand the financial and the behavioral causes that make something happen.
Then, we want to keep only what is useful, the investment insight, and build knowledge layer by layer repeating this process continuously. Eventually, we can translate this knowledge into equations, building out a map – an investment model – that can assist us in taking decisions more rationally and objectively.
However, just as travelers do not expect their maps to be 100% correct but only a faithful representation of reality, investors do not expect their models to be perfect, but only to catch the most important relationships between financial variables.
As George Box – a famous statistician and pioneer in the field of time series analysis – once said: “all models are wrong: the practical question is how wrong do they have to be to not be useful”. In other words, this is why research matters.
Incidentally, this is also why investment firms continuously invest in research. Renaissance Technology – one of the biggest quantitative hedge funds – has its investment models built by a team of mathematicians, physicists, and engineers that sift through vast amounts of data to extrapolate profitable trade signals.
And judging by the performance of their flagship Medallion Fund – posting an average net annual return of about 39% since 1988 – applying the scientific method to research is clearly something that pays off in the long term.
Show, Don't Tell: Anecdotes Are Over
To an extent, the scientific approach to research resembles what the military call reconnaissance, that is, obtaining information from the outside exploration of uncharted territory.
So when researchers – be them doctors, scientists or investors – venture to gather data, review the scientific literature and develop experiments, what they are actually trying to do is to formulate meaning and bridge ideas that we did not know were previously connected.
However, investment research is widely known for being a tricky domain. In this sense, several reasons make the academic literature on the topic very different from other fields of research.
Most notably, it is frequent that the majority of studies rely on outdated statistical techniques, do not take into account real-world peculiarities (e.g. trading costs) or are flawed because researchers are subject to several biases that lower the quality of their research.
Indeed, one common error is that researchers may consciously or subconsciously manipulate data and perform statistical tests so to produce targeted results, an error known as p-hacking.
For example, if they were to assess the validity of the results only by the p-value (i.e. a statistical measure that tells us if a result is statistically significant or not) they may end up accepting results that are indeed only false positives.
Also, another source of error is represented by the prominence of investing strategies to look good only on paper but fail when put in practice.
This is the result of the so-called backtest overfitting, which occurs when good results (in terms of performance) are artificially achieved by systematically tweaking the deployment of the strategy thousands and thousands of times going back in history.
As a consequence, the risk is to be fooled by well-told stories instead of having robust evidence, data and proofs that confirm or discard the initial hypotheses. Indeed, if you are a scientist, anecdotes are pretty much useless if not supported by adequate experiments and tests. This is how the scientific method works.
There Are No Shortcuts to Success
For the past 60 years, the increasing complexity of financial markets, combined with the desire to bring rationality to the investment process has created large and ever-growing interest in the application of quantitative techniques to build investment models.
However, when looked at closely, their results have been disappointing or mixed. Indeed, we often find ourselves in either one of the two cases: over-complex and sophisticated statistical models that we do not fully understand (and do not control), or seemingly performing models that become obsolete quickly as soon as things change. Yet, evolution here seems to be the keyword that this “first” generation of investment models has not been able to address properly.
In particular, a lot can be explained by a concept well-known in the world of computer science: garbage in, garbage out.
In this sense, if we persist to fit something which is intrinsically built to be static (i.e. the majority of econometric models) to something that – like financial markets – change continuously, we cannot expect good results, except out of luck.
In addition, traditional quant researchers face a big computational obstacle: they should be able to simultaneously identify the relevant financial variables to solve (or predict) the problem and how to aggregate them correctly.
Indeed, since each simulation needs to be set up and analyzed one by one, even a group of hard-working quants will not be able to explore all possible combinations and will end up making some sort of assumptions. Complexity, in turn, calls for a scalable way to automate this process.
Eventually, there is no shortcut for success in investing. Rather, instead of searching for the secret formula to apply, there is a twofold change of perspective that has emerged recently.
We have been assisting on the one hand to a “new generation” of models that are leveraging the increasing power of cloud computing and artificial intelligence to dig deeper into data and adapt to evolving scenarios.
In addition, the application of the scientific method in investment research is contributing to building knowledge from the bottom up instead of following mainstream economic models that often rely on unrealistic assumptions.
The result of this decades-long process is a transition from traditional statistical techniques to a new set of models for decision-making.
For instance, in Exhibit 2 financial market dynamics are represented as a big square. Ideally, an investment model will only be able to explain (and exploit) a portion of it.
However, when markets evolve, traditional models do not follow the change and are stuck looking at the past while new ones could take advantage of their ability to move with the market dynamics over time and integrate new information to come up with better investment decisions.
How Does Science-Driven Investing Look?
Apart from being a famous mathematician, philosopher and astronomer, Galileo Galilei is also known for having been the first modern scientist as well as the father of the scientific method.
Galileo – who at the beginning of the 17th century developed the first prototype of the modern telescope – began investigating the sky and had many groundbreaking discoveries: the irregularities of Moon’s surface, Jupiter’s four largest satellites and the existence of sunspots.
According to him, scientific reasoning had to be based on solid foundations. This meant that the proposed explanations of phenomena – called hypotheses – must be confirmed or discarded by the direct observation of nature, that is, by collecting data and conducting experiments.
Later on, this framework has become universally accepted, mostly because it has been possible to expand it far beyond the world of exact sciences and be generalized also to many other fields, like investing.
Indeed, often we hear that financial markets are somewhat inefficient, or that investors commit systematic errors exploitable by following a rule-based strategy. For instance, the mean-reversion effect – caused by investors’ over or underreaction to the news – is one of them, and is the tendency of the best-performing (and worst-performing) stocks to revert back to their average value in the short-term.
However, starting from this intuition, successfully designing an investment strategy is not immediate but an incremental process.
It requires not only that we understand its root causes and economic rationale, but most importantly that we understand the reasons why it should continue to perform in the future and which is the most efficient way to exploit it.
In this sense, applying the following principles of the scientific method to investment research offers us a methodical and unbiased framework to develop such investment insights.
Step 1: Understand the problem
Perform a wide, cross-referenced, and thorough review of the academic literature relative to the phenomenon and retrace its evolution throughout history: from its first appearance to the current state of the art.
From this, we get a 360-degrees and objective evaluation of the circumstances and explore it under several dimensions: its fundamental drivers, its economic rationale and the reasons why it should persist in the future.
Step 2: Build hypotheses
Based on the previous research, we can formulate several hypotheses in the form of non-ambiguous and measurable questions that are going to be tested scientifically. Most importantly, hypotheses must lead to the development of expectations that will be subsequently compared to the output of the experiments.
Step 3: Develop an Experimental Plan
Subsequently, the set of hypotheses is systematically tested through a sequence of experiments. Usually, it means we perform a large number of simulations aimed at isolating the contribution of each component of the strategy (e.g. analysis period, holding period, rebalancing frequency, etc) to its final performance. In this way, one can explore all possible combinations and does not rely on unrealistic assumptions that could lead to unexpected results.
Step 4: Analyze Output and Draw Conclusions
Eventually, the output of the experiments is aggregated, analyzed and compared to the initial hypotheses to evaluate, case by case, if they should be accepted or rejected.
Feedback – the conclusion of this process – is essential to build knowledge and repeat this cycle with new important insights.
Indeed, especially when performing this kind of research, non-findings are as valuable as findings, since they offer additional insights and shed new light on which path to follow and where to orient future research.
A Lab For Investment Insights
To a certain extent, applying the above-mentioned scientific protocol to investment research is anything but an easy task.
Indeed, it requires using the same rigor and carefulness that a pharmaceutical company uses during the development of a new drug, meaning that the research process should be at the same time scientific and scalable to not leave anything unaccounted for.
In other words, we need to build a laboratory for investment insights. Indeed, complexity calls for an efficient way to automate this process. Yet, only recently it has become possible to quickly make sense of the massive amounts of data and simultaneously look at data in all its depths and dimensions.
For example, thanks to the advances in cloud computing, now millions of experiments can be performed simultaneously in parallel and then aggregated, ideally performing the work of thousands of researchers but a lot faster.
Along with this, artificial intelligence and machine learning techniques have revolutionized the way we analyze and see data, catching a lot of the information previously gone unnoticed. As a result, this new framework introduces a big difference with respect to the traditional research approach.
Indeed, embracing it allows researchers to be less prone to bias and naive interpretations of the empirical findings and focus on what matters the most: understanding the underlying dynamics of financial markets, developing new experiments and building investment insights.
The Secret Ingredient of Performance
Everyone who has been hiking knows well that stability is extremely important. Indeed, if we imagine ourselves in a mountain landscape, we would be rather walking on a surface which is rather smooth, instead of one which has a lot of peaks and valleys.
To a certain extent, this is also the key concept to understand the secret ingredient of investing. Indeed, each time we make an investment decision, we do not face a single choice but several simultaneously.
Drawing from the previous parallel, to have a smooth journey we want to avoid putting ourselves (or our investments) in an unstable position, that is, one from which we could suddenly fall for a single misstep.
In this sense, having a tight grip on the ground is the best way of hiking and investing. This is true also for an additional reason: stable things tend to persist – and thus to be profitable.
However, we rarely encounter stability when we analyze financial markets. Instead, as they tend to mutate as soon as new information is available, they are also intrinsically noisy and unstable. In this sense, stability should be seen as the desirable end of a long signal-processing activity rather than the normality. Yet, the previous case of the mean-reversion makes a good example of how instability is to be avoided and carefully managed when we develop investment strategies to exploit this effect.
Besides, it also offers us the opportunity to frame it into a practical context and visually see which is the benefit of using state-of-the-art artificial intelligence techniques to achieve it.
Imagine that, following a large number of experiments that we have run, we end up with a situation like the one described by the surface on the left in Exhibit 4.
This plot, obtained by performing a Multivariate Surface Analysis of the return of the mean-reversion strategy on the holding period and observation window, shows a highly unstable surface.
Indeed, although the strategy yields a positive average daily return, it presents many peaks and valleys. As a result, if we were to invest in something which presents this profile, we would not be able to control the output accurately.
Instead, the plot on the right of Exhibit 4, shows the same surface when artificial intelligence is applied to processing the signal obtained before.
Indeed, by performing this kind of analysis millions of times to explore all possible combinations of parameters, the result is a surface similar to the one we observed earlier (that presents positive average daily returns) but with smoothed out peaks and fewer edges.
In this sense, a smoother and more stable surface translates into better investment outcomes: because we do not depend anymore on a lucky combination of parameters but because instead we controlled it and understood its root causes.
Case Study: Studying Market Inefficiencies
So, if we get back to the previous example, we can imagine designing an experiment to assess the existence of this phenomenon and subsequently, what the optimal time frame is to exploit such signal.
The goal is to identify the observation windows in which this signal is more stable and presents itself with higher consistency. In order to do so, we took all the S&P 500 stocks in the period from 2012 to 2018 and for several observation windows that range from 7-days to 50-days, calculated their relative returns, and ranked them accordingly into deciles.
After that, we held the portfolio formed by each decile from 1 up to 10 trading days following the observation window to see if we would observe the mean-reversion effect. If the effect does exist, we would expect to have a great divergence in the performance from top to bottom deciles. This means that we would expect to see bottom deciles (e.g. deciles 1-3) outperform and exhibit positive performance in the following days, and vice versa (i.e. that top deciles should perform poorly, reversing back to their average values).
Performing a Multivariate Surface Analysis of the return of the reversal strategy on the holding period and observation window, we were able to immediately picture that in the first decile (i.e. made up by the worst-performing stocks in the observation period) the mean-reversion effect is not present and, paradoxically, leads to negative average returns.
Conversely, we see the effect stabilizing on average positive values from the second decile onwards.
The intuition is indeed confirmed by looking at the last two deciles (i.e. the deciles 9-10) in which we observe a negative performance across each decile.
Think. See. Test. Repeat.
Eventually, rigorous research is the foundation of any successful investment strategy. This is especially true when we want to understand the reason why things happen and, from that standpoint, make informed decisions.
In this context, the scientific method offers us a way to cut through complexity and see beyond the noise, where the valuable piece of information lies.
However, it is not a matter of how sophisticated our tools are but of the assumptions that we make during our research. If they are unrealistic, they are likely to lead us on the wrong road.
Indeed, the naive application of traditional statistical techniques to build investment models has produced insufficient results, precisely because they are not meant to capture and follow something that evolves continuously, like the financial markets. Yet, recently overcoming these shortcomings has become possible.
Thanks to the breakthroughs in the fields of artificial intelligence and cloud computing, we now have unprecedented firepower to look at the depths of data and do the work of thousands of analysts.
Because of that, a new research approach has emerged – to factor in science and technology and build investment insight. As a result, we can forgo old assumptions and instead explore all possible combinations.
These new tools allow us to run and aggregate millions of experiments simultaneously and have a meaningful way to quickly make sense of massive amounts of data.
In other words, they make us able to capture what previously went unnoticed.
From them, we discover the secret ingredient behind long-term performance: stability. In this sense, if we reach stability we can control performance and have a clue about what it means making the right call, every time.
