History does not disclose its alternatives.
– Lord Acton
Investors do not know the impact of different alternatives in history. In fact, history is subject to discovery and this process of historical discovery is subject to biases as we try to sift through facts.
Central bankers tell us the economy would have been far worse without the various QE policies, but we do not know for sure. We only have the path that was taken. Counterfactuals are a poor way of measuring success or failure. Econometric models can simulate different scenarios for a policy change or shock, but if the policy is truly new, this is all guess work. We do not know for sure what will be the reaction of market participants.
Investors have to weigh alternatives and conduct the what-ifs scenarios in an uncertain world. In fact, the more uncertainty, the more scenarios have to be run. History will not be enough because price or performance time series are just one path for a given distribution of return and risk. The past tells us one path of market behavior that may not be repeated again. We expect otherwise, but the assumption, in an uncertain world, is that more what-ifs are necessary.
This lack of knowledge of potential behavior is especially the case with analyzing managers. If we simulate returns for two managers with the same distribution we will not get the same path of performance. We know what will happen if there are many paths generated, but there is potential variation with any given path from a single distribution.
So what can an investor do to minimize risks? Force discussion of alternative scenarios with managers. The process of thinking through alternatives is critical to understand what could be the potential risk of some policy or market shock. For the systematic modeler, this means looking at a lot of history and decomposing shocks to see not what is the average impact but what could be the potential extremes. There is value looking at outliers.