As previously written by Covenant Capital Management
“So, how did you guys do last month?”
It seems like the most natural and logical question to ask an investment manager. But is it the right question to ask? Most managers either cringe or rejoice internally upon hearing this question, depending on how they performed the previous month. Almost implied in the question is the notion that the manager’s recent performance indicates his ‘skill’ at investing. But was it skill or luck (good or bad) that constituted that month’s performance? And how can one possibly separate the two?
A Laboratory for Luck and Skill
Luckily (or skillfully), there is a scientific approach to the problem. We can consider a game of chance with some known degree of ‘inherent skill.’ For instance, imagine a trader who has a 50% chance of winning each trade he takes. Suppose the trader makes twice as much on his winning trades than he loses on his losing trades. This is equivalent to being paid 2 to 1 on a fair coin flip. The long-run expected outcome of each trade is $0.501. This $0.50 expectancy is the ‘inherent skill’ of the trader. But an outside observer does not have this knowledge. The outside observer only has the trader’s track record and must use it to decipher the trader’s abilities somehow. So how is this best accomplished?
Another way to pose the question above is to say, “How should one look at a track record in a way that will minimize the effects of randomness?” In the scenario above, randomness manifests in whether a series of actual trades are winners or losers. Even though the chance of winning each trade is 50%, this does not mean that in 10 trades, exactly five will be winners. There is some possibility that the trader will only win 3 of the following ten trades – the same possibility that ten coin flips will yield only three heads. If the trader wins 3 (or fewer) trades over the course of 10, his actual performance per trade is negative (-$0.10 per trade for three wins) as opposed to his positive $0.50 expectancy. Thus, his performance over these ten trades vastly differs from his long-term expectancy.
The binomial distribution is the mathematical tool for determining the likelihood of observing three or fewer wins in 10 trades if the underlying chance of winning is 50%. It turns out that the chance of winning three or fewer trades in this scenario is 17.2%. Put another way, even though the trader has a skill that implies he should expect to make $0.50 per trade, there is roughly a 1 in 5 chance that throughout ten trades, he will lose money – simply due to randomness. Similarly, there is some chance that the trader will win six or more trades out of 10, in which case his realized performance is well above his $0.50 expectancy. According to the binomial distribution, the trader’s chance of winning six or more out of 10 trades is 37.7%. There is a 37.7% chance that the trader’s performance over ten trades will be better than his long-term expectancy.
Time Will Tell
The key to separating ‘skill’ from ‘luck’ lies in the fact that the odds in the scenarios given above actually change depending on the number of (independent) observations3. For instance, while the chance of winning three or fewer trades out of 10 is 17.2%, the chance of winning six or fewer trades out of 20 is only 5.8%. Figure 1 illustrates how the odds of observing a winning percentage of less than or equal to 30% for a situation with an actual winning percentage of 50% diminish as the number of observations increases.
From the exercise above, it should be clear that if a trader truly has the skill, it will be more evident over the long run than in the short run. Put another way, the short-term good and bad luck offset over the long run, and all that remains is the manager’s skill or ‘market edge.’
About the author
Scot Billington co-founded Covenant Capital Management, a boutique CTA managing client assets for over 15 years. He co-developed the CCM trading models and manages the CCM office in Chicago. Founded in 1999, Covenant Capital Management, LLC (CCM) is an alternative asset manager. CCM is registered with the Commodity Futures Trading Commission as a Commodity Trading Advisor (CTA) and manages client assets using global futures markets as an investment vehicle. CCM trades in over 40 markets, including equity indexes, fixed income, foreign exchange, agricultural, metals, soft commodities, and energy. CCM offers three primary programs, the Original Program, the Aggressive Program, and the Optimal Program. The Aggressive Program is a more leveraged version of the Original Program.
1 The expectancy or long-run average is $0.50 since 50% of the trades result in a $2.00 gain and 50% of the trades result in a $1.00 loss. Expected Outcome = $0.50 = $2*(0.50) – $1*(0.50).