“Diversification is the Only Free Lunch”
I’m sure everyone has heard this old adage at some point in their trading career. Most people probably shrug it off and go back to watching The Big Short and dreaming of putting on that one career-making trade. Or maybe they’re still trying to figure out how to pick every single top and bottom on one instrument, thinking all they need is that one perfect strategy. The idea of concentrated bets and being right is just so sexy. Compare this to trading a basket of strategies across styles, time frames, and asset classes. Hopefully the thought alone hasn’t bored you to sleep. Because what I hope to prove with this post is that diversification can combine multiple small edges into a portfolio with both reduced risk AND increased reward . This was a true turning point in how I thought about trading, and hopefully it will be for you too.
If you’re just joining the series now, I’d suggest starting with Part 1 here. Jupyter notebook for this post can be found here. To run the code in the notebook, you’ll likely need to install two packages. This can be done as follows:
pip install fix-yahoo-finance
pip install cma
Also, this article builds on works by Howard Bandy and Ralph Vince, so for more thorough explanations of the fundamentals please check out their work.
Back to Basics: Two Coins
We started this series by investigating the effect position sizing had on a basic game of coin flips. Like it or not, that’s where we’ll start again. Only this time, we’re going to use two coins. Truly exciting stuff. To show the effects varying levels of correlation have, we’ll try three different scenarios: two coins that are non-correlated, coins that are positively correlated, and two that are negatively correlated.
For the first two games, the payoffs are the same as before: heads will double whatever you bet, tails you lose 100% of your wager. In the first (non-correlated) scenario, potential outcomes are as follows:
Since both coins will be flipped simultaneously, we’ll have to place our bets prior to the flips. We can place our bets on each coin independently of each other, bounded by the amount that would cause total ruin. In this case, if the sum of the two bet sizes >= 100%, we will be guaranteed to lose all of our money when scenario 4 occurs. The weighted return R for each outcome k based on these bet sizes is calculated according to the following equation:
where:
N = the total number of independent outcomes to bet on (in this case, 2 coin flips)
f = the fraction of bankroll staked on each outcome
x = the return of each outcome
Once we have this set of weighted returns, we can calculate our Geometric Holding Period Return (GHPR) as in the previous articles.
def calc_weighted_returns(returns, weights):
weighted_returns = returns.multiply(weights).sum(axis=1)
return weighted_returns
def calc_ghpr(returns):
returns_array = np.array(returns)
twr = np.product(1 + returns_array)
ghpr = twr ** (1 / len(returns_array)) - 1
return ghpr
Our goal now is to maximize the GHPR for the combination of two bets by finding the optimal fraction of our bankroll to stake on each outcome. We can do so by iterating through each combination of bet sizes, calculating the GHPR value for each, and then seeing where the peak is.
def get_worst_losses(returns):
worst_losses = np.min(returns, axis=0)
abs_worst = np.abs(worst_losses)
return np.array(abs_worst)
def weight_combinations(returns, steps=100):
num_components = returns.shape[1]
worst_losses = get_worst_losses(returns)
potential_weights = np.linspace(0, 1, steps+1)[:-1]
weights_list = [list(potential_weights)] * num_components
combos = list(itertools.product(*weights_list))
combos_array = np.array(combos)
norm_combos = np.divide(combos_array, worst_losses)
return norm_combos
def f_multiple_returns(returns):
strategies = returns.columns
potential_weights = weight_combinations(returns)
f_curve = pd.DataFrame()
for i in range(len(potential_weights)):
weights = potential_weights[i]
weighted_returns = calc_weighted_returns(returns, weights)
ghpr = calc_ghpr(weighted_returns)
for j in range(len(strategies)):
f_curve.loc[i, strategies[j]] = weights[j]
f_curve.loc[i, 'ghpr'] = ghpr
optimal_f = f_curve.loc[f_curve['ghpr'].idxmax()]
return {'f_curve':f_curve, 'optimal_f':optimal_f}
two_coins = pd.DataFrame({'coin 1':np.array([2,2,-1,-1]), 'coin 2':np.array([2,-1,2,-1])})
two_coins.index = np.arange(1,5)
two_coins.index.name = 'Scenario'
f_two_coins = f_multiple_returns(two_coins)
results = f_two_coins['optimal_f'].to_frame()
results.columns = ['']
print(results)
Output:
coin 1 0.230000
coin 2 0.230000
ghpr 0.119119
As we can see from the results above, the optimal strategy to maximize the long-term growth of our bankroll is to allocate 23% of our capital towards each outcome and bet on heads. This means that our total risk is 46% to the account per event (two flips). The expected GHPR per event for this strategy is 11.9%. Also notice that any combination that results in risking > 100% of the account in total guarantees a catastrophic loss.
If you’ll remember back to the first example from Part 1, the optimal fraction to stake on a single coin flip with the same odds was 25% of the bankroll, with an expected GHPR of 6.07%. With two uncorrelated coins, the risk and potential reward are both nearly doubled. At this point, you may be thinking, “why don’t you just bet twice as much on one coin?”. The following scenario will illustrate why diversification benefits us here.
Correlated Coins
Let’s examine the case of flipping two coins that are perfectly correlated with one another; that is, if one comes up heads the other one does as well and vice versa. There are only two possible scenarios for this game:
We can examine the optimal strategy of this game the same way we did the first.
two_coins_corr = pd.DataFrame([[2,2], [-1,-1]], columns=['Coin 1', 'Coin 2'])
f_two_coins_corr = f_multiple_returns(two_coins_corr)
results_corr = f_two_coins_corr['optimal_f'].to_frame()
results_corr.columns = ['']
print(results_corr)
Output:
Coin 1 0.00000
Coin 2 0.25000
ghpr 0.06066
The optimal strategy in this case is to follow the exact same strategy as if their were only one coin to bet on. In fact, just neglect the second bet altogether and stake 25% of your account on the first one, as in the single-coin scenario.
Negative Correlation
If you take nothing else away from this article, I hope this has some impact. This was one of the biggest “A-Ha!” moments I’ve had since discovering financial markets. Here, we’ll examine the case of a second coin that is negatively correlated with the first. If the first coin lands heads-up, we know with certainty that the second will come up tails. But here’s the twist: this second coin will actually have a negative expectancy.
The payouts for this second coin are as follows: on tails, you still lose 100% of what you wager. However, if it comes up heads, you only win 60% of what you wager.
At first glance, one might wonder why you’d even allocate any of your capital to this second coin. The arithmetic expectancy for such a game is to lose 20% of your wager per play; that’s worse than pretty much every game in a casino! But let’s examine the results if we wager on these two negatively correlated outcomes simultaneously:
two_coins_neg_corr = pd.DataFrame([[2,-1.0], [-1,0.6]], columns=['Coin 1', 'Coin 2'])
f_two_coins_neg_corr = f_multiple_returns(two_coins_neg_corr)
results_corr = f_two_coins_neg_corr['optimal_f'].to_frame()
results_corr.columns = ['']
print(results_corr)
Output:
Coin 1 0.790000
Coin 2 0.990000
ghpr 0.130646
Several things are surprising about this result (to me, anyway!). First, our maximum GHPR has actually increased from 11.9% to 13.1%. Second, it’s suggested that we wager almost 180% of our bankroll in total on each bet-able event (required the use of borrowed money or leverage). Finally, we should be allocating a larger stake to the coin that’s guaranteed to lose money over time. These all seem counter-intuitive.
Now, this example is contrived and for illustrative purposes only so we won’t dig too deeply into it. What we’ve constructed is essentially a Dutch Book, where we can guarantee a profit by betting on both of these two outcomes. No casino will offer such a scenario to you, and markets will likely arbitrage away any similar opportunities. However, it demonstrates the power that negative correlation has, and why we should seek it out if at all possible. The same effect can be seen with less-than-perfect negative correlation, but it would have made this toy example too messy.
Drawdown Constraints
If you’ve been following along, in Part 2 we discussed why “optimal f” wasn’t exactly optimal, namely due to the extreme drawdowns encountered at the maximum level of leverage. We’ll address this here by limiting our potential solutions to those where our risk tolerance isn’t exceeded. We’ll modify some of the functions from the previous article to accommodate multiple returns streams.
def get_equity_curve(returns):
equity_curve = (1 + returns).cumprod(axis=0)
# Set the starting value of every curve to 1
# normalized_eq = raw_eq / raw_eq[0]
return equity_curve
def calc_twr(equity_curve, start_at_1=True):
eq_arr = np.array(equity_curve)
if start_at_1 == True:
twr = eq_arr[-1]
else:
twr = eq_arr[-1] / eq_arr[0]
return twr
def calc_ghpr_eq(equity_curve):
twr = calc_twr(equity_curve)
ghpr_eq = twr ** (1 / len(equity_curve)) - 1
return ghpr_eq
def calc_drawdown(equity_curve):
eq_series = pd.DataFrame(equity_curve)
drawdown = eq_series / eq_series.cummax() - 1
return drawdown
def calc_max_drawdown(equity_curve, percent=True):
abs_drawdown = np.abs(calc_drawdown(equity_curve)).values
max_drawdown = np.max(abs_drawdown)
if percent == True:
return max_drawdown * 100
else:
return max_drawdown
def calc_ulcer_index(equity_curve):
drawdown = calc_drawdown(equity_curve)
ulcer_index = np.sqrt(np.mean(drawdown**2)) * 100
return ulcer_index
def ideal_f_multiple(returns, time_horizon=250, n_curves=1000,
drawdown_limit=20, certainty_level=95, steps=100,
bound_factor=1):
start = time.time()
strategies = returns.columns
potential_weights = weight_combinations(returns, steps, bound_factor)
f_curve = pd.DataFrame()
for i in range(len(potential_weights)):
# Record weights
weights = potential_weights[i]
for j in range(len(strategies)):
f_curve.loc[i, strategies[j]] = weights[j]
weighted_returns = calc_weighted_returns(returns, weights)
# Generate n random equity curves
reordered_returns = np.random.choice(weighted_returns, size=(time_horizon, n_curves))
curves = get_equity_curve(reordered_returns)
curves_df = pd.DataFrame(curves)
# Calculate GHPR and Maximum Drawdown for each equity curve
curves_drawdown = calc_max_drawdown(curves_df)
curves_ghpr = calc_ghpr_eq(curves_df)
# Calculate drawdown at our certainty level
drawdown_percentile = np.percentile(curves_drawdown, certainty_level)
# Calculate median ghpr value
ghpr_median = np.median(curves_ghpr)
if drawdown_percentile <= drawdown_limit:
ghpr = ghpr_median
else:
ghpr = 0
f_curve.loc[i, 'ghpr'] = ghpr * 100
f_curve.loc[i, 'drawdown'] = drawdown_percentile
optimal_f = f_curve.loc[f_curve['ghpr'].idxmax()]
elapsed = time.time() - start
print('Ideal f calculated in {}s'.format(elapsed))
return {'f_curve':f_curve, 'optimal_f':optimal_f}
For this example, we’ll (finally) use some actual market data. We’ll use the fix-yahoo-finance package (shout-out to Ran Aroussi) to grab daily OHLC data for the S&P 500 (SPY) and Long-Term Treasury (TLT) ETFs from 2005 to 2015. This window was arbitrarily chosen, but 10 years seems like a decent time window, and the period captures bullish, bearish, high volatility and low volatility periods. Once we have the price data, we can calculate the daily returns to use in our position sizing optimization. We’ll also visualize how a buy/hold strategy would have performed for each.
import fix_yahoo_finance as yf
start_date = '2005-01-01'
end_date = '2015-01-01'
# Get returns for SPY and TLT
etf_returns = pd.DataFrame()
for symbol in ['SPY', 'TLT']:
df = pdr.get_data_yahoo(symbol, start=start_date,
end=end_date, auto_adjust=True)
etf_returns[symbol] = df['Close'].pct_change()
get_equity_curve(etf_returns).plot()
Next, we’ll iterate through all combinations of allocation weights to find what’s optimal, given our risk tolerance. The settings used here attempt to impose a maximum drawdown limit of 20% with 95% confidence.
f_etfs_dd = ideal_f_multiple(etf_returns, bound_factor=4)
print(f_etfs_dd['optimal_f'])
Output:
Ideal f calculated in 114.47582197189331s
SPY 0.355518
TLT 0.644208
ghpr 0.033799
drawdown 18.683212
The first thing that one will notice about the graphic above is that only a small region of the search space has non-zero GHPR values assigned to it. Outside of this region, we lack confidence that our drawdown constraint will be satisfied.
The optimization indicates that we should allocate approximately 36% of our capital towards SPY and 64% towards TLT. The fact that these weights sums to 100% is merely a coincidence, not a feature/constraint of the model. The predicted median GHPR for this allocation strategy is 0.034% daily (8.9% annualized), with 95% of the generated equity curves having a maximum drawdown <= 18.7%.
Multiple Strategies
This approach is great for two returns streams, but the real benefits of diversification come from having many non-correlated strategies. Unfortunately, a few problems arise when venturing outside of the two-dimensional return space. First, visualization becomes significantly more difficult; kind of a pain, but not a huge deal. More importantly, the time it takes to search through the combinations of portfolio weights grows exponentially.
It took nearly two minutes to find the optimal solution with only two series of returns. To achieve diversification, we want to combine strategies across markets, styles, and timeframes. This can lead to a portfolio with dozens of individual return streams. When you consider the fact that for each combination of weights we’re generating thousands of equity curves to evaluate returns and drawdown, this is a prohibitively time-consuming process. Luckily, we have some time-saving tricks up our sleeves.
In a previous article, I examined a number of popular performance metrics to find a proxy for drawdown-constrained GHPR. The results were that Sharpe Ratio was strongly correlated with our metric, and was not time-dependent. Instead of finding the maximum GHPR, we’ll optimize for the maximum Sharpe Ratio of each weighted returns series. This way, we don’t need to generate thousands of potential equity curves to assess what the optimal combination of weights is. Doing so is an order-of-magnitude time savings.
The second technique to radically reduce the amount of time it takes to find the optimal portfolio allocation is to forego brute-force iteration of possible combinations in favor of non-linear optimization. We’ll use the cma package we installed earlier to accomplish this. I won’t go into the details of this algorithm; for more info, read the documentation here. Given a fitness function and a series of inputs, this algorithm will find a local (and hopefully global) minimum in much less time than it would take to try every possible combination of inputs.
However, using this approach only yields the relative combination of weights to use; leverage isn’t factored in. For this reason, we’ll split the optimization routine into two parts:
- Find the relative allocation weights that maximize the Sharpe Ratio of the portfolio.
- Determine the leverage to apply to the portfolio that maximizes GHPR while respecting given risk/drawdown limits.
To demonstrate this approach, we’ll create three “dummy” strategies on our two ETFs: two mid-to-long term trend-following and one short-term mean-reversion. As a proxy for trend, we’ll use a rolling z-score, defined as follows:
where 
First, we’ll calculate strategy signals with 5, 50, and 100 day lookback periods for SPY and TLT. From these signals, we get our strategy returns.
def rolling_z_score(series, lookback):
mu = series.rolling(lookback).mean()
sigma = series.rolling(lookback).mean()
return (series - mu) / sigma
strategies = pd.DataFrame(index=etf_returns.index)
strategy_returns = pd.DataFrame(index=etf_returns.index)
spy = etfs['SPY']
spy_returns = spy['Open'].pct_change().shift(-2)
strategies['SPY_5'] = rolling_z_score(spy['Close'], 5, 20) * -1
strategies['SPY_50'] = rolling_z_score(spy['Close'], 50, 20)
strategies['SPY_100'] = rolling_z_score(spy['Close'], 100, 20)
tlt = etfs['TLT']
tlt_returns = tlt['Open'].pct_change().shift(-2)
strategies['TLT_5'] = rolling_z_score(tlt['Close'], 5, 20) * -1
strategies['TLT_50'] = rolling_z_score(tlt['Close'], 50, 20)
strategies['TLT_100'] = rolling_z_score(tlt['Close'], 100, 20)
for column in strategies.columns:
symbol = column[:3]
if symbol == 'SPY':
strategy_returns[column] = strategies[column].multiply(spy_returns)
elif symbol == 'TLT':
strategy_returns[column] = strategies[column].multiply(tlt_returns)
Next, we’ll use our optimization routine to find the weights that maximize the sharpe ratio for our portfolio.
def sharpe_ratio(x):
return np.mean(x) / np.std(x)
class Portfolio:
def __init__(self, returns, regularized=False, c=1e-6):
self.returns = returns
self.regularized = regularized
self.c = c
def fitness_function(self, weights):
weighted_returns = calc_weighted_returns(self.returns, weights)
neg_sharpe_ratio = -1 * sharpe_ratio(weighted_returns)
penalty = self.c * np.linalg.norm(weights, ord=1)
if self.regularized:
return neg_sharpe_ratio
else:
return neg_sharpe_ratio + penalty
def optimize_portfolio(self, display=False):
print('Optimizing Portfolio...')
num_strategies = self.returns.shape[1]
initial_guess = [0.5] * num_strategies
optimization_results = cma.fmin(
self.fitness_function, initial_guess, 0.25,
options={'bounds':[0,1], 'verb_disp':display, 'tolfun':0.0001}
)
return optimization_results[0]
portfolio = Portfolio(strategy_returns)
results = portfolio.optimize_portfolio()
optimal_weights = pd.DataFrame(np.round(results, 2), index=strategy_returns.columns, columns=[''])
optimal_weights
Output:
The results of this optimization indicate that the ideal strategy would’ve been to allocate the majority of our capital towards short-term mean-reversion strategies, and limited amounts to the longer-term trend-following strategies.
Step one is complete. We’ve determined the relative weights to apply to each strategy in order to maximize the sharpe ratio, but to apply them as calculated could result in too much risk or too little reward. To address this, we can get the returns for a weighted portfolio using the results of our optimization routine. It’s then possible to apply the original “ideal f” calculations from previous articles (I won’t post the code here) to determine the appropriate amount of leverage for our risk goals.
We calculate that the ideal fraction to stake on this portfolio of strategies is 32%. The final step is to multiply the un-leveraged portfolio weights by this value to arrive at the final portfolio allocations:
Finally, visualize the results of this portfolio’s performance:
Caveats
As with any “optimal” solutions in the realm of trading strategies, there is the opportunity to generate pretty-looking backtests that lead to lack-luster (or disastrous) live-money performance. When implementing the techniques discussed above, it’s helpful to be aware of some of the drawbacks.
First and foremost, the multi-strategy implementation did not account for trading costs. If the strategies in your portfolio are in and out of the market frequently, these can quickly become significant and must be accounted for. Next, the strategy returns used for optimization must be out-of-sample. Real-money returns are best, paper trading returns are a runner-up, and out-of-sample backtest returns can be used if one of the former aren’t available. However, if you optimize a group of strategies, and then optimize a portfolio of these strategies on the same in-sample data, you will reap what you sow. Don’t say you weren’t warned.
The next set of issues deals with the nature of historical returns used to perform these calculations. It’s assumed that the relationships between strategies in the future will resemble those in the past. If these relationships break down, the portfolio could be very badly positioned. As implemented above, the worst loss used in the leverage calculation is based on the weighted returns. A loss in one strategy can be compensated for gains in another during the same period. To account for instances where all of your strategies go against you simultaneously, you can combine the worst loss for each individual strategy historically, multiplied by the portfolio weights.
Finally, we must assume that the future will be less kind than the past. Alpha decay will affect most strategies, and live-trading results typically under-perform even the most rigorous backtests. We should also assume that our largest drawdown is in front of us, as it’s mathematically guaranteed that the longer one trades, the higher the probability of a large drawdown. Ditto for our worst loss. To mitigate this, we can apply a safety factor to inflate the largest historical loss, thereby decreasing the amount of leverage applied. We can also set a maximum drawdown threshold lower than we think necessary; this will also serve to effectively reduce the optimal leverage.
Conclusion
This series has walked through a guide to position sizing that takes into account a practitioner’s most relevant goals: achieving the highest possible return from a trading strategy or portfolio of trading strategies while constraining the maximum risk. We highlighted the time-dependent nature of drawdowns and why many possible equity paths must be accounted for to accurately assess risk according to this metric. Finally, I this post illustrated the benefits of diversification, not only as a way of reducing risk, but also increasing profit. If one can increase their risk-adjusted returns, all that is necessary to increase absolute returns is to increase leverage to suit their risk appetite.
If you enjoyed this article (or didn’t), please leave a comment below! Also feel free to follow on Twitter, connect on Linkedin, and/or join the Quant Talk Telegram chat here: https://t.me/joinchat/GrWzrxH7Z0X_65JD3NLGMw