There is a difference between merely using an optimiser and trading optimally. Many index players would be better off focusing on stratified sampling, rather than worrying about optimisers.
Here we use the term optimiser to mean “an algorithm that seeks the minimum point of active risk subject to given constraints”. The active risk is a function of the weights of all positions in the portfolio and is invariably calculated using a covariance matrix. Optimisers are very sensitive to changes in the covariance matrix. You can get an optimiser to generate quite different trade lists simply by changing the covariance matrix a little bit.
The construction of a covariance matrix is complex. So complex, in fact, that nearly everyone leaves it down to a very few specialist ‘risk model’ providers. They have their proprietary recipes, and all their clients take the (same) data they are given, whether or not the recipe makes sense for the strategy being managed.
Covariance matrix construction can be done using a variety of methods ranging from brute-force to very subtle. You can use short return histories of 60 days or less, or you can use very long histories. You can use an equal weighting of returns, or you can weight recent returns higher than older returns, or you can even ignore some returns. All these variations and more are used in practice, and they all produce different covariance matrices and hence different ‘optimised’ results.
Even if God (or Barra) could hand you the ideal covariance matrix construction method there are still problems.
Covariances (or equivalently, correlations) are not stable. They change over time. A great example from currencies is the CHF vs EUR at the time of the Greek debt crisis. Within a few days, two assets with a 10+ year history of near-perfect correlation developed a significant negative correlation. The reason? An idiosyncratic problem with one asset (the Euro) not replicated by its close substitute (the Swiss Franc) drove many to sell EUR and buy CHF.
Investors should think hard about this. Regarding two assets as substitutes for each other just because of a history of correlation is dangerous. Another such example: Oil and Natural Gas. They were highly correlated until the widespread exploitation of abundant shale gas made the price of the latter decline steadily.
Investors should be fully aware that optimisers take advantage of these observed correlations to produce results. You ask an optimiser to match a 1000 security index with 100 securities, and it will place some big bets on those correlations. When correlations diverge, the user of a long-dated covariance matrix calculated with a long history may be the last to know. If your covariance matrix is based on two-years of daily returns and a correlation that was strong suddenly becomes negative, you will have to wait a year for your covariance matrix to reflect even 50% of the effect of the ‘new regime’.
In practice, we see a lot of skilled investment professionals spending a lot of time on rather mundane or senseless tasks ‘getting the optimiser to work’. These are summed up in these categories:
The last category is particularly sad: if the portfolio manager knows what the right result looks like, what value is the optimiser adding?
Stratified sampling is far easier to understand and implement than covariance-based optimisation. It works by slicing the benchmark and fund up into segments and insisting that invested weight of fund and benchmark match each other, in aggregate, over each segment. This is the ‘stratified sampling rule’. Like optimisation, it is best to consider several factors simultaneously.
For example, in an international equity portfolio, segment the portfolio by
and run an algorithm to bring the portfolio into obeyance of the stratified sampling rule, along each of these dimensions simultaneously.
This type of algorithm produces very satisfactory results in practice, and overcomes many of the objections to optimisation: