Euromoney’s risk survey stands up to scrutiny

In his extensive new book, Country Risk – The Bane of Foreign Investors, independent risk expert Norbert Gaillard of NG Consulting assesses the performance accuracy of Euromoney’s survey-based sovereign risk indicator ratings, along with those produced by Moody’s, Standard & Poor’s and Institutional Investor.

Gaillard looks at three periods of “the globalization era”: first, the wave of defaults in less developed countries during 1982–1984; second, the eurozone crisis during 2009–2013; and third, the “sovereign bond years” during 1995–2013.

Gaillard explains the selection of the first two periods is underpinned by the surge in total public debt in default, while the third encompasses the most recent and active years on bond markets.

This starts in 1995, he says, because for the first time since the interwar years the sovereign rating coverage by the two credit rating agencies exceeded 50 issuers, of which more than half were non-industrialized countries.

How does Euromoney stack up?

The ECR survey is certainly not infallible, and nor does it profess to be.

Unlike other means of gauging country risk, however, it attempts to ‘crowd source’ by collating the views of many experts. Academic studies suggest the consensus approach, such as in economic forecasting, tends to show a greater degree of accuracy than the individual views they comprise.

It should also show more rapidly changes in opinion, given what has been largely understood to be a slower response function from the credit rating agencies, especially Moody’s. This was borne out during the eurozone crisis.

Euromoney fares perhaps less well for the 1995-2013 period, notably with respect to Argentina and Antigua and Barbuda. This is partially down to the methodology employed (which is periodically reviewed) and the fact that Euromoney’s risk scoring does not strictly assess sovereign borrower risk per se, but country risk.

The distinction is non-trivial.

As Gaillard states: “The Euromoney Country Risk survey evaluates several aspects of a country’s investment risk: the risk of default on a bond, the risk of losing direct investment, the risk to global business relations, and so forth.

“This particular feature should be borne in mind when comparing and analyzing the performance of the various raters.”

1982-84 debt crisis

The book offers the first ever analysis of the performance of sovereign risk indicators during the debt crisis of the early 1980s. Moody’s and S&P were not producing sufficient sovereign debt ratings at this time, so Gaillard focuses solely on the performance of Euromoney and Institutional Investor in this timeframe.

He notes that when using a three-year horizon to judge the accuracy of the respective risk ratings, Euromoney’s scoring approach underpredicted the risks for Mexico, while Institutional Investor misjudged Venezuela. Notably, though, the latter had more countries with “excessively high ratings” than Euromoney.

The top third of Euromoney classifications were default-free, suggesting there were no major mistakes ranking countries. Moreover, the majority of defaulting countries were assigned low or exceptionally low scores.

Gaillard also calculates cumulative accuracy profile curves, from which he computes accuracy ratios (ARs) that assess whether low ratings (scores in Euromoney’s case) are assigned to issuers that default and high ratings to issuers that do not.

The analysis reveals three-year ARs (ranging from -1 to +1, where +1 is the maximum accuracy) for Institutional Investor and Euromoney, which are 0.347 and 0.454, respectively.

This shows that not only are the ARs greater than zero –showing the ratings can distinguish between good or bad debtors – but also that Euromoney was more accurate than Institutional Investor for the period in question.

2009-2013 eurozone crisis

As for the eurozone crisis of 2009-2013, the study includes Moody’s and S&P and it ascertains how all four rating sources anticipated the five bailout packages for Greece (2010 and again in 2012), Ireland (2010), Portugal (2011), Spain (2012) and Cyprus (2013).

Once again, Euromoney fares well.

The three-year ARs for Euromoney and Institutional Investor are higher than both Moody’s and S&P for the 2009-2012 and 2010-2013 periods (from September to September).

Whereas Euromoney was slower to downgrade Greece, it outperformed both Moody’s and S&P with respect to Ireland and Spain.

Gaillard makes two important points endorsing Euromoney’s survey for this period. The first is that Moody’s and S&P realized the credit position of the two countries was weakening but were reluctant to lower their ratings.

This slower response has been discussed previously by Euromoney and is something Gaillard himself has also remarked upon.

The two credit rating agencies “feared multi-notch downgrades would trigger massive sales of sovereign bonds and thereby exacerbate the debt crisis”, Gaillard says.

“That concern stemmed from the institutional overreliance on CRA ratings through their incorporation into regulatory rules and investors’ ‘prudential’ rules.”

A second point is that Euromoney and Institutional Investor benefited from “the greater granularity of their rating scales”. For example, as risk aversion increased in 2007-2008, they assigned Germany a rating higher than Spain and Ireland.

Gaillard suggests the 20-notch rating scale used by Moody’s and S&P might have been too narrow to fully distinguish between different eurozone members such as Germany and Spain.

While also providing a comprehensive and detailed explanation of the subject, and its historical development, the lengthy tome provides a useful critique of the various approaches to quantifying investor risk.

It notes, for instance, the drawback of incorporating debt indicators, which are lagging, and the benefit of giving more attention to indicators of speculative behaviour such as stock exchange and real estate indices, private sector credit to GDP, or bank assets to GDP, which all of the various risk measures fail to do.

It is something that has not gone unnoticed. Recently, Euromoney has sought to place greater weight on the political and economic risk scoring of its contributing experts, as it is felt that those changes can provide valuable early warning signs.

The views of those experts are also invaluable and are regularly featured to provide investors with expert insight.

Then there are the subtle changes in scores from quarter to quarter, that can provide useful signalling which credit rating agencies tend to avoid until ratings actions are announced.

Gaillard’s analysis, welcome as it is, does carry some warnings. It is not clear, for instance, whether the three-year period of analysis is the most suitable one, nor is it clear that the study – partly for the reasons mentioned – is comparing like with like.

Still, Euromoney Country Risk is listening, and constantly refining. Encouragingly, too, it would appear to have had an extremely useful predictive quality. Long may it continue.