Hurricane Katrina’s floating casinos were an expensive lesson in how poor data quality can cost insurers millions of dollars. Two years on, Matthew Grant looks at what has improved.

By 2004, there remained little doubt in the insurance and reinsurance industry about the need for catastrophe models. For many, models had become “reassuringly complicated” and a critical part of underwriting and reinsurance placement. So when seven major hurricanes rolled into the US between 2004 and 2005 there was widespread surprise at the meaningful differences between actual and modelled losses.

Initial reaction was to blame the models. But when companies compared the information entered into them with what they had actually been insuring, a different story emerged. Descriptions of insured locations were often incomplete and, at times, wrong. The most extreme, and widely-cited, example was the destruction of the floating casinos of Louisiana causing losses many millions of dollars greater than had been expected. Much of this was attributed to business interruption. These massive barges proved highly vulnerable to storm surge, but had been incorrectly identified in the models. Less dramatically, but even more significant, were the thousands of properties located close to the sea, but identified only by their area zip code.

Standard and Poor’s was quick to identify how reinsurers had been impacted by this poor data, commenting shortly after Hurricane Katrina that: “Data quality is an area where underwriting and modelling can highlight differing views and results… Catastrophe losses may vary significantly from modelled results if assumptions related to these portfolio assumptions are different from the true exposure detail… Reinsurers whose adjustments to the models more accurately captured the true quality of the portfolio being reinsured outperformed competitors in 2004…”

Today, data quality is embedded in the philosophies of enterprise risk management, and is of interest to all rating agencies. AM Best places the most emphasis on data quality when rating an organisation’s catastrophe risk management approach. Best’s supplementary rating questionnaire includes specific questions about data vintage and geocoding resolution and it has also provided useful recommendations on good data management.

Identifying poor data

“Missing the "floating casinos" can seriously damage an insurer's health

Blaming poor data may seem a convenient scapegoat when a catastrophe model fails to exactly represent real events, but changes to loss estimates of four times on a single building (see graph) or 25% across a whole portfolio are not uncommon when data quality is improved. Catastrophe models are complex and deal mainly in probabilities. There is never a completely “right answer”. But data is more black and white; you either know precisely where the insured building is located or you don’t. Somebody, somewhere, knows how big it is, what it is made of and when it was built.

However, just because we know what quality data looks like, doesn’t mean it is easily available. Many risk managers are still somewhat ambivalent about their own data. They look to their broker for advice on what is required for insurance placement and are concerned that providing better data may result in higher insurance costs. There is confusion over data standards and often little understanding of how catastrophe models are used during underwriting.

Missing information is relatively easy to spot; information that appears complete, but is wrong, is more challenging. Should a chain of retail outlets with 100 locations all coded with the same building date, construction and size be taken as accurate data or the result of inappropriate use of Excel? Humans are very good at using intuition to spot anomalies in patterns and can catch some of these errors, but accurately reviewing extensive insurance schedules is unrealistic. And even if problems are identified, a decision needs to be made on how to correct the error. For example, should an implausible 11 storey wood framed building actually be entered as a one storey wood construction, or an 11 storey steel frame building? Should a 150 bedroom residential building be coded as a five bedroom property, or is it actually one of Her Majesty’s prisons? Missing the “floating casinos” can seriously damage an insurer’s health.

Mapping software and aggregation tools go some way to resolving these problems, but still rely on a lot of manual intervention. Reliably assessing the quality of a portfolio and having the confidence that all significant errors have been corrected is tough. Incentives for data entry typically focus on getting the job done, rather than getting it done correctly.

Tools for improvement

“These massive barges proved highly vulnerable to storm surge, but had been incorrectly identified in the models

So data may be black and white, but assessing it for completeness, resolution and accuracy still requires some form of modelling. Software can be taught how to recognise and then deal with oddities in a systematic and auditable way using predefined rules, or heuristics, that replicate expert intuition. Designing good heuristics is not easy. They need to be smart enough to decide when data is clearly wrong, and how it should be changed, but retain the ability to ask for human intervention where there is ambiguity.

The best heuristics are able to learn from experience, creating processes that get increasingly automated, reliable and accurate with use. Combining modelled heuristics with metrics that monitor data resolution and completeness allows a company to understand its progress in improving data quality over time and across business units. Once measured, quality can be managed.

Ironically, it was probably easier for insurers to get information about properties they were insuring 200 years ago than it is today. From 1867 to 1970 Sanborn Fire Insurance Maps documented the rise of American cities with building level detail and colour-coded construction classes. But as labour became more expensive and cities experienced rapid growth, developing such detailed maps became less viable. It is only in recent years, as the cost of computer storage has reduced and performance has increased, that it is again possible to provide similar levels of detailed information for the insurance market.

Having lost the habit of collecting good data, the industry needs to relearn how to identify and store the information required to make educated decisions. Legacy systems that are not designed to capture the detail required need to be upgraded or replaced. Incentives should be provided to record and share the data.

Even the best heuristics need help. An independent view of the building construction, occupancy type and valuation provides additional information to complement or even replace the original data. Today there is no one comprehensive source of commercial property data in the US for insurers. In contrast, the construction and real estate investment industry has access to robust databases that contain some of the key information needed by underwriters. There are companies, RMS included, converting this information into a form that insurers can use. Coupled with aerial photographs and building surveys, insurers will once again have an independent view to help them understand what they are underwriting, where it is, and what it is worth.

“Ironically, it was probably easier for insurers to get information about properties they were insuring 200 years ago than it is today

To be successful, both the databases and heuristics need to be accurate and timely. Companies that are assessing data quality well are using systems integrated into the underwriting process, so there is no meaningful increase in analysis time.

Learning from the past, reinsurers too are starting to take account of data quality in the selection of portfolios they write, and will penalise or even refuse to write business that doesn’t meet their standards.

Getting it right

All catastrophe events provide new insights for cat models, and Katrina and the other hurricanes of 2004 and 2005 were no exception. The models have long since been updated to reflect these lessons, but the problem of poor quality data still affects the industry and remains one of the biggest barriers to accurate loss assessment, portfolio management and underwriting.

With rates starting to slip after another year of relatively quiet hurricane activity, there are worrying signs that risks are once again being written in some markets with insufficient attention to location, construction type and size. Ultimately, the actions of insurers and reinsurers themselves, and not the technical solution, will determine how well prepared the industry is for the next major event. Catastrophe models may not be perfect, but going back to basics and getting the data right is essential for maximising their use. As Occam’s razor notes, “All things being equal, the simplest solution tends to be the right one.”