The variation in estimated losses following Hurricane Katrina exposed the fallibility of catastrophe models, argues Ronn Mullins

When Hurricane Katrina tore across Florida on 25 August 2005, then roared into the Gulf of Mexico, smashing into the Gulf Coast near New Orleans on 29 August, the three leading catastrophe modelling companies - EQECAT, AIR Worldwide and Risk Management Services (RMS) - had already programmed their computers to probe the trillions of data sets to calculate the estimated losses from that catastrophe.

On 29 August 2005, before Katrina made landfall, EQECAT released its preliminary estimate of insured losses ranging from $15bn to more than $30bn. A day later, EQECAT lowered its estimate range from $9bn to $16bn. Excluded from its estimates was potential insured damage to oil rigs and platforms in the Gulf. On 2 September, EQECAT raised its final estimate of insured losses from wind damage to between $14bn and $22bn. On 22 September, it estimated that losses to the offshore oil and gas industry, private automobiles and marine insurance and to commercial property insured for flood damage would add between $11bn to $19bn, bringing EQECAT's total estimated losses from Katrina to between $25bn and $41bn.

AIR's estimate on 30 August of $17bn to $25bn, included wind and water damage to onshore residential, commercial and auto policies. The estimate did not include the demand surge, indirect business interruption or damage to offshore assets. It said 10% of the total flood loss would be covered by private insurance. On 29 September, AIR released its final scenario of insured losses at $26bn. Adding a 30% demand surge, brought the estimated insured loss to $34bn. The firm added another $10bn for insured and uninsured flood and storm surge, raising its total to $44bn.

On 2 September, RMS's preliminary insured loss estimate ranged from $20bn to $35bn (excluding flood losses in New Orleans), with total economic losses of more than $100bn. Seven days later, the modeller estimated losses to be from $40bn to $60bn; $15bn to $25bn was related to the Great New Orleans Flood. Total economic losses were expected to exceed $125bn. RMS considered a demand surge of at least 20%.

ISO's Property Claim Service (PCS) unit on 4 October issued an initial estimate of $34.4bn for insured property losses for Hurricane Katrina, making it the costliest US catastrophe ever. PCS's estimate represented anticipated insured losses for personal and commercial property lines of insurance covering fixed property, personal property, vehicles, boats, related property items, business interruption and additional living expenses. Not included were losses to utilities, agriculture, aircraft, ocean marine (including oil drilling platforms) and property insured under the federal flood insurance programme. PCS said it would resurvey re/insurers in 60 days as more claims were filed.

Since PCS bases its estimate on reported and incurred but not reported claims from insurance and reinsurance companies, the PCS figure is considered by the industry to be closer to what the expected losses will be from a catastrophe. Gary Kerney, assistant vice president, PCS, said its estimate is the "best approximation based on what was known." He added that "a whole lot of things are going on that can change the total cost in an instant" and mentioned several lawsuits challenging coverage for flood damage, degree of demand surge, political pressures and regulatory actions.

Estimates variation

That catastrophe modelling remains an inexact science can be seen in the strong differences in the three modelling agencies' estimates of the losses from Hurricane Katrina. Most of the variation can be explained by the diverse variables that the three modellers included in their estimates, but the explosive nature of Hurricane Katrina and the flooding caused by the breach in the levees in New Orleans made a more accurate estimate difficult, if not impossible, because the modellers had never collected data of such enveloping ruin and had no benchmark to compare the losses to.

Steve Lowe, global P/C practice leader for Tillinghast, Towers Perrin, said Hurricane Katrina is expected to result in $40bn to $55bn in private insurance payments. He discounts the PCS estimate as it does not include losses for marine lines, energy, business that is written directly in non-US insurers, into the London and Bermuda markets, or placed with captives of US companies.

"Our estimate includes insured losses borne by the global re/insurance markets," Lowe said. "We have specific provisions for marine business and liability claims for lawsuits that may be brought not only to adjudicate coverages, but those that may be lodged against agents, nursing homes and others."

Kerney said that PCS doesn't assert that every claim and dollar paid is counted in its estimates, but said PCS provides explanations for markets that it doesn't report on. "I think that we are talking oranges and apples," he added as explanation for the disparity between figures of modellers, PCS and others.

Who's first?

That EQECAT's first estimate of the insured losses from Katrina preceded the hurricane making landfall on the Gulf Coast may be a sign of the competitive nature among the three main modellers and the race to be first with a loss estimate. Thomas Larsen, senior vice president, EQECAT, said, "I don't think we are trying to be first, but there is the job of coming up with the answers to reduce the uncertainty and clarify things for the public and insurers. Are we modellers competing or are we trying to help our clients? It is a little of both. Are we out there just to get our name in the press? No, it is that we have the information and want to get it out to the public as soon as possible."

AIR estimates losses after it has thoroughly analysed all of the available data and processed it in the hurricane model, explained Karen Clark, president and CEO of AIR Worldwide Corporation. "Since there is so much uncertainty in the data just after landfall," she said, "AIR usually analyses the data with multiple assumptions numerous times before deciding on a loss estimate. This can take time and is done without consideration of when other modellers will issue loss estimates."

Dr Robert Muir-Wood, chief research officer at RMS, dismissed the very idea of being in competition with other modelling agencies. "We don't really watch what our competitors are doing. We have the largest market share, and we are driven by our clients, and they are most interested in knowing what we are saying."

Strides ahead

So what do those in the industry who use the cat models think? Samira Barakat, chief risk officer, GE Insurance Solutions, explains "it is very important that models are used to determine losses from disasters. We buy models from three or four modelling companies and we tweak all of them a bit depending on which of our products are being studied. When we have an event, the modelling companies start sending us storm tracks, which we run to obtain our latest exposures. Once we get a high-level estimate of our losses, then we talk to our underwriters and we validate our losses." GE Insurance Solutions said its net loss estimate from Hurricane Katrina was $298m after tax. But Barakat admits she doesn't solely depend on cat models. "We have to work in other factors to get a true estimate," she said. "We are slower than our colleagues in getting out our numbers because we don't want to have to come back with revisions."

"Cat models can be improved a lot," Barakat said, "but they are way, way better than five years ago, but we want to get better." Some areas she mentioned as needing improvement are flood modelling, demand surge, accuracy of building codes and cluster modelling. What effect do four hurricanes coming close together as in Florida last year, or Hurricanes Katrina and Rita in 2005, have on losses and demand surge?

"Insurance companies need to reassess how they use models," said Lowe of Tillinghast. "There is a risk that we refer to as the 'model risk', which is the risk that the model doesn't capture correctly. Insurers and reinsurers need to be conservative in the interpretation of the cat model output, rather than taking their results at face value."

AIR's Clark believes the top priority for the industry should be a thorough review of the data being entered into the models. "Location, replacement value and construction information are key," she said. "As a result of Katrina, AIR will be putting more emphasis on storm surge and flood models and continue to study demand surge. The catastrophe modelling process requires commitment from both the insurance company and the modelling company," she added.

Katrina has reset the agenda for modelling companies on many topics," admits Muir-Wood. "To more accurately predict losses in the future, RMS is working on a major research programme to examine $14bn of claims from 2004." Larsen said EQECAT is learning new things all the time. "We believe our estimates for Katrina were credible in the areas that we modelled. Our job is to eliminate surprises, and we do try to address that."

- Ronald Gift Mullins is a freelance journalist based in New York City.