The Model They Did Not Use: How a Commercial Insurer Used Market Research and IDIs to Understand Why Underwriters Were Ignoring Its Predictive Risk Scoring Output

Predictive risk scoring models in commercial insurance underwriting face an adoption challenge that is distinct from the performance validation challenge the analytics team typically focuses on. A model can demonstrate strong discrimination between high and low loss ratio outcomes in backtesting and still achieve near-zero adoption in daily practice if the underwriters who are supposed to use it cannot understand why the model is generating the score it is, cannot explain it to a broker or a manager when challenged, and do not trust an output that they cannot interrogate in the way they can interrogate their own reasoning.

The client’s analytics team had built and validated a predictive risk scoring model that performed well against historical loss data and had been deployed to underwriters across the commercial property book. The model generated a score for every submission, displayed in the underwriting workstation alongside the conventional risk information.

The analytics leadership’s assumption had been that a technically validated model would be adopted naturally once underwriters saw it in their workflow. The two years of non-adoption data suggested that assumption was wrong, but the analytics team had no structured understanding of why.

Key Challenges

• No independent benchmarking of predictive model adoption dynamics at comparable commercial lines carriers to contextualise the client’s own adoption failure.
• No direct feedback from underwriters about the specific reasons they were not incorporating the model output into their pricing decisions.
• Two years of non-adoption attributed to cultural resistance without investigating whether the model’s interface and output design were contributing factors’s interface and output presentation were contributing factors.
• A technically validated model producing near-zero operational impact because underwriters were not incorporating its output into pricing decisions.
• No understanding of what information underwriters would need alongside the model score to be willing to let it influence their pricing decision.
• Analytics leadership pressure to demonstrate practical adoption return on the model development investment before the next budget cycle.

A predictive risk scoring model that underwriters do not trust is not producing the risk differentiation it was built to deliver, regardless of how well it performs in backtesting. Underwriter trust in a model output depends less on the model’s statistical performance than on the underwriter’s ability to interrogate the output, verify it against their own risk knowledge, and explain it to a broker when challenged.