Override

Our research contradicts the common belief that people prefer understandable systems. Decision-makers sometimes override simpler systems due to what we term “overconfident troubleshooting.” They believe they grasp a model’s reasoning better than they do, relying on experience to compensate for perceived model flaws. Individuals often create stories about interpretable algorithms to justify ignoring their guidance. For instance, one allocator adjusted suggested product quantities, attributing a sales spike to the NFL draft, despite uncertainty about its actual contribution. Nonetheless, the allocator was confident in this explanation.