MEng Thesis: Identifying Heterogeneity in Decision-Making

Abstract

Individuals often make different decisions when faced with the same problem. This variation arises naturally in decision-making processes due to differences in agent specific factors, such as training, personal preference, and opinion. In many real-world scenarios, understanding where this variation occurs is critical to inform research and policy-making. For instance, in a clinical context, it is important to identify the types of patients on which the treatment they receive depends greatly on which doctor they visit. Identifying such regions of disagreement can reveal gaps in knowledge or opportunities where best practices can be clarified. In the medical context, it can directly assist the development of more comprehensive treatment guidelines, or suggest hypotheses to be tested in medical trials. In this thesis, we present algorithmic methods to identify heterogeneity in decision-making, by characterizing the regions of disagreement where variation can be attributed largely to the decision-maker. These methods range from approximate methods to exact solutions. We provide generalization bounds where possible and test each method’s performance and computational efficiency using a comprehensive set of synthetic experiments. To demonstrate how these algorithms can be used to obtain insights in clinical decision-making, we present an extensive case study on decision-making for first-line diabetes patients, using an observational dataset from a large insurance provider. We identify subpopulations of patients where this first-line decision varies by provider, and evaluate the effect of this variation on downstream outcomes. Our algorithms are implemented in an easily-usable way and are available to the public.