Decision-oriented Optimal Experimental Design and Data Collection

Abstract:

Classical optimal experimental design is a branch of statistics that seeks to construct ("design") a data collection effort ("experiment") that minimizes ("optimal") the uncertainty associated with some quantity of interest. In many real world problems, we are interested in these quantities to help us make a decision. Minimizing the uncertainty associated with the quantity can help inform the decision, but a more holistic approach is possible where the experiment is designed to maximize the information that it provides to the decision-making process. The difference is subtle, but it amounts to focusing on the end-goal (the decision) rather than an intermediary (the quantity). We describe one approach to decision-oriented optimal experimental design that utilizes Bayesian-Information-Gap Decision Theory which combines probabilistic and non-probabilistic methods for uncertainty quantification. In this approach, experimental designs that have a high probability of altering the decision are deemed worthwhile. On the other hand, experimental designs that have little chance or no of altering the decision need not be performed.