Using Experts' Beliefs to Inform Public Policy: Capturing and Using the Views of Many

Cost-effectiveness decision modelling (CEDM) is widely used to inform healthcare resource allocation, however there is often a paucity of data to quantify the level of uncertainty around model parameters. Expert elicitation has been proposed as a method for quantifying uncertainty when other sources of evidence are not available.
Elicitation refers to formal processes for quantifying experts’ beliefs about uncertain quantities, typically as probability distributions. It is generally conducted with multiple experts to minimise bias and ensure representation of experts with different perspectives.
In CEDM, priors are most commonly elicited from individual experts then pooled mathematically into an aggregate prior that is subsequently used in the model. When pooling priors mathematically, the investigator must decide whether to weight all experts equally or assume that some experts in the sample should be given ‘more say’. The choice of method for deriving weights for experts’ priors can affect the resulting estimates of uncertainty, yet it is not clear which method is optimal.
This thesis develops an understanding of the methods for deriving weights in opinion pooling.
A literature review is first conducted to identify the existing methods for deriving weights. Differences between the identified methods are then analysed and discussed in terms of how they affect the role of each method in elicitation.
The developed principles are then applied in a case study, where experts’ priors on the effectiveness of a health intervention are elicited, and used to characterise parametric uncertainty in a CEDM. The findings are used to analyse and compare different methods for weighting priors, and to observe the consequences of using different methods in the decision model.
The findings improve the understanding of how different weighting methods capture experts’ ‘contributions’ while the choice of methods for deriving weights is found to influence the decision generated by the model.