Introduction
Non-inferiority (NI) trials are becoming more frequently used in UK publicly funded trials, often aimed at providing evidence for a new treatment to replace an existing one in the NHS. The new treatment must be non-inferior on the primary efficacy outcome as well as have other non-efficacy benefits, such as an improved safety profile. A major design component of these is the non-inferiority margin (NIM) which is used determine non-inferiority and should represent an acceptably unimportant difference. The chosen NIM has a major impact on the design and subsequent outcome of the trial. Setting the NIM, however, can be difficult and has been shown in a review to be often subjective and inconsistent. Additionally, the NIM is poorly justified, creating issues of transparency and interpretability of the design and results of the trial.
Further complexity arises when a potential increase in the NIM, i.e. a higher level of acceptable inferiority, is considered. This would only be possible due to the trade-off with the non-efficacy benefit of the treatment. Benefit-risk methods, which aim to create transparency and consistency when trading-off between multiple outcomes, could be implemented here to aid with the adjustment.
This thesis aims to assess whether benefit-risk methods can support setting and adjusting the NIM to improve the decision-making behind selecting a NIM and transparency in the justification subsequently provided.
Methods
This thesis answers the overall aim through five main objectives completed via seven research methods as follows:
1. What are the areas for improvement when setting the NIM?
A review of the NI literature coupled with a review of publicly funded NI trials showed the areas for improvement when designing NI trials compared with the methodological literature.
2. What is the potential solution?
The literature on benefit-risk methods was evaluated to assess whether the concepts map to the needs of NI trials. Following this, a formal scoping review was completed to consider where benefit-risk has been used in conjunction with NI trials.
3. Which are potentially suitable benefit-risk methods?
A two-round e-Delphi survey (n=28) was conducted to identify appropriate characteristics a benefit-risk method should have to be considered useful. These characteristics were then used to rate available benefit-risk methods (n=45), resulting in a ranked list based on their potential usefulness.
4. How can the methods be practically applied?
The four highest ranked benefit-risk methods were applied across two real-life case studies to demonstrate their potential. Results were presented to other stakeholders (n=6) during semi-structured interviews where they highlighted any barriers to use.
5. Does the benefit-risk method aid setting the NIM?
Methods were assessed through application to the case studies and interviews to evaluate their usefulness to both aiding the choice of appropriate NIM, as well as creating the justification for the NIM chosen.
Results
This thesis has shown that publicly funded NI trials are falling short of recommendations when setting and reporting the NIM. The properties of benefit-risk methods map to the needs of NI trials and a scoping review identified numerous cases of the concept of benefit-risk being discussed in NI trials – although rarely to aid with choosing the NIM.
Benefit-risk methods were ranked by seven criteria which reflected the principles underpinning the methods (transparency, interpretability and robustness) as well as practicality of the methods (ability to consider different outcome types, include uncertainty and be implemented in software if required). The top four ranked methods were the Benefit-Risk Action Team (BRAT) framework, the Unified Methodologies for Benefit-Risk Assessment (UMBRA) framework, Multi-Criteria Decision Analysis (MCDA), and the Food and Drug Administration’s (FDA) Benefit-Risk Framework (BRF). Frameworks were particularly highly ranked methods as were narrative-based (qualitative) methods although these might include quantitative information.
On reflection, the FDA BRF was the most straight-forward to implement and could be used by all trial teams as a starting point to be more explicit about the considerations. MCDA was the most complex and quantitative but is the only one to output with a value suggestion that can be used for the NIM. This approach raises concerns about obscuring information used in decision-making; however, trial teams found it beneficial, as they found selecting a value to be arbitrary. The BRAT framework presented the most amount of information and was considered the most useful in situations where quality data is available. Members of the trial teams thought that all methods would assist with the justification produced for the NIM.
Discussion
Benefit-risk methods can help provide structure and transparency for the decision making of the NIM when considering a trade-off with the non-efficacy benefits. The use of formal methods provides a level of robustness and consistency that will help to improve the design of NI trials. They additionally support an improved justification which will be more understandable to all stakeholders, including patients. This thesis has demonstrated a proof of concept for using benefit-risk methods to select the NIM which should be further tested to enhance implementation.
