Understanding variations in the outcome of a smoking cessation programme in Tuberculosis patients in Pakistan - the role of fidelity?

I conducted a review to show that smoking cessation in TB patients can have positive TB outcomes. Behavioural interventions (BI) for smoking cessation are known to be effective and cost effective but there are important variations in quit rates that are hard to explain. My aim was to develop and validate a fidelity index for BIs in smoking cessation, and use it to explore the extent to which variations in smoking quit outcomes reflect the degree to which providers implement the various components and the quality of delivery of a smoking cessation programme in TB treatment settings.
I developed and tested a theoretically mapped fidelity index comprising two sub-indices: ‘Adherence’ for measuring compliance (37 items) and ‘Quality’ for measuring provider competence (8 items). Items were rated as fully, partially, or not, implemented against a behaviourally anchored response scale. Fidelity was measured in a prospective study of 18 providers in TB clinics in Pakistan (154 patients) whose sessions were audio-recorded and then coded. These providers had participated in delivering the same BIs as part of the ASSIST trial four years earlier.
Reliability was assessed in three ways. There was good inter-coder reliability using Kripendorff’s alpha, Principal Components Analysis showed the items of the index were coherent in measuring fidelity and Generalisability theory showed that the index reliably differentiated between providers by capturing the variation in their BI delivery practice.
Provider Adherence and Quality of provision were positively correlated. I tested the assumption that relative provider practice was consistent between the ASSIST trial in 2010 and the fidelity study in 2014. Using Kendall’s W coefficient of concordance on data from a self-recorded checklist used in both studies, I found moderate to strong concordance. I then used binomial regression analysis to estimate the relationship between fidelity and ASSIST trial provider-level quit rates. This showed that the provider-level quit rate was positively associated with Quality of interaction (odds ratio: 2.15; 95% CI, 1.43 to 3.24) but negatively associated with Adherence to BI content (odds ratio: 0.55; 95% CI, 0.40 to 0.77). A negative interaction was found between Adherence and Quality and quit rates.
This research makes several contributions to the field. We have a better understanding of the impact of smoking on TB. I developed and validated a new, theoretically-informed, fidelity index which can be used to quantify and score delivery of BI ingredients in a standardised way and used to better understand how BIs influence outcomes. I report that the quality of delivery of BIs maybe as important as content in influencing smoking cessation.