Development and testing of a novel measure to assess fidelity of implementation : example of the mini-AFTERc Intervention

Abstract

Background Fidelity of implementation (FOI) reflects whether an intervention was implemented in clinical practice according to the originally developed manual and is a key aspect in understanding intervention effectiveness. To illustrate this process of developing a fidelity measure, this study uses the Mini-AFTERc, a brief psychological intervention aimed at managing breast cancer patients’ fear of cancer recurrence, as an example. Objectives To illustrate the development of an FOI measure through (1) applying this process to the Mini-AFTERc intervention, by including the design of a scoring system and rating criteria; (2) content validating the FOI measure using thematic framework analysis as a qualitative approach; (3) testing consistency of the FOI measure using interrater reliability. Methods The FOI measure was developed, its scoring system modified and the rating criteria defined. Thematic framework analysis was conducted to content validate the FOI measure using nine intervention discussions between four specialist cancer nurses and four breast cancer patients, and one simulated breast cancer patient. Intraclass-correlation was conducted to assess interrater reliability. Results The qualitative findings suggested that the Mini-AFTERc FOI measure has content validity as it was able to measure all five components of the Mini-AFTERc intervention. The interrater reliability suggested a moderate to excellent degree of reliability among three raters, rICC = 0.84, 95% CI [0.51, 0.96]. Conclusion The study has illustrated the steps that an FOI measure can be developed through a systematic approach applied to the Mini-AFTERc intervention. The FOI measure was found to have content validity and was consistently applied, independently, by three researchers familiar with the Mini-AFTERc intervention. Future studies should determine whether similar levels of interrater reliability can be obtained by distributing written and/or video instructions to researchers who are unfamiliar with the FOI measure, using a larger sample. Employing developed and validated FOI measures such as the one presented for the Mini-AFTERc would facilitate implementation of interventions in the FCR field in clinical practice as intended.