“How do you know you know that you're doing ACT?” An experimental study of simulated therapy sessions to explore the measurement properties of the Acceptance and Commitment Therapy Fidelity Measure (ACT-FM)

The Acceptance and Commitment Therapy-Fidelity Measure (ACT-FM) offers a user-friendly assessment of ACT delivery in clinical trials and clinical practice. However, the ACT-FM's measurement properties have not been adequately explored. This study tests whether it performs two key functions: 1) discriminates between therapy sessions delivered with different levels of fidelity to ACT principles; 2) enables reliable scoring. In total, 14 clinicians with experience in ACT were recruited to an online repeated measures experimental study. Using the ACT-FM, they rated three audio clips of simulated therapy sessions with differing levels of ACT fidelity (high, mixed, and low). Audio clips were otherwise identical in terms of clinician-patient dyad, problem (anxiety), and length (15 min). Analyses indicated that ACT-FM scores for Total ACT Consistency and Total ACT Inconsistency domains differed significantly between audio clips and in the expected directions. Intraclass correlation coefficients indicated moderate inter-rater reliability on both domains and there was 80 % agreement among raters at the item level. However, wide confidence intervals suggest imprecision in reliability estimates. These findings suggest that the ACT-FM effectively discriminates between ACT delivered with various levels of fidelity. Yet, further study of interrater reliability is required.