Cancer Clinical Trials: Identifying & Testing Approaches to Impact Evaluation

Most cancer research is performed with the aim of generating new knowledge that leads to benefits such as improved treatments, higher cure rates, and better cancer prevention. Evaluating these downstream effects of research, often referred to as research impact, is of increasing importance to all cancer research stakeholders. There is currently no consensus surrounding the optimal way to approach this evaluation. The work in this thesis aimed to address this gap by exploring existing approaches to research impact assessment, to identify which approaches have been applied previously for cancer research, and to test a number of these approaches within the context of a case study of one cancer clinical trial. The Short Course Oncology Treatment (SCOT) trial was chosen for the purposes of the case study.

A systematic literature review identified methods, frameworks, and categories of impact frequently used to perform research impact assessment. The methods included surveys, interviews, documentary analysis, and economic assessments. Fourteen empirical examples of impact assessment specific to cancer research were identified; demonstrating the type of methods used previously to evaluate the impact specifically within this field. Focusing more closely on previous impact assessments relevant to clinical trials, the research impact case studies submitted to the United Kingdom government’s research performance exercise for universities in 2014 were screened to find examples of assessments of the impact of cancer clinical trials specifically. Many of these trials were phase III trials that met their primary endpoint, but earlier phase trials and those with negative findings were also impactful. Policy impact was the most commonly described downstream effect. There was a gap within these case studies in the use of real world evidence to demonstrate the impact of cancer trials on clinical practices and health.

Using the lessons learned in the first part of the thesis, the aforementioned surveys, economic evaluation methods and an interrogation of real world prescribing data were tested as methods to assess the impact of the SCOT trial. These analyses demonstrated a high level of awareness of SCOT trial results amongst clinicians and impact of this trial on clinician attitudes and self-reported practice. Important differences in prescribing practices according to patient age and disease stage emerged. Using this information on practice change, it was estimated that implementation of SCOT trial findings could translate to over $150 million USD savings over five years for the healthcare systems from which SCOT trial patients were recruited. Adding the monetised quality adjusted life-year (QALY) gains from implementation to this calculation meant that the gains from implementation of SCOT were vastly in excess of the original investment to conduct the SCOT trial. Lastly, analysis of individual patient level chemotherapy prescribing data at a local (one health board in Scotland) level showed a significant change in the duration of adjuvant treatment prescribed for patients with colorectal cancer after the SCOT trial results were disseminated. After establishing, for the first time, linkage of chemotherapy prescribing data at a pan-Scottish level, a significant change in practice was also identified at a national level.

Evaluating the impact of the SCOT trial as part of this thesis demonstrated its rapid and significant effects on new knowledge, future research, policy, clinical practice, health, and monetary savings for the health service. This assessment also allowed reflection on the pathway to these impacts occurring, as well as on how impacts could be maximised for other trials. Future research to understand which impacts from clinical trials are meaningful to clinical trials units, policy makers, and patients would be useful. More investment by funders and governments to support access to healthcare datasets that can be used to assess clinical practice change in response to trials would make impact assessments more straightforward in future.

Cancer Research UK (CRUK) [C61974/A24293]. Data was conducted at Glasgow University. Data rests in Glasgow data repository. See DOI for further particulars.