Validating a theory of planned behavior questionnaire for assessing changes in professional behaviors of medical students

Introduction: Teaching professionalism is a fundamental aspect of medical undergraduate education, delivering important domains of professional attitudes, ethics, and behaviors. The effects of educational interventions can be assessed by measuring the change in such domains, but validated assessment tools for these professionalism domains are lacking. In this study, we constructed and conducted expert validation of a modified theory of planned behavior (TPB) questionnaire to assess changes in professional behaviors (PBs) in medical students. 

Methods: To validate that, we modified an existing TPB questionnaire, and an 18-item questionnaire was subjected to expert panel evaluation using the content validation method. The clarity and relevance of items were assessed using a four-point rating scale (i.e., 1 = not relevant to 4 = highly relevant). Ratings of experts and free-text comments were analyzed. Quantitative evaluation of relevance and clarity was undertaken through analyses of the Item-level Content Validity Index (I-CVI) and Scale-level Content Validity Index (S-CVI). A qualitative assessment of the comments of experts was conducted to refine items, any disagreements were discussed, and a consensus decision was developed among authors for item changes. 

Results: Quantitative evaluation of the Item-level Content Validity Index (I-CVI) scored 0.9–1 for relevance and 0.7–1 for clarity. Qualitative evaluation resulted in (i) changes to the wording of items (e.g., choices such as “worthless/worthwhile” were replaced with “not important/important”); and (ii) suggestion of the addition of social media in the construct of subjective norms. 

Discussion: The proposed tool exhibits content validity and can assess TPB constructs in professionalism education. This study of content validity may help to ensure the modified TPB questionnaire accurately measures the TPB constructs, ensuring its effectiveness in accurately measuring the TPB constructs for PB in diversified educational medical institutions.