Confirmatory factor analysis comparing incentivized experiments with self-report methods to elicit adolescent smoking and vaping social norms

Many adolescent smoking prevention programmes target social norms, typically evaluated with self-report, susceptible to social desirability bias. An alternative approach with little application in public health are experimental norms elicitation methods. Using the Mechanisms of Networks and Norms Influence on Smoking in Schools (MECHANISMS) study baseline data, from 12–13 year old school pupils (n = 1656) in Northern Ireland and Bogotá (Colombia), we compare two methods of measuring injunctive and descriptive smoking and vaping norms: (1) incentivized experiments, using monetary payments to elicit norms; (2) self-report scales. Confirmatory factor analysis (CFA) examined whether the methods measured the same construct. Paths from exposures (country, sex, personality) to social norms, and associations of norms with (self-reported and objectively measured) smoking behavior/intentions were inspected in another structural model. Second-order CFA showed that latent variables representing experimental and survey norms measurements were measuring the same underlying construct of anti-smoking/vaping norms (Comparative Fit Index = 0.958, Tucker Lewis Index = 0.951, Root Mean Square Error of Approximation = 0.030, Standardized Root Mean Square Residual = 0.034). Adding covariates into a structural model showed significant paths from country to norms (second-order anti-smoking/vaping norms latent variable: standardized factor loading [β] = 0.30, standard error [SE] = 0.09, p < 0.001), and associations of norms with self-reported anti-smoking behavior (β = 0.40, SE = 0.04, p < 0.001), self-reported anti-smoking intentions (β = 0.42, SE = 0.06, p < 0.001), and objectively measured smoking behavior (β = − 0.20, SE = 0.06, p = 0.001). This paper offers evidence for the construct validity of behavioral economic methods of eliciting adolescent smoking and vaping norms. These methods seem to index the same underlying phenomena as commonly-used self-report scales.