Impacts of study design on sample size, participation bias, and outcome measurement: A case study from bicycling research

Introduction

Measuring bicycling behaviour is critical to bicycling research. A common study design question is whether to measure bicycling behaviour once (cross-sectional) or multiple times (longitudinal). The Physical Activity through Sustainable Transport Approaches (PASTA) project is a longitudinal cohort study of over 10,000 participants from seven European cities over two years. We used PASTA data as a case study to investigate how measuring once or multiple times impacted three factors: a) sample size b) participation bias and c) accuracy of bicycling behaviour estimates.

Methods

We compared two scenarios: i) as if only the baseline data were collected (cross-sectional approach) and ii) as if the baseline plus repeat follow-ups were collected (longitudinal approach). We compared each approach in terms of differences in sample size, distribution of sociodemographic characteristics, and bicycling behaviour. In the cross-sectional approach, we measured participants long-term bicycling behaviour by asking for recall of typical weekly habits, while in the longitudinal approach we measured by taking the average of bicycling reported for each 7-day period.

Results

Relative to longitudinal, the cross-sectional approach provided a larger sample size and slightly better representation of certain sociodemographic groups, with worse estimates of long-term bicycling behaviour. The longitudinal approach suffered from participation bias, especially the drop-out of more frequent bicyclists. The cross-sectional approach under-estimated the proportion of the population that bicycled, as it captured ‘typical’ behaviour rather than 7-day recall. The magnitude and directionality of the difference between typical weekly (cross-sectional approach) and the average 7-day recall (longitudinal approach) varied depending on how much bicycling was initially reported.

Conclusions

In our case study we found that measuring bicycling once, resulted in a larger sample with better representation of sociodemographic groups, but different estimates of long-term bicycling behaviour. Passive detection of bicycling through mobile apps could be a solution to the identified issues.