Nonstandard errors

Albert J. Menkveld; Anna Dreber; Felix Holzmeister; Juergen Huber; Magnus Johannesson; Michael Kirchler; Sebastian Neusüß; Michael Razen; Utz Weitzel; null Fincap Team; Fearghal Kearney; Tony Klein; Liangyi Mu

doi:10.1111/jofi.13337

Nonstandard errors

Albert J. Menkveld^*, Anna Dreber, Felix Holzmeister, Juergen Huber, Magnus Johannesson, Michael Kirchler, Sebastian Neusüß, Michael Razen, Utz Weitzel, Fincap Team, Fearghal Kearney, Tony Klein, Liangyi Mu

^*Corresponding author for this work

Research output: Contribution to journal › Article › peer-review

9 Citations (Scopus)

47 Downloads (Pure)

Abstract

In statistics, samples are drawn from a population in a data-generating process (DGP). Standard errors measure the uncertainty in estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence-generating process (EGP). We claim that EGP variation across researchers adds uncertainty—nonstandard errors (NSEs). We study NSEs by letting 164 teams test the same hypotheses on the same data. NSEs turn out to be sizable, but smaller for more reproducible or higher rated research. Adding peer-review stages reduces NSEs. We further find that this type of uncertainty is underestimated by participants.

Original language	English
Pages (from-to)	2339-2390
Journal	Journal of Finance
Volume	79
Issue number	3
Early online date	17 Apr 2024
DOIs	https://doi.org/10.1111/jofi.13337
Publication status	Published - Jun 2024

Access to Document

10.1111/jofi.13337Licence: CC BY

Nonstandard errors
Copyright 2024 the authors. This is an open access article published under a Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution and reproduction in any medium, provided the author and source are cited.
Final published version, 1.16 MBLicence: CC BY

Cite this

@article{e359c6f01ac74568b825e9a1e743d9a1,

title = "Nonstandard errors",

abstract = "In statistics, samples are drawn from a population in a data-generating process (DGP). Standard errors measure the uncertainty in estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence-generating process (EGP). We claim that EGP variation across researchers adds uncertainty—nonstandard errors (NSEs). We study NSEs by letting 164 teams test the same hypotheses on the same data. NSEs turn out to be sizable, but smaller for more reproducible or higher rated research. Adding peer-review stages reduces NSEs. We further find that this type of uncertainty is underestimated by participants.",

author = "Menkveld, {Albert J.} and Anna Dreber and Felix Holzmeister and Juergen Huber and Magnus Johannesson and Michael Kirchler and Sebastian Neus{\"u}{\ss} and Michael Razen and Utz Weitzel and {Fincap Team} and Fearghal Kearney and Tony Klein and Liangyi Mu",

year = "2024",

month = jun,

doi = "10.1111/jofi.13337",

language = "English",

volume = "79",

pages = "2339--2390",

journal = "Journal of Finance",

issn = "0022-1082",

publisher = "Wiley-Blackwell",

number = "3",

}

TY - JOUR

T1 - Nonstandard errors

AU - Menkveld, Albert J.

AU - Dreber, Anna

AU - Holzmeister, Felix

AU - Huber, Juergen

AU - Johannesson, Magnus

AU - Kirchler, Michael

AU - Neusüß, Sebastian

AU - Razen, Michael

AU - Weitzel, Utz

AU - Fincap Team, null

AU - Kearney, Fearghal

AU - Klein, Tony

AU - Mu, Liangyi

PY - 2024/6

Y1 - 2024/6

N2 - In statistics, samples are drawn from a population in a data-generating process (DGP). Standard errors measure the uncertainty in estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence-generating process (EGP). We claim that EGP variation across researchers adds uncertainty—nonstandard errors (NSEs). We study NSEs by letting 164 teams test the same hypotheses on the same data. NSEs turn out to be sizable, but smaller for more reproducible or higher rated research. Adding peer-review stages reduces NSEs. We further find that this type of uncertainty is underestimated by participants.

AB - In statistics, samples are drawn from a population in a data-generating process (DGP). Standard errors measure the uncertainty in estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence-generating process (EGP). We claim that EGP variation across researchers adds uncertainty—nonstandard errors (NSEs). We study NSEs by letting 164 teams test the same hypotheses on the same data. NSEs turn out to be sizable, but smaller for more reproducible or higher rated research. Adding peer-review stages reduces NSEs. We further find that this type of uncertainty is underestimated by participants.

U2 - 10.1111/jofi.13337

DO - 10.1111/jofi.13337

M3 - Article

SN - 0022-1082

VL - 79

SP - 2339

EP - 2390

JO - Journal of Finance

JF - Journal of Finance

IS - 3

ER -

Nonstandard errors

Abstract

Access to Document

Fingerprint

Cite this