Nonstandard Errors

Colaboración

doi:10.1111/jofi.13337

Nonstandard Errors

Colaboración

Research output: Contribution to journal › Article › peer-review

35 Scopus citations

Abstract

In statistics, samples are drawn from a population in a data-generating process (DGP). Standard errors measure the uncertainty in estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence-generating process (EGP). We claim that EGP variation across researchers adds uncertainty—nonstandard errors (NSEs). We study NSEs by letting 164 teams test the same hypotheses on the same data. NSEs turn out to be sizable, but smaller for more reproducible or higher rated research. Adding peer-review stages reduces NSEs. We further find that this type of uncertainty is underestimated by participants.

Original language	English
Pages (from-to)	2339-2390
Number of pages	52
Journal	Journal of Finance
Volume	79
Issue number	3
DOIs	https://doi.org/10.1111/jofi.13337
State	Published - Jun 2024

Bibliographical note

Publisher Copyright:
© 2024 The Authors. The Journal of Finance published by Wiley Periodicals LLC on behalf of American Finance Association.

Access to Document

10.1111/jofi.13337

Cite this

@article{3c494e5d31ed4e54bd68398e1d0e6b05,

title = "Nonstandard Errors",

abstract = "In statistics, samples are drawn from a population in a data-generating process (DGP). Standard errors measure the uncertainty in estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence-generating process (EGP). We claim that EGP variation across researchers adds uncertainty—nonstandard errors (NSEs). We study NSEs by letting 164 teams test the same hypotheses on the same data. NSEs turn out to be sizable, but smaller for more reproducible or higher rated research. Adding peer-review stages reduces NSEs. We further find that this type of uncertainty is underestimated by participants.",

author = "Colaboraci{\'o}n and Menkveld, \{Albert J.\} and Anna Dreber and Felix Holzmeister and Juergen Huber and Magnus Johannesson and Michael Kirchler and Sebastian Neus{\"u}{\ss} and Michael Razen and Utz Weitzel and David Abad-D{\'i}az and Menachem Abudy and Tobias Adrian and Yacine Ait-Sahalia and Olivier Akmansoy and Alcock, \{Jamie T.\} and Vitali Alexeev and Arash Aloosh and Livia Amato and Diego Amaya and Angel, \{James J.\} and Avetikian, \{Alejandro T.\} and Amadeus Bach and Edwin Baidoo and Gaetan Bakalli and Li Bao and Andrea Barbon and Oksana Bashchenko and Bindra, \{Parampreet C.\} and Bj{\o}nnes, \{Geir H.\} and Black, \{Jeffrey R.\} and Black, \{Bernard S.\} and Dimitar Bogoev and Correa, \{Santiago Bohorquez\} and Oleg Bondarenko and Bos, \{Charles S.\} and Ciril Bosch-Rosa and Elie Bouri and Christian Brownlees and Anna Calamia and Cao, \{Viet Nga\} and Gunther Capelle-Blancard and Romero, \{Laura M.Capera\} and Massimiliano Caporin and Allen Carrion and Tolga Caskurlu and Bidisha Chakrabarty and Jian Chen and Mikhail Chernov and William Cheung and Kervel, \{Vincent Van\}",

note = "Publisher Copyright: {\textcopyright} 2024 The Authors. The Journal of Finance published by Wiley Periodicals LLC on behalf of American Finance Association.",

year = "2024",

month = jun,

doi = "10.1111/jofi.13337",

language = "English",

volume = "79",

pages = "2339--2390",

journal = "Journal of Finance",

issn = "0022-1082",

publisher = "Wiley-Blackwell Publishing Ltd",

number = "3",

}

TY - JOUR

T1 - Nonstandard Errors

AU - Colaboración

AU - Menkveld, Albert J.

AU - Dreber, Anna

AU - Holzmeister, Felix

AU - Huber, Juergen

AU - Johannesson, Magnus

AU - Kirchler, Michael

AU - Neusüß, Sebastian

AU - Razen, Michael

AU - Weitzel, Utz

AU - Abad-Díaz, David

AU - Abudy, Menachem

AU - Adrian, Tobias

AU - Ait-Sahalia, Yacine

AU - Akmansoy, Olivier

AU - Alcock, Jamie T.

AU - Alexeev, Vitali

AU - Aloosh, Arash

AU - Amato, Livia

AU - Amaya, Diego

AU - Angel, James J.

AU - Avetikian, Alejandro T.

AU - Bach, Amadeus

AU - Baidoo, Edwin

AU - Bakalli, Gaetan

AU - Bao, Li

AU - Barbon, Andrea

AU - Bashchenko, Oksana

AU - Bindra, Parampreet C.

AU - Bjønnes, Geir H.

AU - Black, Jeffrey R.

AU - Black, Bernard S.

AU - Bogoev, Dimitar

AU - Correa, Santiago Bohorquez

AU - Bondarenko, Oleg

AU - Bos, Charles S.

AU - Bosch-Rosa, Ciril

AU - Bouri, Elie

AU - Brownlees, Christian

AU - Calamia, Anna

AU - Cao, Viet Nga

AU - Capelle-Blancard, Gunther

AU - Romero, Laura M.Capera

AU - Caporin, Massimiliano

AU - Carrion, Allen

AU - Caskurlu, Tolga

AU - Chakrabarty, Bidisha

AU - Chen, Jian

AU - Chernov, Mikhail

AU - Cheung, William

AU - Kervel, Vincent Van

PY - 2024/6

Y1 - 2024/6

N2 - In statistics, samples are drawn from a population in a data-generating process (DGP). Standard errors measure the uncertainty in estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence-generating process (EGP). We claim that EGP variation across researchers adds uncertainty—nonstandard errors (NSEs). We study NSEs by letting 164 teams test the same hypotheses on the same data. NSEs turn out to be sizable, but smaller for more reproducible or higher rated research. Adding peer-review stages reduces NSEs. We further find that this type of uncertainty is underestimated by participants.

AB - In statistics, samples are drawn from a population in a data-generating process (DGP). Standard errors measure the uncertainty in estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence-generating process (EGP). We claim that EGP variation across researchers adds uncertainty—nonstandard errors (NSEs). We study NSEs by letting 164 teams test the same hypotheses on the same data. NSEs turn out to be sizable, but smaller for more reproducible or higher rated research. Adding peer-review stages reduces NSEs. We further find that this type of uncertainty is underestimated by participants.

UR - https://www.scopus.com/pages/publications/85190503164

U2 - 10.1111/jofi.13337

DO - 10.1111/jofi.13337

M3 - Article

AN - SCOPUS:85190503164

SN - 0022-1082

VL - 79

SP - 2339

EP - 2390

JO - Journal of Finance

JF - Journal of Finance

IS - 3

ER -

Nonstandard Errors

Abstract

Bibliographical note

Access to Document

Other files and links

Fingerprint

Cite this