Crowdsourcing hypothesis tests: making transparent how design choices shape research results.

Elizabeth Mullen; Jun Pang; Jennifer Ray; Diego A. Reinero; Jesse Reynolds; Walter Sowden; Daniel Storage; Runkun Su; Christina M. Tworek; Jay J. Van Bavel; Daniel Walco; Julian Wills; Xiaobing Xu; Kai Chi Yam; Xiaoyu Yang; William A. Cunningham; Martin Schweinsberg; Molly Urwitz; The Crowdsourcing Hypothesis Tests Collaboration; Sarah M. G. Otner; Eric L. Uhlmann; Eric-Jan Wagenmakers; Andrew Proctor; Daniel M. Bartels; Christopher W. Bauman; William J. Brady; Felix Cheung; Andrei Cimpian; Simone Dohle; M. Brent Donnellan; Adam Hahn; Michael P. Hall; William Jiménez-Leal; David J. Johnson; Richard E. Lucas; Beno├«t Monin; Andres Montealegre; Justin F. Landy; Miaolei (Liam) Jia; Isabel L. Ding; Domenico Viganola; Warren Tierney; Anna Dreber; Magnus Johannesson; Thomas Pfeiffer; Charles R. Ebersole; Quentin F. Gronau; Alexander Ly; Don van den Bergh; Maarten Marsman; Koen Derks

doi:10.1037/bul0000220

Crowdsourcing hypothesis tests: making transparent how design choices shape research results.

Elizabeth Mullen
, Jun Pang
, Jennifer Ray
, Diego A. Reinero
, Jesse Reynolds
, Walter Sowden
, Daniel Storage
, Runkun Su
, Christina M. Tworek
, Jay J. Van Bavel
, Daniel Walco
, Julian Wills
, Xiaobing Xu
, Kai Chi Yam
, Xiaoyu Yang
, William A. Cunningham
, Martin Schweinsberg
, Molly Urwitz
, The Crowdsourcing Hypothesis Tests Collaboration
, Sarah M. G. Otner

Eric L. Uhlmann, Eric-Jan Wagenmakers, Andrew Proctor, Daniel M. Bartels, Christopher W. Bauman, William J. Brady, Felix Cheung, Andrei Cimpian, Simone Dohle, M. Brent Donnellan, Adam Hahn, Michael P. Hall, William Jiménez-Leal, David J. Johnson, Richard E. Lucas, Beno├«t Monin, Andres Montealegre, Justin F. Landy, Miaolei (Liam) Jia, Isabel L. Ding, Domenico Viganola, Warren Tierney, Anna Dreber, Magnus Johannesson, Thomas Pfeiffer, Charles R. Ebersole, Quentin F. Gronau, Alexander Ly, Don van den Bergh, Maarten Marsman, Koen Derks

Research output: Contribution to journal › Article › peer-review

Original language	English
Pages (from-to)	451-479
Journal	Psychological Bulletin
Volume	146
Issue number	5
Early online date	16 Jan 2020
DOIs	https://doi.org/10.1037/bul0000220
Publication status	Published - May 2020

Bibliographical note

Note: This research was funded by Institut Européen d'Administration des Affaires, Jan Wallanders och Tom Hedelius Stiftelse samt Tore Browaldhs Stiftelse, Knut och Alice Wallenbergs Stiftelse, Austrian Science Fund [grant number FWF, SFB F63], Swedish Foundation for Humanities and Social Sciences, and the Marsden Fund [grant number 16-UOA-190] and [grant number 17-MAU-133].

Keywords

Business and management studies

Access to Document

10.1037/bul0000220

https://doi.org/10.1037/bul0000220

Cite this

Mullen, E., Pang, J., Ray, J., Reinero, D. A., Reynolds, J., Sowden, W., Storage, D., Su, R., Tworek, C. M., Van Bavel, J. J., Walco, D., Wills, J., Xu, X., Yam, K. C., Yang, X., Cunningham, W. A., Schweinsberg, M., Urwitz, M., The Crowdsourcing Hypothesis Tests Collaboration, ... Derks, K. (2020). Crowdsourcing hypothesis tests: making transparent how design choices shape research results. Psychological Bulletin, 146(5), 451-479. https://doi.org/10.1037/bul0000220

@article{8dc817e25010442c9ea859f5d559fe55,

title = "Crowdsourcing hypothesis tests: making transparent how design choices shape research results.",

keywords = "Business and management studies",

author = "Elizabeth Mullen and Jun Pang and Jennifer Ray and Reinero, \{Diego A.\} and Jesse Reynolds and Walter Sowden and Daniel Storage and Runkun Su and Tworek, \{Christina M.\} and \{Van Bavel\}, \{Jay J.\} and Daniel Walco and Julian Wills and Xiaobing Xu and Yam, \{Kai Chi\} and Xiaoyu Yang and Cunningham, \{William A.\} and Martin Schweinsberg and Molly Urwitz and \{The Crowdsourcing Hypothesis Tests Collaboration\} and Otner, \{Sarah M. G.\} and Uhlmann, \{Eric L.\} and Eric-Jan Wagenmakers and Andrew Proctor and Bartels, \{Daniel M.\} and Bauman, \{Christopher W.\} and Brady, \{William J.\} and Felix Cheung and Andrei Cimpian and Simone Dohle and Donnellan, \{M. Brent\} and Adam Hahn and Hall, \{Michael P.\} and William Jim{\'e}nez-Leal and Johnson, \{David J.\} and Lucas, \{Richard E.\} and Beno├«t Monin and Andres Montealegre and Landy, \{Justin F.\} and Jia, \{Miaolei (Liam)\} and Ding, \{Isabel L.\} and Domenico Viganola and Warren Tierney and Anna Dreber and Magnus Johannesson and Thomas Pfeiffer and Ebersole, \{Charles R.\} and Gronau, \{Quentin F.\} and Alexander Ly and \{van den Bergh\}, Don and Maarten Marsman and Koen Derks",

note = "Note: This research was funded by Institut Europ{\'e}en d'Administration des Affaires, Jan Wallanders och Tom Hedelius Stiftelse samt Tore Browaldhs Stiftelse, Knut och Alice Wallenbergs Stiftelse, Austrian Science Fund [grant number FWF, SFB F63], Swedish Foundation for Humanities and Social Sciences, and the Marsden Fund [grant number 16-UOA-190] and [grant number 17-MAU-133].",

year = "2020",

month = may,

doi = "10.1037/bul0000220",

language = "English",

volume = "146",

pages = "451--479",

journal = "Psychological Bulletin",

issn = "0033-2909",

publisher = "American Psychological Association",

number = "5",

}

Mullen, E, Pang, J, Ray, J, Reinero, DA, Reynolds, J, Sowden, W, Storage, D, Su, R, Tworek, CM, Van Bavel, JJ, Walco, D, Wills, J, Xu, X, Yam, KC, Yang, X, Cunningham, WA, Schweinsberg, M, Urwitz, M, The Crowdsourcing Hypothesis Tests Collaboration, Otner, SMG, Uhlmann, EL, Wagenmakers, E-J, Proctor, A, Bartels, DM, Bauman, CW, Brady, WJ, Cheung, F, Cimpian, A, Dohle, S, Donnellan, MB, Hahn, A, Hall, MP, Jiménez-Leal, W, Johnson, DJ, Lucas, RE, Monin, B, Montealegre, A, Landy, JF, Jia, M, Ding, IL, Viganola, D, Tierney, W, Dreber, A, Johannesson, M, Pfeiffer, T, Ebersole, CR, Gronau, QF, Ly, A, van den Bergh, D, Marsman, M & Derks, K 2020, 'Crowdsourcing hypothesis tests: making transparent how design choices shape research results.', Psychological Bulletin, vol. 146, no. 5, pp. 451-479. https://doi.org/10.1037/bul0000220

TY - JOUR

T1 - Crowdsourcing hypothesis tests

T2 - making transparent how design choices shape research results.

AU - Mullen, Elizabeth

AU - Pang, Jun

AU - Ray, Jennifer

AU - Reinero, Diego A.

AU - Reynolds, Jesse

AU - Sowden, Walter

AU - Storage, Daniel

AU - Su, Runkun

AU - Tworek, Christina M.

AU - Van Bavel, Jay J.

AU - Walco, Daniel

AU - Wills, Julian

AU - Xu, Xiaobing

AU - Yam, Kai Chi

AU - Yang, Xiaoyu

AU - Cunningham, William A.

AU - Schweinsberg, Martin

AU - Urwitz, Molly

AU - The Crowdsourcing Hypothesis Tests Collaboration, null

AU - Otner, Sarah M. G.

AU - Uhlmann, Eric L.

AU - Wagenmakers, Eric-Jan

AU - Proctor, Andrew

AU - Bartels, Daniel M.

AU - Bauman, Christopher W.

AU - Brady, William J.

AU - Cheung, Felix

AU - Cimpian, Andrei

AU - Dohle, Simone

AU - Donnellan, M. Brent

AU - Hahn, Adam

AU - Hall, Michael P.

AU - Jiménez-Leal, William

AU - Johnson, David J.

AU - Lucas, Richard E.

AU - Monin, Beno├«t

AU - Montealegre, Andres

AU - Landy, Justin F.

AU - Jia, Miaolei (Liam)

AU - Ding, Isabel L.

AU - Viganola, Domenico

AU - Tierney, Warren

AU - Dreber, Anna

AU - Johannesson, Magnus

AU - Pfeiffer, Thomas

AU - Ebersole, Charles R.

AU - Gronau, Quentin F.

AU - Ly, Alexander

AU - van den Bergh, Don

AU - Marsman, Maarten

AU - Derks, Koen

N1 - Note: This research was funded by Institut Européen d'Administration des Affaires, Jan Wallanders och Tom Hedelius Stiftelse samt Tore Browaldhs Stiftelse, Knut och Alice Wallenbergs Stiftelse, Austrian Science Fund [grant number FWF, SFB F63], Swedish Foundation for Humanities and Social Sciences, and the Marsden Fund [grant number 16-UOA-190] and [grant number 17-MAU-133].

PY - 2020/5

Y1 - 2020/5

KW - Business and management studies

U2 - 10.1037/bul0000220

DO - 10.1037/bul0000220

M3 - Article

SN - 0033-2909

VL - 146

SP - 451

EP - 479

JO - Psychological Bulletin

JF - Psychological Bulletin

IS - 5

ER -