| CARVIEW |
ResearchBox #431 - 'Colada[99] - Hyping Fisher'
We are zipping them into a single file for an easy download.
The zip file so far is
You downloaded so far
| Show file names | |
| Show file names | |
| Show file IDs | |
| Show file IDs | |
| Show timestamps | |
| Show timestamps |
png - Table in QJE listing reproduced scenarios [17]
2021-10-10 | csv - Summary FPR results [19]
2021-10-10 | 1. R Code - Run Simulations [18]
2021-10-10 2. R Code - Make figure [13]
2021-10-09 | Figure with simulation results (png) [16]
2021-10-09 rds - Every simulation result (12 Mb)[7]
2021-10-04 | ||||
| R Code - Simulate dropping 1 from Normal data [9]
2021-10-04 |
Previewing files
Files can be previewed by clicking on blue font.
Codebooks can be previewed by clicking on
'DEAR READER' MESSAGE FROM THE AUTHORS
Original analyses
The original paper by Young (2019),"Channeling Fisher", is available from https://doi.org/10.1093/qje/qjy029
MATA code, (STATA programming language for matrix calculations) to reproduce calculations in the original paper is available from:
Young, Alwyn, 2018, "Replication Data for: 'Channeling Fisher: Randomization Tests and the Statistical Insignificance of Seemingly Significant Experimental Results'", https://doi.org/10.7910/DVN/JX6HCJ, Harvard Dataverse, V1 (https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/JX6HCJ )
The MATA code for the simulations is about 1500 lines of code without comments (beyond section headers), so I coded in R from scratch, following only the descriptions in the paper.
The QJE article also reports reanalyses of 53 published papers, but the data for those papers is not posted, nor linked to from the online materials for the QJE paper. To reproduce those analyses (about instability of results to removal of individual observations) you'd need to on your own find the datasets for the 53 original papers, figure out the code used in the original studies, then understand the code behind the QJE paper in relation to that original code.
This version: October 06, 2021
(may be edited at any time)
BOX INFORMATION
SUPPLEMENTARY FILES FORUri SImonsohn, '[99] Hyping Fisher: The Most Cited 2019 QJE Paper Relied on an Outdated STATA Default to Conclude Regression p-values Are Inadequate', Data Colada
https://datacolada.org/99
CITING THIS RESEARCHBOX
Simonsohn, U. (2025). ResearchBox 431, 'Colada[99] - Hyping Fisher', https://ResearchBox.org/431. Zenodo. https://doi.org/10.5281/zenodo.15032943
LICENSE FOR USE
All content posted to ResearchBox is under a CC By 4.0 License (all use is allowed as long as authorship of the content is attributed). When using content from ResearchBox please cite the original work, and provide a link to the URL for this box (https://researchbox.org/431).
BOX PUBLIC SINCE
October 12, 2021
BOX CREATORS
Uri Simonsohn (urisohn@gmail.com)
ABSTRACT
The paper titled "Channeling Fisher: Randomization Tests and the Statistical Insignificance of Seemingly Significant Experimental Results" (.htm) is currently the most cited 2019 article in the Quarterly Journal of Economics (372 Google cites). It delivers bad news to economists running experiments: their p-values are wrong. To get correct p-values, the article explains, they need to run, instead of regressions, something called "randomization tests" (I briefly describe randomization tests in the next subsection). For example, the QJE article reads: "[the results] show the clear advantages of randomization inference . . . [it] is superior to all other methods" (p.585). In this post I show that this conclusion only holds when relying on an unfortunate default setting in STATA. In contrast, when regression results are computed using the default setting in R, the supposed superiority of the randomization test goes away.