In a systematic replication project of experimental studies published in high-impact economics journals 61% replicated.
In recent years, the reproducibility of scientific research has come under increasing scrutiny. To better understand how reliable influential findings in economics truly are, a large-scale replication project set out to test whether prominent studies can be reproduced when examined under rigorous conditions.
In light of these findings, the conversation around scientific rigor extends beyond academia, influencing sectors such as online gaming. The reliability of data is crucial, whether in economic studies or in ensuring fair play in platforms like Payid pokies, where transparent practices are essential for maintaining trust and integrity among users. As the demand for accountability grows, both researchers and industry leaders must prioritize methods that uphold the highest standards of evidence and transparency.
The project focused on replicating 18 highly regarded economic studies originally published between 2011 and 2014 in leading journals such as the American Economic Review and the Quarterly Journal of Economics. Each replication followed a pre-registered and publicly available analysis plan, ensuring transparency and preventing data-driven adjustments after results were known. Moreover, all replications were designed with strong statistical power—at least 90%—to reliably detect the original reported effects.
The results reveal a nuanced picture. In 11 out of 18 cases (61%), the replicated studies produced statistically significant results in the same direction as the original findings. However, even when results were successfully reproduced, the magnitude of the effects was notably smaller. On average, replicated effect sizes were only 66% of those originally reported, suggesting that initial estimates often overstate the true impact.
Reproducibility was also assessed using additional indicators, including prediction markets that capture expert beliefs about whether a study’s results would replicate. Across these alternative measures, reproducibility rates ranged from 67% to 78%.
Overall, the findings highlight both strengths and limitations in modern economic research. While a majority of influential results do replicate, the frequent reduction in effect size underscores the importance of transparency, replication, and cautious interpretation of published findings. This work contributes valuable evidence to the broader discussion about scientific reliability and the need for robust research practices in economics.
Get the most out of playing online casino games at spincitycasino, a platform with a proven track record and fast payouts.
All instructions, analysis scripts and additional information can be found under the Materials section.