|
Institute for Replication@i4replication.bsky.social |
446 followers 0 following 251 posts
The Institute for Replication (I4R) works to improve the credibility of science by promoting and conducting replications.
i4replication.org
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
Still looking for participants for rep games on August 25th:
www.surveymonkey.ca/r/I4R_EEA-ES...
Games are the day before the EEA/ESEM annual meeting and start at about 11:00, allowing participants to get early AM flight/train into Rotterdam. Location is just across train station!
0 replies 3 reposts 0 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
We had lots of fun in Seattle last Friday! 11 teams reproducing/replicating a paper each. A few teams uncovered major coding errors and that results were not robust. But most teams were happy with what they worked with: packages are complete/no issues and results robust!
0 replies 1 reposts 0 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
We are closing (in-person) registration for our games at the Society for Political Methodology. 40+ in-person participants, so it should be an amazing event! If you want to virtually participate, please reach out to Derek Mikola (derek.mikola@gmail.com) ASAP. Big thanks to Kevin Esterling!
0 replies 0 reposts 0 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
Rep Games High Season -Games today in Grenoble (focus on collecting new data) -Games this Friday in Seattle before WEAI conf -Games in Innsbruck next week after Summer school Time to schedule games for 2025. Contact instituteforreplication@gmail.com if you want to organize!
0 replies 2 reposts 3 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
Special Event Alert!
Join us for Replication Games on September 26th in beautiful Växjö, Sweden! This event will focus on register data, bringing together faculty and graduate students from all social science fields.
Register here: surveymonkey.ca/r/I4R_Vaxjo_...
0 replies 3 reposts 1 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
In our third report today, we reproduced “Discretionary Tax Changes and the Macroeconomy”.
econstor.eu/bitstream/10...
Author: "Replicability is extremely important, and I am grateful for all your important work. Many thanks for sharing the paper. I will not be writing a response at this time."
0 replies 1 reposts 2 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
In our second report released today, we reproduced “Commercial Imperialism? Political Influence and Trade during the Cold War”. Results remain significant at 10% level in 58% of our sensitivity analysis.
econstor.eu/bitstream/10...
We did not get a response from the authors.
0 replies 0 reposts 2 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
In our first report today, we reproduced “Coercive Contract Enforcement: Law and the Labor Market in Nineteenth Century Industrial Britain”. Results robust in 50% of our sensitivity analysis.
econstor.eu/bitstream/10...
The authors were satisfied with our report and did not respond.
0 replies 0 reposts 1 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
We have three new reproduction reports out! They are part of our AER project: econpapers.repec.org/paper/zbwi4r...
0 replies 0 reposts 1 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
The Sheffield AI Replication games are going strong! Check out i4replication.org/games.html to find a Games near you, including our next AI event at Cornell this Summer!
0 replies 1 reposts 2 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
A reproduction by Courbe et al. of "Contested killings: The mobilizing effects of community contact with police violence"
Replication: econpapers.repec.org/paper/zbwi4r...
Original authors provided feedback and thanked the replicators.
0 replies 0 reposts 0 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
3rd new reproduction by Krūminas et al. of "When Do Politicians Appeal Broadly? The Economic Consequences of Electoral Rules in Brazil"
econpapers.repec.org/paper/zbwi4r...
Original author thanked the replicators.
0 replies 0 reposts 0 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
2nd new reproduction is by Yang and Huang of "Do Female Officers Police Differently? Evidence from Traffic Stops":
econpapers.repec.org/paper/zbwi4r...
We did not get a response from the original authors.
0 replies 0 reposts 0 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
1st new reproduction is by Degroot et al of "Kin Networks and Institutional Development":
econpapers.repec.org/paper/zbwi4r...
Original author thanked the replicators and acknowledged reporting errors: "Thank you very much for this! I think you make an important contribution to the profession"
0 replies 0 reposts 0 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
We're thrilled to share 4 new reproductions/replications!
econpapers.repec.org/paper/zbwi4r...
0 replies 1 reposts 1 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
Still looking for faculty and graduate students for the Innsbruck Replication Games on July 5th. Economists, political scientists and psychologists are welcome!
Register here: surveymonkey.ca/r/Innsbruck_...
0 replies 4 reposts 2 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
It is a wrap for the Sydney Replication Games! Special thanks to Jason Collins and the University of Technology Sydney for hosting the event.
0 replies 0 reposts 0 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
Join us for the Oxford Replication Games on September 27th! We are looking for faculty and graduate students; all social science fields! Participants get coauthorship to a meta-paper combining reproductions/replications!
Register: www.surveymonkey.ca/r/Oxford_Gam...
@politicsoxford.bsky.social
0 replies 0 reposts 0 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
We've had a wonderful time in Melbourne reproducing/replicating papers! Thanks to Romain Gauriot and Deakin University for hosting us. Next stop: Sydney on June 11th
0 replies 0 reposts 1 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
We've got a new DP:
J. Fitzgerald implements equivalence testing methods in a systematic reproduction of 135 null claims made in 81 articles from top economics journals.
econpapers.repec.org/paper/zbwi4r...
0 replies 4 reposts 4 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
We've got a new DP:
Ankel-Peters & Schmidt discuss the limits of evidence-based policy, with evidence from rural electrification in the Global South
www.econstor.eu/bitstream/10...
0 replies 2 reposts 2 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
Still looking for participants for the AI Replication Games
@ Cornell on August 12! Virtual participation is possible and coauthorship to a meta paper is granted. Join (already) 40+ registered participants!
Register here: www.surveymonkey.ca/r/Cornell_Te...
0 replies 0 reposts 1 likes
|
Alexandra de Gendre
@adegendre.bsky.social
[ View ] |
@i4replication.bsky.social strikes back!!
This is fantastic work. Lots of food for thought on improving the reliability of estimates and the quality of scientific findings.
0 replies 2 reposts 3 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
New DP: econpapers.repec.org/paper/zbwi4r...
Which AER papers were robust (in terms of the % of robustness checks significant at 5%) was predictable by our survey participants, but the survey participants were overly optimistic and overestimated the degree of robustness reproducibility
0 replies 5 reposts 8 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
New DP: econpapers.repec.org/paper/zbwi4r...
Here is the distribution of absolute t/z-values in the original AER studies (red for benchmark, clear for robustness checks) vs. our robustness checks (blue). On the whole, 46% of the robustness checks we ran were significant at a 95% confidence level
0 replies 0 reposts 1 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
New DP from I4R's board members, co-director, chair & others: econpapers.repec.org/paper/zbwi4r.... We reproduced & conducted sensitivity analysis from 17 AER papers. Robustness varies between 17% & 88% across studies. A survey of economists suggest that they overestimate robustness.
0 replies 2 reposts 7 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
We've got news! We are now reproducing/replicating studies published in Political Analysis (2024-onward). To get things started, we are having games during Polmeth's annual conference (July 20th).
Register here: surveymonkey.ca/r/I4R_PolMet.... Virtual participation is possible!
0 replies 1 reposts 4 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
Not too late to register to the Seattle Replication Games on June 28th. Only economics for this event (or full team registration for other fields). We already have 40+ participants, so it should be fun! Register by emailing abrodeur@uottawa.ca your subfields & your statistical software(s) knowledge
0 replies 0 reposts 0 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
Join us for the Innsbruck Replication Games on July 5th! We are looking for faculty and graduate students; all social science fields! Participants get coauthorship to a meta-paper combining reproductions/replications!
Register here: surveymonkey.ca/r/Innsbruck_...
0 replies 2 reposts 0 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
Well that was fast! 72 hours later & we are almost full for Grenoble. Still a few more in-person spots available: surveymonkey.ca/r/Grenoble_R...
. We will keep the virtual option open for a while.
Seems like we need more events focusing on replicating online experiments (or in France?).
0 replies 0 reposts 2 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
Excited to have games in Grenoble (June 26)! Focus will be on online experiments.
Teams will spend most of the day preparing collection of new data. Costs to recruits participants will be covered by I4R & organizers.
Register here: www.surveymonkey.ca/r/Grenoble_R...
@paolocrosetto.bsky.social
0 replies 0 reposts 2 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
Pleased to announce the AI Games @ Sheffield on June 17, 2024. Virtual participation is possible & coauthorship to a meta paper is granted. Participants will be randomly assigned to a team of Machine with restricted human assistance, Cyborg or Human
Register: www.surveymonkey.ca/r/Sheffield_...
0 replies 0 reposts 1 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
Thanks to the 50+ economists, political scientists and psychologists who attended our event in Ottawa! We had lots of fun and hope you all enjoyed.
Registration is open for many events: Madrid (May 17), Melbourne (June 7), Sydney (June 11), Sheffield (June 17), ...
i4replication.org/games.html
0 replies 0 reposts 5 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
Third reproduction: "Technology gaps, trade, and income"
Reproduction: econpapers.repec.org/paper/zbwi4r...
Response: "thank the replicators for taking the time to replicate the paper and extend the analysis in several interesting directions"
0 replies 0 reposts 3 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
2nd reproduction:
"Web of Power: How Elite Networks Shaped War and Politics in China"
Reproduction: econpapers.repec.org/paper/zbwi4r...
Response: osf.io/s85mk/
0 replies 0 reposts 2 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
We're thrilled to share 3 new reproductions.
1-"Understanding Cultural Persistence and Change"
Reproduction: econpapers.repec.org/paper/zbwi4r...
Authors' response: econpapers.repec.org/paper/zbwi4r...
Correction: academic.oup.com/restud/artic...
0 replies 0 reposts 4 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
Last chance to register to our AI games on May 3rd. Virtual participation is possible. The event is from 9AM to 4PM ET. Participants will be randomly assigned to one of three teams: Machine, Cyborg or Human. All social science fields! Register here: www.surveymonkey.ca/r/Ottawa_Ter...
0 replies 0 reposts 1 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
3rd and 4th DPs:
3-"Student Performance, Peer Effects, and Friend Networks" econstor.eu/bitstream/10...
We thank the authors for suggestions.
4-"Evaluating Deliberative Competence" Reproduction: econpapers.repec.org/paper/zbwi4r...
We thank the authors for suggestions.
0 replies 0 reposts 0 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
We're thrilled to share four new DPs.
1-Can Technology Solve the Principal-Agent Problem?" econpapers.repec.org/paper/zbwi4r...
Authors' response: osf.io/b7dn2/
2-"Coase and Cap-and-Trade": econpapers.repec.org/paper/zbwi4r...
We thank the authors' for suggestions.
0 replies 0 reposts 0 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
Brown Games is underway in beautiful Providence (nice view from our room)!
Today, we have 11 teams reproducing econ articles and a few poli sci and NHB articles. All 1st year grad econ students at Brown participate!
Future Games: i4replication.org/games.html
0 replies 1 reposts 3 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
Join us for the Madrid Replication Games on May 17th! We are looking for faculty and graduate students; all social science fields! Participants get coauthorship to a meta-paper combining reproductions!
Register here: www.surveymonkey.ca/r/I4R_Madrid...
Tapas and delicious Spanish food are on us!
0 replies 0 reposts 2 likes
|
Brian Nosek
@briannosek.bsky.social
[ View ] |
Announcing *Lifecycle Journals* a new model for scholarly publishing that meets the existing needs of the reward system, integrates many innovations in scholarly communication, and provides a values-aligned, community-led solution.
Read more and sign-up for updates:
www.cos.io/lifecyclejou...
2 replies 20 reposts 34 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
Northwestern Games is underway! Participants work in teams to reproduce a paper throughout the day.
Today, we have 11 teams reproducing @psychscience.bsky.social, Nature Human Behaviour, econ/poli sci articles.
Find a replication games near you: i4replication.org/games.html
0 replies 0 reposts 3 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
Still looking for more economists, psychologists and social scientists to join our Melbourne and Sydney reproducibility events!!
Register here for Melbourne: www.surveymonkey.ca/r/I4R_Deakin...
And here for Sydney: www.surveymonkey.ca/r/I4R_Sydney...
0 replies 2 reposts 2 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
To conclude, we believe the scale of this ongoing project has the potential to change research norms and researchers’ behavior through the adoption of more rigorous methodologies and deterring questionable research practices.
0 replies 2 reposts 5 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
Lastly, we ask all replicators in the individual survey about the quality of the replication package they reproduced and their views of the discipline. Over 40% report that the quality of the replication package gave them a more optimistic view of the discipline.
1 replies 1 reposts 2 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
Most teams find a negative relationship between replicators’ experience and robustness reproducibility, while finding no relationship between robustness reproducibility and the provision of intermediate or even raw data combined with the necessary cleaning codes.
1 replies 0 reposts 2 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
8- We then investigate eight additional research questions using a many-analysts approach. Six teams independently answered questions such as “Does reproducibility/replicability rate depend on replicators’ academic experience or experience coding?”
1 replies 0 reposts 1 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
computationally reproduce the original authors’ results. Another 17% were unable to access the original authors’ data. Interestingly, about two-thirds of replicators mentioned that interacting with the original authors improved the quality of their report.
1 replies 0 reposts 1 likes
|
|
Institute for Replication
@i4replication.bsky.social
[ View ] |
7- Did original authors help? We asked replicators whether their team or I4R contacted, or attempted to contact, the original authors for clarifications. About 40% responded “yes”. About 10% reached out because the replication package was unclear, while 17% needed help to ...
1 replies 0 reposts 2 likes