|
Carlisle Rainey 👨💻📊📚@carlislerainey.bsky.social |
784 followers 519 following 263 posts
🆕 sharing new papers
🧪 posting about open science
👨🏫 political scientist at FSU
🔬 experimental design
🎲 Bayesian modeling
✅ reproducibility
Web: www.carlislerainey.com
Google Scholar: scholar.google.com/citations?user=otXLf3wAAAAJ
|
|
Carlisle Rainey 👨💻📊📚
@carlislerainey.bsky.social
[ View ] |
Important for me to note that the new(ish) R package {clarify} uses direct transformation, while the older Stata software CLARIFY and R package {Zelig} use averaged simulations.
iqss.github.io/clarify/
0 replies 0 reposts 0 likes
|
|
Carlisle Rainey 👨💻📊📚
@carlislerainey.bsky.social
[ View ] |
My paper comparing averaged simulations to directly transformed point estimates is published in the latest issue of @psrm.bsky.social (along with lots of other great papers!).
www.cambridge.org/core/journal...
1 replies 0 reposts 3 likes
|
|
Carlisle Rainey 👨💻📊📚
@carlislerainey.bsky.social
[ View ] |
In my suggested formula, the effect is assumed. The pilot data is used to estimate the SE in the planned study. Or perhaps more intuitively, to estimate the SD.
0 replies 0 reposts 1 likes
|
|
Carlisle Rainey 👨💻📊📚
@carlislerainey.bsky.social
[ View ] |
Thanks for the mention!
Here’s an example. Links to a more full paper in the post.
www.carlislerainey.com/blog/2024-06...
1 replies 0 reposts 2 likes
|
|
Carlisle Rainey 👨💻📊📚
@carlislerainey.bsky.social
[ View ] |
New Post! 🥳
Power Analysis Using Pilot Data, Part 2: An Example
This post follows up on last week's post and gives an example of a power analysis using actual pilot data.
We've got the main study completed as well, so we can see how we did!
#polisky #stats
www.carlislerainey.com/blog/2024-06...
0 replies 1 reposts 7 likes
|
|
Carlisle Rainey 👨💻📊📚
@carlislerainey.bsky.social
[ View ] |
To connect Kane's actions with the ratio (or power), I annotated the screenshot of his Table 1. - Yellow highlights flag actions that increase the treatment effect. - Blue highlights flag action items that shrink the SE. - Red stars indicate relevant, but unclear. This is a helpful list!
0 replies 0 reposts 0 likes
|
|
Carlisle Rainey 👨💻📊📚
@carlislerainey.bsky.social
[ View ] |
Many of Kane's action items help make sure that treatments have large effects and that estimates are precise. The paper is well worth a read *before* you find yourself with null results.
1 replies 0 reposts 0 likes
|
|
Carlisle Rainey 👨💻📊📚
@carlislerainey.bsky.social
[ View ] |
See the post and early paper below for more on this ratio.
Post: www.carlislerainey.c...
Early Paper: github.com/carlisler...
1 replies 0 reposts 0 likes
|
|
Carlisle Rainey 👨💻📊📚
@carlislerainey.bsky.social
[ View ] |
I like to think about power as the ratio of the treatment effect to the standard error. Thus, powerful experiments have (1) treatments with oomph and (2) precise estimates of those treatments.
1 replies 0 reposts 0 likes
|
|
Carlisle Rainey 👨💻📊📚
@carlislerainey.bsky.social
[ View ] |
One of my favorite recent papers is Kane (2024).
I like to think of it as a paper about power.
While pitched (effectively and usefully) as a paper about compelling null results, many of the action items are ways to boost power.
DOI: doi.org/10.1017/XPS....
CC: @uptonorwell.bsky.social
/search?q=%23polisky'>#polisky
1 replies 3 reposts 7 likes
|
|
Carlisle Rainey 👨💻📊📚
@carlislerainey.bsky.social
[ View ] |
While still about 70% of articles do not include data and code, the figure above shows a massive shift in norms, requirements, and infrastructure.
The improvement is not accidental, but due to a decades-long, deliberate effort by leaders our field.
Preprint: osf.io/a5yxe
0 replies 0 reposts 2 likes
|
|
Carlisle Rainey 👨💻📊📚
@carlislerainey.bsky.social
[ View ] |
New Version! 🎉
"Data and Code Availability in Political Science Publications from 1995 to 2022"
Now conditionally accepted at @pspolisci.bsky.social
w/ Harley Roe (@harleyroe.bsky.social), Qing Wang, and Hao Zhou
1 replies 4 reposts 8 likes
|
|
Carlisle Rainey 👨💻📊📚
@carlislerainey.bsky.social
[ View ] |
Updated!
Code here: gist.github.com/carlislerain...
0 replies 0 reposts 1 likes
|
|
Carlisle Rainey 👨💻📊📚
@carlislerainey.bsky.social
[ View ] |
Also, Macartan shared this one Twitter (I meant to link in the post, but forgot)
declaredesign.org/blog/posts/p...
0 replies 0 reposts 0 likes
|
Alex Coppock
@aecoppock.bsky.social
[ View ] |
+1 to the main takeaway from this post that pilots are great for reasoning about your eventual standard error, less great for learning about the treatment effect.
Complements this chapter nicely: book.declaredesign.org/lifecycle/pl...
And I couldn't resist rewriting these loops in DeclareDesign:
1 replies 4 reposts 10 likes
|
|
Carlisle Rainey 👨💻📊📚
@carlislerainey.bsky.social
[ View ] |
Do you mind if I put this code in a gist and share in the post? Also, I always appreciate you pointing to relevant resources from DeclareDesign—there’s so much good stuff there!
2 replies 0 reposts 1 likes
|
|
Carlisle Rainey 👨💻📊📚
@carlislerainey.bsky.social
[ View ] |
New Post! 🥳
"Power Analysis Using Pilot Data: Simulations to Illustrate"
In this post, I discuss how you can use pilot data to predict the statistical power in a planned study.
#polisky #stats #metasci
4-06-03-pilot-poww'>wer/'>www.carlislerainey.c...
0 replies 8 reposts 19 likes
|
|
Carlisle Rainey 👨💻📊📚
@carlislerainey.bsky.social
[ View ] |
Preprint: osf.io/preprints/soc...
0 replies 0 reposts 1 likes
|
|
Carlisle Rainey 👨💻📊📚
@carlislerainey.bsky.social
[ View ] |
"Improving Small-Area Estimates of Public Opinion by Calibrating to Known Population Quantities" > We illustrate our approach using a pre-election poll measuring support for an abortion referendum, finding that the method reduces county-level error by two-thirds. 67%!
1 replies 1 reposts 3 likes
|
|
Carlisle Rainey 👨💻📊📚
@carlislerainey.bsky.social
[ View ] |