Remember: press the S key to view speaker notes otherwise you won't get context for what I'm trying to do here.
Road to Rigor
Community for Rigor
Goals for Today
Explore rigor issues in research
Identify motives for improving rigor
Begin a conversation about rigor in our lab
1. Is rigor in crisis?
A Tale of Two Exclusion Criteria
Paper 1: No significant difference between 3ft. or 6ft. distancing.
Studied covid spread in 537,336 students and 99,390 staff across 251 eligible school districts over 16 weeks of instruction, excluding weeks with less than 5% of student body attendance.
Found no significant difference in covid spread among staff or students between 3ft. and 6ft. programs.
Paper 2: Yes there is.
Paper 1 replicates, but why only exclude weeks with <5% student body attendance?
If you exclude low attendance weeks with over 5.00815% student body attendance from the dataset, there is a statistically significant difference between distancing programs.
Key takeaway: what data you exclude can completely change the message of a paper.
Rigor makes research more reliable
P-Hacking data by altering exclusion criteria is a common problem.
Preregistering a study and its exclusion criteria enhances trustworthiness of findings.
Conducting research to justify exclusion criteria enhances trustworthiness of findings.
Taking the time to explore if exciting results could be confounded by boring things leads to better science.
Gunnar Blohm on rigor issues
Rigor issues affect research
Takeaways
Unreliable research can stem from any step of the research process
Scientists are still working to understand the full impact of unreliable research
There is a growing movement to take scientific rigor seriously
2. Motives and Method
Preregistration: Rigor is Worth the Effort
blah blah you should preregister it's very good.
Takeaways
Our motivations for doing science make rigor worth it
While it's a lot of work to be rigorous, rigorous practices can make me a better, more efficient scientist
3. Solving Rigor Problems
Start with an everyday issue
Takeaways
Common problems may result from rigor issues
We can work together to make our lives easier
There are many resources out there to help us improve!
