Curated resources

There are more than 700 resources submitted so far in our database. We are currently curating a new and improved version that is compliant with OER Commons for greater findability, accessibility, interoperability, and reusability (FAIR) of these resources.


If you notice there is an educational resource, research article or pedagocial tool missing in our database, please consider adding it here on FORRT’s resource submission form or via the direct link.


Loading search data...

*

Hail the impossible: p-values, evidence, and likelihood.

Significance testing based on p-values is standard in psychological research and teaching. Typically, research articles and textbooks …

Author(s): Johansson, T.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

1,500 scientists lift the lid on reproducibility

Survey sheds light on the ‘crisis’ rocking research.

Author(s): Monya Baker

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

A 21 word solution.

One year after publishing “False-Positive Psychology,” we propose a simple implementation of disclosure that requires but …

Author(s): Simmons, Joseph P. and Nelson, Leif D. and Simonsohn, Uri, A

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

A Bayesian Perspective on the Reproducibility Project: Psychology

We revisit the results of the recent Reproducibility Project: Psychology by the Open Science Collaboration. We compute Bayes factors—a …

Author(s): Alexander Etz and Joachim Vandekerckhove

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

A collaborative approach to infant research: Promoting reproducibility, best practices, and theory-building.

The ideal of scientific progress is that we accumulate measurements and integrate these into theory, but recent discussion of …

Author(s): Frank et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

A Duty to Describe: Better the Devil You Know Than the Devil You Don’t

Although many researchers have discussed replication as a means to facilitate self-correcting science, in this article, we identify …

Author(s): Sacha D Brown, David Furrow, Daniel F Hill, Jonathon C Gable, Liam P Porter, W Jake Jacobs

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

A manifesto for reproducible science

Improving the reliability and efficiency of scientific research will increase the credibility of the published scientific literature …

Author(s): Munafo, M. R., et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

A multilab preregistered replication of the ego-depletion effect.

Good self-control has been linked to adaptive outcomes such as better health, cohesive personal relationships, success in the workplace …

Author(s): Hagger, M. S., Chatzisarantis, N. L., Alberts, H., Anggono, C. O., Batailler, C., Birt, A. R., ... & Calvillo, D. P.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

A Power Primer

One possible reason for the continued neglect of statistical power analysis in research in the behavioral sciences is the …

Author(s): Jacob Cohen

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

A Powerful Nudge? Presenting Calculable Consequences of Underpowered Research Shifts Incentives Toward Adequately Powered Designs

If psychologists have recognized the pitfalls of underpowered research for decades, why does it persist? Incentives, perhaps: …

Author(s): Will M. Gervais, Jennifer A. Jewell, Maxine B. Najle, and Ben K. L. Ng

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

A proposal for a new editorial policy in the social sciences.

“… there’s this desert prison, see, with an old prisoner, resigned to his life, and a young one just arrived. The young one talks …

Author(s): G. William Walster &T. Anne Cleary

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s):

Link to resource

A Reliability-Generalization Study of Journal Peer Reviews: A Multilevel Meta-Analysis of Inter-Rater Reliability and Its Determinants

Background: This paper presents the first meta-analysis for the inter-rater reliability (IRR) of journal peer reviews. IRR is defined …

Author(s): Lutz Bornmann, Rüdiger Mutz and Hans-Dieter Daniel

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

A Short (Personal) Future History of Revolution 2.0

Crisis of replicability is one term that psychological scientists use for the current introspective phase we are in—I argue instead …

Author(s): Barbara A. Spellman

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

A Social Priming Data Set With Troubling Oddities

A recent paper by Chatterjee, Rose, and Sinha (2013) reported impressively large “money priming” effects: incidental exposure to …

Author(s): Harold Pashler, Doug Rohrer, Ian Abramson, Tanya Wolfson &Christine R. Harris

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

A Software Tool for Removing Patient Identifying Information from Clinical Documents

We created a software tool that accurately removes all patient identifying information from various kinds of clinical data documents, …

Author(s): Friedlin, F. J., & McDonald, C. J.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s):

Link to resource

A survey of the statistical power of research in behavioral ecology and animal behavior.

We estimated the statistical power of the first and last statistical test presented in 697 papers from 10 behavioral journals. First …

Author(s): Jennions, M. D., & Møller, A. P.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Life Science, Math & Statistics

Tag(s):

Link to resource

A systematic review of statistical power in software engineering experiments.

Statistical power is an inherent part of empirical studies that employ significance testing and is essential for the planning of …

Author(s): Dybå, T., Kampenes, V. B., & Sjøberg, D. I.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

A tale of two papers

An abstract about transparency and robustness for two papers

Author(s): Michael Inzlicht

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian, Researcher

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Blog, Reproducibility Crisis and Credibility Revolution

Link to resource

A Unified Framework to Quantify the Credibility of Scientific Findings

Societies invest in scientific studies to better understand the world and attempt to harness such improved understanding to address …

Author(s): Etienne P. LeBel, Randy J. McCarthy, Brian D. Earp, Malte Elson and Wolf Vanpaemel

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Transparency, Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

A vast graveyard of undead theories publication bias and psychological science’s aversion to the null.

Publication bias remains a controversial issue in psychological science. The tendency of psychological science to avoid publishing null …

Author(s): Christopher J. Ferguson, Moritz Heene

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Abandon statistical significance.

We discuss problems the null hypothesis significance testing (NHST) paradigm poses for replication and more broadly in the biomedical …

Author(s): McShane, B. B., Gal, D., Gelman, A., Robert, C., & Tackett, J. L.

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Academic Research in the 21st Century: Maintaining Scientific Integrity in a Climate of Perverse Incentives and Hypercompetition

Over the last 50 years, we argue that incentives for academic scientists have become increasingly perverse in terms of competition for …

Author(s): Marc A. Edwards and Siddhartha Roy

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Accurary of effect size estimates from published psychological research

Monte-Carlo simulation was used to model the biasing of effect sizes in published studies. The findings from the simulation indicate …

Author(s): ANDREW BRAND, MICHAEL T. BRADLEY, LISA A. BEST, AND GEORGE STOICA

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Adjusting for Publication Bias in MetaAnalysis. Perspectives on Psychological Science, 11(5), 730–749.

We review and evaluate selection methods, a prominent class of techniques first proposed by Hedges (1984) that assess and adjust for …

Author(s): McShane, B. B., Böckenholt, U., & Hansen, K. T

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Always use Welch t-test

This is a blogpost describing why we should use Welch t-test instead of Student t-test

Author(s): Daniel Lakens

Type of resources: Reading, R Code

Primary user(s): Student, Teacher, Researcher

Subject area(s): Applied Science, Education, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Blog, Code, Reproducibility Crisis and Credibility Revolution

Link to resource

An agenda for purely confirmatory research.

The veracity of substantive research claims hinges on the way experimental data are collected and analyzed. In this article, we discuss …

Author(s): Wagenmakers, E.-J., Wetzels, R., Borsboom, D., van der Mass, H. L. J., & Kievit, R. A.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

An assessment of the magnitude of effect sizes: Evidence from 30 years of meta-analysis in management.

This study compiles information from more than 250 meta-analyses conducted over the past 30 years to assess the magnitude of reported …

Author(s): Paterson, T. A., Harms, P. D., Steel, P., & Credé, M

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

An exploratory test for an excess of significant findings

Background The published clinical research literature may be distorted by the pursuit of statistically significant results. Purpose: We …

Author(s): John PA Ioannidis and Thomas A Trikalinos

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s):

Link to resource

An Introduction to Registered Replication Reports at Perspectives on Psychological Science

An article about an Introduction to Registered Replication Reports at Perspectives on Psychological Science

Author(s): Daniel J. Simons, Alex O. Holcombe, Barbara A. Spellman

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Analytic Review as a Solution to the Misreporting of Statistical Results in Psychological Science

In this article, we propose analytic review (AR) as a solution to the problem of misreporting statistical results in psychological …

Author(s): John Sakaluk, Alexander Williams, Monica Biernat

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences

This classic text on multiple regression is noted for its nonmathematical, applied, and data-analytic approach. Readers profit from its …

Author(s): Jacob Cohen, Patricia Cohen, Stephen G. West, Leona S. Aiken

Type of resources: Textbook

Primary user(s): Student, Teacher, Researcher

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Book

Link to resource

Are We Wasting a Good Crisis? The Availability of Psychological Research Data after the Storm

To study the availability of psychological research data, we requested data from 394 papers, published in all issues of four APA …

Author(s): Wolf Vanpaemel, Maarten Vermorgen, Leen Deriemaecker and Gert Storms

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

At what sample size do correlations stabilize?

Sample correlations converge to the population value with increasing sample size, but the estimates are often inaccurate in small …

Author(s): Felix D. Schonbrodt and Marco Perugini

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Bad Science blog on the Guardian webpages

Articles about Bad Science

Author(s): Ben Goldacre

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Blog, Reproducibility Knowledge, Open Science

Link to resource

Bad Science log, post-Guardian

Blogs about bad science

Author(s): Ben Goldacre

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Math & Statistics

Tag(s): Blog

Link to resource

Badges to Acknowledge Open Practices: A Simple, Low-Cost, Effective Method for Increasing Transparency

Beginning January 2014, Psychological Science gave authors the opportunity to signal open data and materials if they qualified for …

Author(s): Mallory C. Kidwell et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Bargain Basement Bayes

A blog about bayesian Statistics

Author(s): David Funder

Type of resources: Reading

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Blog

Link to resource

Bayes Factor

Bayes factors are somewhat essential to Bayesian statistics. Tony O’Hagan explains their basics

Author(s): O'Hagan

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Bayes for Beginners: Probability and Likelihood

A blog about Bayes for Beginners: Probability and Likelihood

Author(s): C. Randy Gallistel

Type of resources: Blog

Primary user(s): Student

Subject area(s): Math & Statistics, Social Science

Tag(s):

Link to resource

Bayesian inference

This blog post is about a new statistical visualization. This time I’ve tried to illustrate the logic of bayesian updating and …

Author(s): Kristoffer Magnusson

Type of resources: Interactive, Student Guide, Teaching/Learning Strategy

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Blog, Interaction

Link to resource

Ben Goldacre: Battling Bad Science

A video about battling bad science

Author(s): TED/Ben Goldacre

Type of resources: Video

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Video

Link to resource

Benefits of Open and High-Powered Research Outweigh Costs

Several researchers recently outlined unacknowledged costs of open science practices, arguing these costs may outweigh benefits and …

Author(s): Etienne P. LeBel, Lorne Campbell and Timothy J. Loving

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Best practices in data analysis and sharing in neuroimaging using MRI

Given concerns about the reproducibility of scientific findings, neuroimaging must define best practices for data analysis, results …

Author(s): Thomas E. Nichols et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Life Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Best research practices in psychology: Illustrating epistemological and pragmatic considerations with the case of relationship science.

In recent years, a robust movement has emerged within psychology to increase the evidentiary value of our science. This movement, which …

Author(s): Finkel, E. J., Eastwick, P. W., & Reis, H. T.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Better P-curves: Making P-curve Analysis More Robust to Errors, Fraud, and Ambitious P-hacking, a Reply to Ulrich and Miller (2015)

When studies examine true effects, they generate right-skewed p-curves, distributions of statistically significant results with more …

Author(s): Uri Simonsohn, Joseph P Simmons, Leif D Nelson

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s):

Link to resource

Beyond Power Calculations: Assessing Type S (Sign) and Type M (Magnitude) Errors

Statistical power analysis provides the conventional approach to assess error rates when designing a research study. However, power …

Author(s): Andrew Gelman1 and John Carlin

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Math & Statistics, Social Science

Tag(s):

Link to resource

Big Correlations in Little Studies: Inflated fMRI Correlations Reflect Low Statistical Power-Commentary on Vul Et Al. (2009)

Vul, Harris, Winkielman, and Pashler (2009), (this issue) argue that correlations in many cognitive neuroscience studies are grossly …

Author(s): Tal Yarkoni

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Life Science, Social Science

Tag(s):

Link to resource

Bite-Size Science and Its Undesired Side Effects

Short and rapid publication of research findings has many advantages. However, there is another side of the coin that needs careful …

Author(s): Marco Bertamini, Marcus R. Munafò

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Bradley Efron: Frequentist accuracy of Bayesian estimates

A video about Frequentist accuracy of Bayesian estimates

Author(s): RoyalStatSoc/Bradley Efron

Type of resources: Student Guide, Teaching/Learning Strategy, Video

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Video, Bayesian

Link to resource

Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs

Effect sizes are the most important outcome of empirical studies. Most articles on effect sizes highlight their importance to …

Author(s): Daniel Lakens

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s):

Link to resource

Calculating the overlap of two normal distributions using monte carlo intergration

I read this post over at the blog Cartesian Faith about Probability and Monte Carlo methods. The post describe how to numerically …

Author(s): Kristoffer Magnusson

Type of resources: Reading, Student Guide, Teaching/Learning Strategy, R code

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Blog, Tutorial

Link to resource

Cargo cult science.

some remarks on science, pseudoscience, and learning how to not fool yourself. Caltech’s 1974 commencement address

Author(s): Richard Feynman

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Change starts with journal editors: In response to Makel (2014).

The editors of the Journal of Advanced Academics comment on Makel (2014). The replicability crisis in psychology is summarized in terms …

Author(s): Matthew T. McBee and Michael S. Matthews

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Chaos in the brickyard

A paper about open science and novelty

Author(s): Forscher, B. K.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Choosing prediction over explanation in psychology: Lessons from machine learning.

Psychology has historically been concerned, first and foremost, with explaining the causal mechanisms that give rise to behavior. …

Author(s): Yarkoni, T., & Westfall, J.

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Chrome Book Data Science

Chromebook Data Science (CBDS) is a free, massive open online educational program offered through Leanpub to help anyone who can read, …

Author(s): Jeff Leek, Ashley Johnson, Shannon Ellis, Aboozar Hadavand, John Muschelli, Sean Kross, Leo Collado-Torres, Leah Jager, Sarah McClymont, Leslie Myint

Type of resources: Full Course, Lesson, Module

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Life Science, Math & Statistics, Social Science

Tag(s): Data Science Tool

Link to resource

Circle of Willis (episode with Sinime Vazire)

Welcome to Episode 8, where I talk to SIMINE VAZIRE, Associate Professor of Psychology at the University of California at Davis, about …

Author(s): Jim Coan

Type of resources: Student Guide, Teaching/Learning Strategy, Podcast

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Podcast, Reproducibility Crisis and Credibility Revolution

Link to resource

Cognitive Science StackExchange site (for psychology Q&A)

A website about questions on psychology

Author(s): Anon

Type of resources: Student Guide, Teaching/Learning Strategy

Primary user(s): Student, Teacher, Adminstrator, Parent, Librarian

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Website

Link to resource

Cohen, J. (1992). Statistical power analysis. Current Directions in Psychological Science, 1(3), 98–101.

A paper about statistical power

Author(s): Jacob Cohen

Type of resources: Primary Source, Reading

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Comparing Published Scientific Journal Articlesto Their Pre-print VersionsMart

Academic publishers claim that they add value to scholarly communications by coordinating reviews and contributing and enhancing text …

Author(s): Martin Klein, Peter Broadwell, Sharon E. Farb, Todd Grappone

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Knowledge

Link to resource

Comparison of Registered and Published Primary Outcomes in Randomized Controlled Trials

Context: As of 2005, the International Committee of Medical Journal Editors required investigators to register their trials prior to …

Author(s): Sylvain Mathieu, Isabelle Boutron, David Moher, Douglas G Altman, Philippe Ravaud

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Life Science, Social Science

Tag(s):

Link to resource

Consequences of prejudice against the null hypothesis.

Examined the consequences of prejudice against accepting the null hypothesis through (a) a mathematical model intended to stimulate the …

Author(s): Anthony G. Greenwald

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Conservative tests under satisficing models of publication bias.

Publication bias leads consumers of research to observe a selected sample of statistical estimates calculated by producers of research. …

Author(s): Justin McCrary, Garret Christensen, Daniele Fanelli

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Constraints on generality statements are needed to define direct replication

Whether or not a replication attempt counts as “direct” often cannot be determined definitively after the fact as a result of …

Author(s): Daniel J. Simons, Yuichi Shoda and D. Stephen Lindsay

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Construct validation in social and personality research: Current practice and recommendations.

The verity of results about a psychological construct hinges on the validity of its measurement, making construct validation a …

Author(s): Flake, J. K., Pek, J., & Hehman, E.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Continuously Cumulating Meta-Analysis and Replicability

The current crisis in scientific psychology about whether our findings are irreproducible was presaged years ago by Tversky and …

Author(s): Sanford L. Braver, Felix J. Thoemmes, Robert Rosenthal

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Correcting for Bias in Psychology: A Comparison of Meta-Analytic Methods

Publication bias and questionable research practices in primary research can lead to badly overestimated effects in meta-analysis. …

Author(s): Evan C. Carter, Felix D. Schönbrodt, Will M. Gervais, and Joseph Hilgard

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Correlational Effect Size Benchmarks

Effect size information is essential for the scientific enterprise and plays an increasingly central role in the scientific process. We …

Author(s): Bosco, F. A., Aguinis, H., Singh, K., Field, J. G., & Pierce, C. A.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

course syllabi for open and reproducible science

A collection of course syllabi from any discipline featuring content to examine or improve open and reproducible research practices. …

Author(s): Ball et al.

Type of resources: Reading

Primary user(s): Student, Teacher, Researchers

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Collection, Reproducibility Knowledge

Link to resource

CREP project

CREP’s mission is to provide training, support, and professional growth opportunities for students and instructors completing …

Author(s): Jon Grahe et al.

Type of resources: Primary Source, Reading, Project

Primary user(s): Student, Teacher, Researchers

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Reproducibility Knowledge, OSF Project

Link to resource

CrossValidated StackExchange site (for statistics Q&A)

A website about questions on statistics

Author(s): Anon

Type of resources: Student Guide, Teaching/Learning Strategy, Unit of Study, Website

Primary user(s): Student, Teacher, Librarian

Subject area(s): Math & Statistics

Tag(s): Website

Link to resource

Curating Research Assets: A Tutorial on the Git Version Control System

Recent calls for improving reproducibility have increased attention to the ways in which researchers curate, share, and collaborate on …

Author(s): Matti Vuorre and James P. Curley

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s):

Link to resource

DAGitty

DAGitty is a browser-based environment for creating, editing, and analyzing causal diagrams (also known as directed acyclic graphs or …

Author(s): Johannes Texor, Maciej Liśkiewicz and Benito van der Zander

Type of resources: Teaching/Learning Strategy, R code

Primary user(s): Researcher

Subject area(s): Applied Science, Life Science, Math & Statistics, Social Science

Tag(s): Website

Link to resource

Data peeking without p-hacking

Blog post going over data peeking without p-hacking

Author(s): Daniel Lakens

Type of resources: Reading, R Code

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Education, Life Science, Math & Statistics, Social Science

Tag(s): Blog, Code, Open Science, Reproducibility Crisis and Credibility Revolution

Link to resource

Data reuse and the open data citation advantage

Background. Attribution to the original contributor upon reuse of published data is important both as a reward for data creators and to …

Author(s): Heather A. Piwowar and Todd J. Vision

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s):

Link to resource

Data Sharing in Psychology: A Survey on Barriers and Preconditions

Despite its potential to accelerate academic progress in psychological science, public data sharing remains relatively uncommon. In …

Author(s): Houtkoop et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Data: Sharing Is Caring

Data sharing promotes scientific progress by permitting replication of prior scientific analyses and by increasing the return on the …

Author(s): Margaret C. Levenstein and Jared A. Lyle

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s):

Link to resource

Defining and distinguishing validity: Interpretations of score meaning and justifications of test use.

The concept of validity has suffered because the term has been used to refer to 2 incompatible concerns: the degree of support for …

Author(s): G.J. Cizek

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s):

Link to resource

Degrees of Freedom in Planning, Running, Analyzing, and Reporting Psychological Studies: A Checklist to Avoid p-Hacking

The designing, collecting, analyzing, and reporting of psychological studies entail many choices that are often arbitrary. The …

Author(s): Jelte M. Wicherts, Coosje L. S. Veldkamp, Hilde E. M. Augusteijn, Marjan Bakker, Robbie C. M. van Aert, and Marcel A. L. M. van Assen

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Degrees of Freedom Tutorial

A lot of researchers seem to be struggling with their understanding of the statistical concept of degrees of freedom. Most do not …

Author(s): Ron Dotsch

Type of resources: Reading, Student Guide, Teaching/Learning Strategy

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Education, Life Science, Math & Statistics, Social Science

Tag(s): Blog, Tutorial

Link to resource

Der Umgang mit Forschungsdaten im Fach Psychologie: Konkretisierung der DFG-Leitlinien. [Data Management in Psychological Science: Specification of the DFG Guidelines].

Calls for public access to research data have been ongoing for some time. For instance, in their “Recommendations for Secure Storage …

Author(s): Schönbrodt, F., Gollwitzer, M., & Abele-Brehm, A.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Detecting and avoiding likely false-positive findings: A practical guide

Recently there has been a growing concern that many published research findings do not hold up in attempts to replicate them. We argue …

Author(s): Wolfgang Forstmeier, Eric-Jan Wagenmakers and Timothy H. Parker

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Life Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Detecting fraud in social science

He’s been called a “Data vigilante.” In this episode, Prof. Uri Simonsohn describes how he detects fraudulent work in …

Author(s): Julia Galef/Uri Simonsohn

Type of resources: Teaching/Learning Strategy, Unit of Study, Podcast

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Podcast, Reproducibility Knowledge, Reproducibility Crisis and Credibility Revolution

Link to resource

Distribution of p-values when comparing two groups

An interactive visualisation of the distribution of p-values when comparing two groups

Author(s): Kristoffer Magnusson

Type of resources: Interactive, Reading, Simulation, Student Guide, Teaching/Learning Strategy

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Blog, Interaction, Simulation, Tutorial

Link to resource

Do Statistical Reporting Standards Affect What Is Published? Publication Bias in Two Leading Political Science Journals

We examine the APSR and the AJPS for the presence of publication bias due to reliance on the 0.05 significance level. Our analysis …

Author(s): Alan Gerber and Neil Malhotra

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s):

Link to resource

Do studies of statistical power have an effect on the power of studies?

The long-term impact of studies of statistical power is investigated using J. Cohen’s (1962) pioneering work as an example. We …

Author(s): Sedlmeier, P. & Gigerenzer, G.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Dyadic Data Analysis (Methodology in the Social Sciences)

Interpersonal phenomena such as attachment, conflict, person perception, helping, and influence have traditionally been studied by …

Author(s): David A. Kenny, Deborah A. Kashy, William L. Cook

Type of resources: Textbook

Primary user(s): Student, Teacher, Researcher

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Book

Link to resource

Easy preregistration will beneft any research

An article about easy preregistration will benefit any research.

Author(s): David T. Mellor and Brian Nosek

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s):

Link to resource

Editorial Bias Against Replication Research

why aren’t more replications published? While there are many possible reasons, one simple one could be that journals prefer not …

Author(s): Neuliep, James W. and Rick Crandall

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Effect of open peer review on quality of reviews and on reviewers’recommendations: a randomised trial

Objectives: To examine the effect on peer review of asking reviewers to have their identity revealed to the authors of the paper. …

Author(s): Susan van Rooyen et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Transparency, Open Science

Link to resource

Effect Size and Power in Assessing Moderating Effects of Categorical Variables Using Multiple Regression: A 30-Year Review

The authors conducted a 30-year review (1969-1998) of the size of moderating effects of categorical variables as assessed using …

Author(s): Herman Aguinis, James C Beaty, Robert J Boik and Charles A Pierce

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Effect Size Estimates: Current Use, Calculations, and Interpretation

The Publication Manual of the American Psychological Association (American Psychological Association, 2001, 2010) calls for the …

Author(s): Catherine O. Fritz et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Effect Size Estimation in Neuroimaging.

A central goal of translational neuroimaging is to establish robust links between brain measures and clinical outcomes. Success hinges …

Author(s): Marianne C. Reddan, Martin A. Lindquist and Tor D. Wager

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Effect size guidelines for individual differences researchers

Individual differences researchers very commonly report Pearson correlations between their variables of interest. Cohen (1988) provided …

Author(s): Gilles E.Gignac and Eva T.Szodorai

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Effect sizes and p values: What should be reported and what should be replicated?

Despite publication of many well-argued critiques of null hypothesis testing (NHT), behavioral science researchers continue to rely …

Author(s): ANTHONY G. GREENWALD, RICHARD GONZALEZ, RICHARD J. HARRIS and DONALD GUTHRIE

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics

Tag(s):

Link to resource

Effect Sizes: Why, When, and How to Use Them

The effect size (ES) is the magnitude of a study outcome or research finding, such as the strength of the relationship obtained between …

Author(s): Rosnow, R. L., & Rosenthal, R.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

egative results are disappearing from most disciplines and countries

Concerns that the growing competition for funding and citations might distort science are frequently discussed, but have not been …

Author(s): Daniele Fanelli

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Eight common but false objections to the discontinuation of significance testing in the analysis of research data.

Logically and conceptually, the use of statistical significance testing in the analysis of research data has been thoroughly …

Author(s): Schmidt, J. E. H. F. L., Hunter, J. E., Harlow, L., Mulaik, S., & Steiger, J.

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Empirical Evaluation of Very Large Treatment Effects of Medical Interventions

Context Most medical interventions have modest effects, but occasionally some clinical trials may find very large effects for benefits …

Author(s): Tiago V. Pereira, PhD Ralph I. Horwitz, MD John P. A. Ioannidis, MD, DSc

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Enabling Open-Science Initiatives in Clinical Psychology and Psychiatry Without Sacrificing Patients’ Privacy: Current Practices and Future Challenges

The psychological and psychiatric communities are generating data on an ever-increasing scale. To ensure that society reaps the …

Author(s): Walsh, C., Xia, W., Li, M., Denny, J., Harris, P., & Malin, B

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Enhancing transparency of the research process to increase accuracy of findings: A guide for relationship researchers

The purpose of this paper is to extend to the field of relationship science, recent discussions and suggested changes in open research …

Author(s): LORNE CAMPBELL TIMOTHY J. LOVING ETIENNE P. LEBEL

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Erroneous analyses of interactions in neuroscience: a problem of significance.

In theory, a comparison of two experimental effects requires a statistical test on their difference. In practice, this comparison is …

Author(s): Nieuwenhuis, S., Forstmann, B. U., & Wagenmakers, E. J.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Estimating the reproducibility of psychological science

Reproducibility is a defining feature of science, but the extent to which it characterizes current research is unknown. We conducted …

Author(s): Open Science Collaboration.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Evaluating the R-Index and the P-Curve

This blog evaluates the R-Index and the P-Curve

Author(s): Jeff Hughes

Type of resources: Reading

Primary user(s): Student, Researcher

Subject area(s): Math & Statistics

Tag(s): Blog

Link to resource

Everything Hertz

A podcast about open science and psychology

Author(s): Sound Cloud

Type of resources: Student Guide, Teaching/Learning Strategy, Podcast

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Podcast, Reproducibility Crisis and Credibility Revolution

Link to resource

Evolution of Reporting P Values in the Biomedical Literature, 1990-2015

Importance: The use and misuse of P values has generated extensive debates. Objective: To evaluate in large scale the P values reported …

Author(s): David Chavalarias, Joshua David Wallach, Alvin Ho Ting Li, John P A Ioannidis

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Excess Success for Psychology Articles in the Journal Science

This article describes a systematic analysis of the relationship between empirical data and theoretical conclusions for a set of …

Author(s): Gregory Francis, Jay Tanzman, William J. Matthews

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Expectations for Replications: Are Yours Realistic?

Failures to replicate published psychological research findings have contributed to a “crisis of confidence.” Several …

Author(s): David J Stanley 1, Jeffrey R Spence

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Experiments with More Than One Random Factor: Designs, Analytic Models, and Statistical Power.

Traditional methods of analyzing data from psychological experiments are based on the assumption that there is a single random factor …

Author(s): Judd, C. M., Westfall, J., & Kenny, D. A.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Exploratory Factor Analysis

This book provides a non-mathematical introduction to the underlying theory of Efa and reviews the key decisions that must be made in …

Author(s): Leandre R. Fabrigar, Dueane T. Wegener

Type of resources: Textbook

Primary user(s): Student, Teacher, Researcher

Subject area(s): Applied Science, Life Science, Math & Statistics, Social Science

Tag(s): Book

Link to resource

Facts Are More Important Than Novelty: Replication in the Education Sciences

Despite increased attention to methodological rigor in education research, the field has focused heavily on experimental design and not …

Author(s): Matthew C. Makel and Jonathan A. Plucker

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Faking Science: A True Story of Academic Fraud

A book about Academic Fraud

Author(s): Diederik Stapel/Translated by Nicholas J. L. Brown

Type of resources: Primary Source

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Textbook, Reproducibility Crisis and Credibility Revolution

Link to resource

False Positive Psychology

A lecture on when analysis goes wrong A look at false-positive psychology

Author(s): Dermot Lynott

Type of resources: Lecture, Lecture Notes, Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Life Science, Math & Statistics, Social Science

Tag(s): Lecture, Reproducibility Knowledge

Link to resource

False-Positive Citations

We describe why we wrote “False-Positive Psychology,” analyze how it has been cited, and explain why the integrity of experimental …

Author(s): Joseph P. Simmons, Leif D. Nelson, and Uri Simonsohn

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant

In this article, we accomplish two things. First, we show that despite empirical psychologists’ nominal endorsement of a low rate of …

Author(s): Joseph P. Simmons

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Falsely Reassuring: Analyses of ALL p-values

A blog post that describes analyses of all p-values

Author(s): Uri Simonsohn

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Blog

Link to resource

Falsifiability Is Not Optional

Finkel, Eastwick, and Reis (2016; FER2016) argued the post-2011 methodological reform movement has focused narrowly on replicability, …

Author(s): LeBel, E. P., Berger, D., Campbell, L., & Loving, T. J. (2017).

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Fearing the future of empirical psychology: Bem’s (2011) evidence of psi as a case study in deficiencies in modal research practice.

In this methodological commentary, we use Bem’s (2011) recent article reporting experimental evidence for psi as a case study for …

Author(s): LeBel, E.P., & Peters, K.R.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Feeling the Future: Experimental Evidence for Anomalous Retroactive Influences on Cognition and Affect

The term psi denotes anomalous processes of information or energy transfer that are currently unexplained in terms of known physical or …

Author(s): Daryl J Bem

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Fishing, Commitment, and Communication: A Proposal for Comprehensive Nonbinding Research Registration

Social scientists generally enjoy substantial latitude in selecting measures and models for hypothesis testing. Coupled with …

Author(s): Macartan Humphreys, Raul Sanchez de la Sierra and Peter van der Windt

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

G*Power

GPower is a tool to compute statistical power analyses for many different t tests, F tests, χ2 tests, z tests and some exact tests. G …

Author(s): The G*Power Team

Type of resources: Reading, Simulation

Primary user(s): Student, Teacher, Researcher

Subject area(s): Math & Statistics

Tag(s): Reproducibility Knowledge, Power Analysis Tool

Link to resource

Giving Community Psychology Away: A case for open access publishing

Amidst increased pressure for transparency in science, researchers and community members are calling for open access to study stimuli …

Author(s): Crystal N. Steltenpohl, Amy J. Anderson, and Katherine M. Daniels

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Life Science, Social Science

Tag(s):

Link to resource

GLIMMPSE

Welcome to GLIMMPSE. The GLIMMPSE software calculates power and sample size for study designs with normally distributed outcomes. …

Author(s): Anon

Type of resources: Reading, Simulation

Primary user(s): Student, Teacher, Researcher

Subject area(s): Math & Statistics

Tag(s): Reproducibility Knowledge, Power Analysis Tool

Link to resource

Good Science, Bad Science

A syllabi about open science: good science and bad science.

Author(s): Eric-Jan Wagenmakers

Type of resources: Syllabus

Primary user(s): Teacher

Subject area(s): Applied Science, Education, Life Science, Math & Statistics, Social Science

Tag(s): Syllabus

Link to resource

Goodman, S. N., Fanelli, D., & Ioannidis, J. P. (2016). What does research reproducibility mean?. Science translational medicine, 8(341), 341ps12-341ps12.

The language and conceptual framework of “research reproducibility” are nonstandard and unsettled across the sciences. In this …

Author(s): Goodman, S. N., Fanelli, D., & Ioannidis, J. P.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Graphical Causal Models

This chapter discusses the use of directed acyclic graphs (DAGs) for causal inference in the observational social sciences. It focuses …

Author(s): Felix Elwert

Type of resources: Primary Source, Reading, Chapter

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Guide Your Students to Become Better Research Consumers

Blog post going over making undergraduate students better consumers of research

Author(s): Beth Morling

Type of resources: Reading, Teaching/Learning Strategy

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Education, Life Science, Social Science

Tag(s): Blog, Open Science

Link to resource

Hack Your Way To Scientific Glory

You’re a social scientist with a hunch: The U.S. economy is affected by whether Republicans or Democrats are in office. Try to show …

Author(s): Anon

Type of resources: Activity/Lab, Interactive, Simulation, Teaching/Learning Strategy

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Demo, Reproducibility Knowledge

Link to resource

HARKing: How Badly Can Cherry-Picking and Question Trolling Produce Bias in Published Results?

The practice of hypothesizing after results are known (HARKing) has been identified as a potential threat to the credibility of …

Author(s): Kevin R. Murphy & Herman Aguinis

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

HARKing: Hypothesizing after the results are known.

This article considers a practice in scientific communication termed HARKing (Hypothesizing After the Results are Known). HARKing is …

Author(s): Kerr, N. L.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

How (and Whether) to Teach Undergraduates About the Replication Crisis in Psychological Science

Over the past 10 years, crises surrounding replication, fraud, and best practices in research methods have dominated discussions in the …

Author(s): William J. Chopik, Ryan H. Bremner, Andrew M. Defever, Victor N. Keller

Type of resources: Primary Source, Reading, Paper

Primary user(s): Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution

Link to resource

How scientists can stop fooling themselves

Sampling simulated data can reveal common ways in which our cognitive biases mislead us.

Author(s): Professor Dorothy Bishop

Type of resources: Reading

Primary user(s): Student, Teacher, Researcher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

How to Make More Published Research True

An essay about How to Make More Published Research True

Author(s): John Ioannidis

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

I Fooled Millions Into Thinking Chocolate Helps Weight Loss. Here’s How.

A blog about reproducibility crisis

Author(s): John Bohannon

Type of resources: Primary Source, Reading, Blog

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Identifying Participants in the Personal Genome Project by Name

We linked names and contact information to publicly available profiles in the Personal Genome Project. These profiles contain medical …

Author(s): Sweeney, Latanya, Akua Abu, and Julia Winn.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Improving the Dependability of Research in Personality and Social Psychology: Recommendations for Research and Educational Practice

In this article, the Society for Personality and Social Psychology (SPSP) Task Force on Publication and Research Practices offers a …

Author(s): Funder et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s):

Link to resource

Improving your statistical inferences

This course aims to help you to draw better statistical inferences from empirical research. First, we will discuss how to correctly …

Author(s): Daniel Lakens

Type of resources: Full Course

Primary user(s): Student

Subject area(s): Applied Science, Life Science, Math & Statistics, Social Science

Tag(s): Course

Link to resource

In Praise of the Null Hypothesis Statistical Test

Jacob Cohen (see record 1995-12080-001) raised a number of questions about the logic and information value of the null hypothesis …

Author(s): Richard L. Hagen

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Inappropriate fiddling with statistical analyses to obtain a desirable p-value: tests to detect its presence in published literature.

Much has been written regarding p-values below certain thresholds (most notably 0.05) denoting statistical significance and the …

Author(s): Gary L. Gadbury and David B. Allison

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Increasing transparency through a multiverse analysis.

Empirical research inevitably includes constructing a data set by processing raw data into a form ready for statistical analysis. Data …

Author(s): Steegen, S., Tuerlinckx, F., Gelman, A., & Vanpaemel, W.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Instead of “playing the game” it is time to change the rules: Registered Reports at AIMS Neuroscience and beyond.

The last ten years have witnessed increasing awareness of questionable research practices (QRPs) in the life sciences [1,2], including …

Author(s): Chambers, C. D., Feredoes, E., Muthukumaraswamy, S. D., & Etchells, P.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Intellectual humility is central to science

Transparency is indispensable for accuracy and correction in science, and is discussed frequently in the credibility revolution. A less …

Author(s): Rink Hoekstra and Simine Vazire

Type of resources: Reading

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Internal conceptual replications do not increase independent replication success

Recently, many psychological effects have been surprisingly difficult to reproduce. This article asks why, and investigates whether …

Author(s): Richard Kunert

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Interpreting confidence intervals

A blog about the visualisation of Confidence intervals

Author(s): Kristoffer Magnusson

Type of resources: Interactive, Simulation, Student Guide, Teaching/Learning Strategy, R code

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Blog, Interaction, Simulation, Tutorial

Link to resource

Interpreting Correlations: an interactive visualization

Correlation is one of the most widely used tools in statistics. The correlation coefficient summarizes the association between two …

Author(s): Kristoffer Magnusson

Type of resources: Interactive, Reading, Simulation, Student Guide, Teaching/Learning Strategy

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Blog, Interaction, Simulation, Tutorial

Link to resource

Interpreting effect sizes: Toward a quantitative cumulative social psychology

Improved research practice is based on estimation of effect sizes rather than statistical significance. We discuss the challenging task …

Author(s): ARTHUR A. STUKAS AND GEOFF CUMMING

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Intro to the special issue

A paper about a special issue on cognitive modelling

Author(s): Kevin Glucka, Paul Bellob, Jerome Busemeyer

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s):

Link to resource

Introducing JASP

A blog about JASP to replace SPSS

Author(s): Jonas Haslbeck

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Blog, Reproducibility Knowledge

Link to resource

Introduction to Mediation, Moderation, and Conditional Process Analysis, First Edition: A Regression-Based Approach (Methodology in the Social Sciences)

Explaining the fundamentals of mediation and moderation analysis, this engaging book also shows how to integrate the two using an …

Author(s): Andrew F. Hayes

Type of resources: Textbook

Primary user(s): Student, Teacher, Researcher

Subject area(s): Applied Science, Social Science

Tag(s): Book

Link to resource

Introduction to Meta-Analysis

A book about meta analyses

Author(s): Borenstein, M., Hedges, L.V., Higgins, J. P. T., & Rothstein, H. R. (2009).

Type of resources: Reading

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Investigating Variation in Replicability: A “Many Labs” Replication Project

Although replication is a central tenet of science, direct replications are rare in psychology. This research tested variation in the …

Author(s): Klein et al.

Type of resources: Primary Source, Reading, Reading

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Investigation and its discontents: Some constraints on progress in psychological research.

Examines several prominent trends in the conduct of psychological research and considers how they may limit progress in the field. …

Author(s): Paul L. Wachtel

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Ioannidis, J. P. A. (2012). Why science is not necessarily self-correcting. Perspectives on Psychological Science, 7, 645-654.

The ability to self-correct is considered a hallmark of science. However, self-correction does not always happen to scientific evidence …

Author(s): John Ioannidis

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Is Psychology Suffering From a Replication Crisis? What Does “Failure to Replicate” Really Mean?

Psychology has recently been viewed as facing a replication crisis because efforts to replicate past study findings frequently do not …

Author(s): Scott E. Maxwell, Michael Y. Lau and George S. Howard

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Is the call to abandon p-values the red herring of the replicability crisis?

A paper about Is the call to abandon p-values the red herring of the replicability crisis?

Author(s): Victoria Savalei and Elizabeth Dunn

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Is the replicability crisis overblown? Three arguments examined

We discuss three arguments voiced by scientists who view the current outpouring of concern about replicability as overblown. The first …

Author(s): Pashler, H., & Harris, C. R.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Is there a credibility crisis in strategic management research? Evidence on the reproducibility of study findings

Recent studies report an inability to replicate previously published research, leading some to suggest that scientific knowledge is …

Author(s): Donald D Bergh, Barton M Sharp, Herman Aguinis and Ming Li

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Is There a Free Lunch in Inference?

The field of psychology, including cognitive science, is vexed by a crisis of confidence. Although the causes and solutions are varied, …

Author(s): Jeffrey N Rouder, Richard D Morey, Josine Verhagen, Jordan M Province, Eric-Jan Wagenmakers

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Math & Statistics, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science, Transparency

Link to resource

It’s Time to Broaden the Replicability Conversation: Thoughts for and From Clinical Psychological Science

Psychology is in the early stages of examining a crisis of replicability stemming from several high-profile failures to replicate …

Author(s): Jennifer L. Tackett et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

John Ioannidis: “Reproducible Research: True or False?” | Talks at Google

A video detailing whether reproducible research is true or false

Author(s): John Ioannidis

Type of resources: Student Guide, Teaching/Learning Strategy, Video

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Video

Link to resource

Just Post It: The Lesson From Two Cases of Fabricated Data Detected by Statistics Alone

I argue that requiring authors to post the raw data supporting their published results has the benefit, among many others, of making …

Author(s): Uri Simonsohn

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science

Tag(s):

Link to resource

Justify your alpha

In response to recommendations to redefine statistical significance to P ≤ 0.005, we propose that researchers should transparently …

Author(s): Daniel Lakens et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Lakens, D. (2015). On the challenges of drawing conclusions from p-values just below 0.05. PeerJ, 3, e1142.

In recent years, researchers have attempted to provide an indication of the prevalence of inflated Type 1 error rates by analyzing the …

Author(s): Daniel Lakens

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Last Week Tonight with John Oliver: Scientific Studies (HBO)

A video about scientific studies

Author(s): Last Week Tonight/John Oliver

Type of resources: Student Guide, Teaching/Learning Strategy, Video

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Video

Link to resource

Latent Variable Modeling with R

This book demonstrates how to conduct latent variable modeling (LVM) in R by highlighting the features of each model, their specialized …

Author(s): W. Holmes Finch, Brian F. French

Type of resources: Textbook

Primary user(s): Student, Teacher, Researcher

Subject area(s): Applied Science, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Book

Link to resource

Learn Statistics With R

Back in the grimdark pre-Snapchat era of humanity (i.e. early 2011), I started teaching an introductory statistics class for psychology …

Author(s): Danielle Navarro

Type of resources: Textbook, R Tutorial

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Statistical Book, Tutorial

Link to resource

Learn Stats with Jamovi

A statistical tutorial about using Jamovi

Author(s): Danielle Navarro and David R Foxcraft

Type of resources: Data Set, Textbook, Tutorial

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s): Statistical Book, Tutorial

Link to resource

Let’s Publish Fewer Papers

A paper about publishing few papers

Author(s): Leif D. Nelson, Joseph P. Simmons & Uri Simonsohn

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Let’s Put Our Money Where Our Mouth Is: If Authors Are to Change Their Ways, Reviewers (and Editors) Must Change With Them

A number of scholars recently have argued for fundamental changes in the way psychological scientists conduct and report research. The …

Author(s): Jon K. Maner

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Likelihood of Null Effects of Large NHLBI Clinical Trials Has Increased over Time

Background: We explore whether the number of null results in large National Heart Lung, and Blood Institute (NHLBI) funded trials has …

Author(s): Kaplan RM, Irvin VL

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Life Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science, Transparency

Link to resource

Logical and methodological issues affecting genetic studies of humans reported in top neuroscience journals.

Genetics and neuroscience are two areas of science that pose particular methodological problems because they involve detecting weak …

Author(s): Grabitz, C.R. et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Life Science, Math & Statistics, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Longitudinal Structural Equation Modeling (Methodology in the Social Sciences)

Featuring actual datasets as illustrative examples, this book reveals numerous ways to apply structural equation modeling (SEM) to any …

Author(s): Todd D. Little

Type of resources: Textbook

Primary user(s): Student, Teacher, Researcher

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Book

Link to resource

Making Prospective Registration of Observational Research a Reality

The vast majority of health-related observational studies are not prospectively registered and the advantages of registration have not …

Author(s): Rafael Dal-Ré, John P. Ioannidis, Michael B. Bracken, Patricia A. Buffler, An-Wen Chan, Eduardo L. Franco, Carlo La Vecchia, ElisabeteWeiderpass

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Making replication mainstream

Many philosophers of science and methodologists have argued that the ability to repeat studies and obtain similar results is an …

Author(s): Rolf A. Zwaan, Alexander Etz Richard E. Lucas and M. Brent Donnellan

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Many hands make tight work: Crowdsourcing research can balance discussions, validate findings and better inform policy

Crowdsourcing research can balance discussions, validate findings and better inform policy, say Raphael Silberzahn and Eric L. Uhlmann.

Author(s): Silberzahn, R., & Uhlmann, E.L.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Many Labs 3: Evaluating participant pool quality across the academic semester via replication.

The university participant pool is a key resource for behavioral research, and data quality is believed to vary over the course of the …

Author(s): Charles R. Ebersole et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Maximizing the Reproducibility of Your Research

A chapter to discuss maximising the reproducibility of research

Author(s): Open Science Collaboration

Type of resources: Primary Source, Reading, Chapter

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Measurement error and the replication crisis.

The assumption that measurement error always reduces effect sizes is false

Author(s): Loken, E., & Gelman, A.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Measurement Matters

This resource list contains reading material on the topic of measurement in psychological sciences. We hope the list will be a useful …

Author(s): Eiko Fried and Jessica Flake

Type of resources: Reading

Primary user(s): Student, Researcher

Subject area(s): Social Science

Tag(s): Collection, Reproducibility Knowledge

Link to resource

Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling

Cases of clear scientific misconduct have received significant media attention recently, but less flagrantly questionable research …

Author(s): Leslie K. John, George Loewenstein, Drazen Prelec

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Meta-analyses are no substitute for registered replications: a skeptical perspective on religious priming

According to a recent meta-analysis, religious priming has a positive effect on prosocial behavior (Shariff et al., 2015). We first …

Author(s): Michiel van Elk, Dora Matzke, Quentin F. Gronau, Maime Guan, Joachim Vandekerckhove and Eric-Jan Wagenmakers

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Meta‐regression approximations to reduce publication selection bias.

Publication selection bias is a serious challenge to the integrity of all empirical sciences. We derive meta-regression approximations …

Author(s): TD Stanley, H Doucouliagos

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Methods for Reliable, Transparent, and Open Science

A syllabus about the methods for reliable, transparent and open science

Author(s): Jeffrey Rouder and Joachim Vandekerckhove

Type of resources: Syllabus

Primary user(s): Teacher

Subject area(s): Applied Science, Education, Life Science, Math & Statistics, Social Science

Tag(s): Syllabus

Link to resource

Mindless statistics

Statistical rituals largely eliminate statistical thinking in the social sciences. Rituals are indispensable for identification with …

Author(s): Gerd Gigerenzer

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s):

Link to resource

Mini Meta-Analysis of Your Own Studies: Some Arguments on Why and a Primer on How

We outline the need to, and provide a guide on how to, conduct a meta-analysis on one’s own studies within a manuscript. Although …

Author(s): Jin X. Goh, Judith A. Hall and Robert Rosenthal

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Mission: P-curve

A blog about the p-curve

Author(s): Will Gervais

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Social Science

Tag(s): Blog

Link to resource

Multilevel Analysis: An Introduction to Basic and Advanced Multilevel Modeling

The main methods, techniques and issues for carrying out multilevel modeling and analysis are covered in this book. The book is an …

Author(s): Tom A.B. Snijders, Roel Bosker

Type of resources: Textbook

Primary user(s): Student, Teacher, Researcher

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Book

Link to resource

Negativity towards negative results: a discussion of the disconnect between scientific worth and scientific culture.

Science is often romanticised as a flawless system of knowledge building, where scientists work together to systematically find …

Author(s): Matosin, N., Frank, E., Engel, M., Lum, J. S., & Newell, K. A.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Neuroscientist Explains

Daniel Glaser apprehensively revisits an article of his that saw some fallout due to a study he cited. But that study was not the only …

Author(s): Daniel Glaser

Type of resources: Student Guide, Teaching/Learning Strategy, Podcast

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Podcast, Reproducibility Knowledge, Reproducibility Crisis and Credibility Revolution

Link to resource

Nick Brown talk: replicability and reproducibility

A video about replicability and reproducibility debate

Author(s): BPSOfficial/Nick Brown

Type of resources: Video

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Video, Replication Research

Link to resource

Novel methods to deal with publication biases: secondary analysis of antidepressant trials in the FDA trial registry database and related journal publications.

Objective: To assess the performance of novel contour enhanced funnel plots and a regression based adjustment method to detect and …

Author(s): Santiago G Moreno, Alex J Sutton, Erick H Turner, Keith R Abrams, Nicola J Cooper, Tom M Palmer, A E Ades

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Null Hypothesis Significance Testing. On the Survival of a Flawed Method

Null hypothesis significance testing (NHST) is the researcher’s workhorse for making inductive inferences. This method has often …

Author(s): J Krueger

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Math & Statistics, Social Science

Tag(s):

Link to resource

Null hypothesis significance testing: a review of an old and continuing controversy. Psychological methods, 5(2), 241-301.

Null hypothesis significance testing (NHST) is arguably the most widely used approach to hypothesis evaluation among behavioral and …

Author(s): Nickerson, R. S. (2000).

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Nuzzo, R. (2014). Statistical errors: P values, the ‘gold standard’ of statistical validity, are not as reliable as many scientists assume. Nature, 506(7487), 150-152.

P values, the ‘gold standard’ of statistical validity, are not as reliable as many scientists assume.

Author(s): Nuzzo, R.

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

On the origins of the .05 level of statistical significance

Examination of the literature in statistics and probability that predates Fisher’s Statistical Methods for Research Workers …

Author(s): MICHAEL COWLES and CAROLINE DAVIS

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

On the plurality of (methodological) worlds: estimating the analytic flexibility of fMRI experiments

How likely are published findings in the functional neuroimaging literature to be false? According to a recent mathematical model, the …

Author(s): Carp, J.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

On the reproducibility of meta-analyses: six practical recommendations

Background: Meta-analyses play an important role in cumulative science by combining information across multiple studies and attempting …

Author(s): Daniel Lakens et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

On the reproducibility of psychological science

Investigators from a large consortium of scientists recently performed a multi-year study in which they replicated 100 psychology …

Author(s): Valen E. Johnson, Richard D. Payne, Tianying Wang, Alex Asher &Soutrik Mandal

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

On the Surprising Longevity of Flogged Horses: Why There Is a Case for the Significance Test

Criticisms of null-hypothesis significance tests (NHSTs) are reviewed. Used as formal, two-valued decision procedures, they often …

Author(s): Robert P. Abelson

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics, Social Science

Tag(s):

Link to resource

One cheer for null hypothesis significance testing

Null hypothesis testing as a tool in research is defended. Six examples are offered of situations in which, if all the researcher could …

Author(s): H. Wainer

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Open Science in Sport and Exercise Psychology: Review of Current Approaches and Considerations for Qualitative Inquiry

Open science practices including open access (OA) publication, open methods, study preregistration, and open data are gaining …

Author(s): Tamminen, K.A. and Poucher, Z.A.

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Open science toolbox

There is a vast body of helpful tools that can be used in order to foster Open Science practices. For reasons of clarity, this toolbox …

Author(s): Lutz Heil

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Reproducibility Knowledge

Link to resource

Open science: many hands make light work

Open science has been highlighted as one of the priorities of the Dutch presidency of the European Union in 2016. Matthew Dovey …

Author(s): Jisc

Type of resources: Student Guide, Teaching/Learning Strategy, Podcast

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Podcast, Reproducibility Crisis and Credibility Revolution

Link to resource

Open Science: What, Why, and How

Open Science is a collection of actions designed to make scientific processes more transparent and results more accessible. Its goal is …

Author(s): Bobbie Spellman, Elizabeth Gilbert and Katherine Corker

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Open sharing of data on close relationships and other sensitive social psychological topics: Challenges, tools, and future directions

This article reports on an adversarial (but friendly) collaboration examining the issues that lie at the intersection of …

Author(s): Joel, S., Eastwick, P., & Finkel, E.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

open stats lab

A website about open statistic labs with data and activities

Author(s): Kevin P. McIntyre

Type of resources: Activity/Lab, Data Set

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Website

Link to resource

Opening Science

The Evolving Guide on How the Internet is Changing Research, Collaboration and Scholarly Publishing

Author(s): Sönke Bartling and Sascha Friesike

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Optimizing Research Payoff

In this article, we present a model for determining how total research payoff depends on researchers’ choices of sample sizes, α …

Author(s): Jeff Miller and Rolf Ulrich

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

P Values and Statistical Practice

An article about P Values and Statistical Practice

Author(s): Andrew Gelman

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

P-curve

An abstract about the p-curve

Author(s): Will Gervais

Type of resources: Reading

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Blog

Link to resource

P-curve visualization updated with log x-axis

My p-curve tool now lets you show the x-axis on a log₁₀ scale, which makes it a lot easier to look at really small p-values

Author(s): Kristoffer Magnusson

Type of resources: Interactive, Reading, Student Guide, Teaching/Learning Strategy

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Blog, Interaction

Link to resource

P-curve: A key to the file-drawer.

Because scientists tend to report only studies (publication bias) or analyses (p-hacking) that “work,” readers must ask, “Are these …

Author(s): Simonsohn, Uri, Nelson, Leif D., Simmons, Joseph P.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

P-Hacking and Other Problems in Psychology Research

What’s wrong with the social sciences? In this episode, Massimo and Julia are joined by Professor Daniel Lakens from the …

Author(s): Julia Galef, Massimo Pigliucci and Benny Pollak

Type of resources: Student Guide, Teaching/Learning Strategy, Podcast

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Podcast, Reproducibility Crisis and Credibility Revolution

Link to resource

PANGEA

PANGEA is the first power analysis program for general ANOVA designs (e.g., Winer, Brown, & Michels, 1991). PANGEA can handle …

Author(s): Jake Westfall

Type of resources: Reading, Simulation

Primary user(s): Student, Teacher, Researcher

Subject area(s): Math & Statistics

Tag(s): Reproducibility Knowledge, Power Analysis Tool

Link to resource

Peer-review practices of psychological journals: The fate of published articles, submitted again.

A growing interest in and concern about the adequacy and fairness of modern peer-review practices in publication and funding are …

Author(s): Peters, D. P., & Ceci, S. J.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Open Science, Open Review

Link to resource

Performing high‐powered studies efficiently with sequential analyses

Running studies with high statistical power, while effect size estimates in psychology are often inaccurate, leads to a practical …

Author(s): Daniel Lakens

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Philosophical Psychology 1989

P. E. Meehl did first 10 sessions (Winter Quarter, Jan–Mar 1989). In the Spring Quarter, several other department members lectured on …

Author(s): P. E. Meehl

Type of resources: Primary Source, Student Guide, Teaching/Learning Strategy, Video

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Course, Video

Link to resource

Point of View: How open science helps researchers succeed

Open access, open data, open source and other open scholarship practices are growing in popularity and necessity. However, widespread …

Author(s): Erin C McKiernan et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Power Analysis and Effect Size in Mixed Effects Models: A Tutorial.

In psychology, attempts to replicate published findings are less successful than expected. For properly powered studies replication …

Author(s): Brysbaert, M. and Stevens, M.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Power failure: why small sample size undermines the reliability of neuroscience

A study with low statistical power has a reduced chance of detecting a true effect, but it is less well appreciated that low power also …

Author(s): Katherine Button et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Power Posing: Reassessing The Evidence Behind The Most Popular TED Talk

A recent paper in Psych Science (.pdf) reports a failure to replicate the study that inspired a TED Talk that has been seen 25 million …

Author(s): Joe Simmons and Uri Simonsohn

Type of resources: Reading

Primary user(s): Student

Subject area(s): Education, Social Science

Tag(s): Blog

Link to resource

Power-up: A reanalysis of ‘power failure’ in neuroscience using mixture modeling.

Recently, evidence for endemically low statistical power has cast neuroscience findings into doubt. If low statistical power plagues …

Author(s): Camilla L. Nord, Vincent Valton, John Wood and Jonathan P. Roiser

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Practical Solutions for Sharing Data and Materials From Psychological Research

Widespread sharing of data and materials (including displays and text- and video-based descriptions of experimental procedures) will …

Author(s): Rick O. Gilmore, Joy Lorenzo Kennedy, and Karen E. Adolph

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Open Science

Link to resource

Pre-registration in social psychology—A discussion and suggested template.

Pre-registration of studies before they are conducted has recently become more feasible for researchers, and is encouraged by an …

Author(s): van ‘t Veer, A.E., & Giner-Sorolla, R.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Transparency

Link to resource

Pre-registration of clinical trials is associated with fewer positive findings

Pre-registration of clinical trials is associated with fewer positive findings

Author(s): Bill Gardner

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Life Science, Math & Statistics, Social Science

Tag(s): Blog, Open Science, Reproducibility Crisis and Credibility Revolution

Link to resource

Prediction Interval: What to Expect When You’re Expecting … A Replication

A challenge when interpreting replications is determining whether the results of a replication “successfully” replicate the original …

Author(s): Jeffrey R.Spence and David J. Stanley

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Preregistration Becoming the Norm in Psychological Science

A blog about Preregistration Becoming the Norm in Psychological Science

Author(s): Brian Nosek and Stephen Lindsay

Type of resources: Reading, blog

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Blog

Link to resource

Priming, Replication, and the Hardest Science

Concerns have been raised recently about the replicability of behavioral priming effects, and calls have been issued to identify …

Author(s): Joseph Cesairo

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Privacy and Data-Based Research

What can we, as users of microdata, formally guarantee to the individuals (or firms) in our dataset, regarding their privacy? We retell …

Author(s): Heffetz, Ori, and Katrina Liggett.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Social Science

Tag(s):

Link to resource

Probing Birth-Order Effects on Narrow Traits Using Specification-Curve Analysis

The idea that birth-order position has a lasting impact on personality has been discussed for the past 100 years. Recent large-scale …

Author(s): Julia M Rohrer, Boris Egloff, Stefan C Schmukle

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Problems in using p-curve analysis and text-mining to detect rate of p-hacking.

Background. The p-curve is a plot of the distribution of p-values reported in a set of scientific studies. Comparisons between ranges …

Author(s): Bishop, D. V, & Thompson, P. A.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Project TIER

Project TIER (Teaching Integrity in Empirical Research) promotes the integration of principles and practices related to transparency …

Author(s): TIER Team

Type of resources: Activity/Lab, Asssessment, Homework/Assignment, Lesson Plan, Module, Syllabus

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Framework, Reproducibility Knowledge

Link to resource

Promoting an open research culture

Author guidelines for journals could help to promote transparency, openness, and reproducibility

Author(s): Nosek et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Promoting Transparency in Social Science Research

Social scientists should adopt higher transparency standards to improve the quality and credibility of research.

Author(s): Miguel, E., C. Camerer, K. Casey, J. Cohen, K. M. Esterling, A. Gerber, R. Glennerster, D. P. Green, M. Humphreys, G. Imbens, D. Laitin, T. Madon, L. Nelson, B. A. Nosek, M. Petersen, R. Sedlmayr, J. P. Simmons, U. Simonsohn, M. Van der Laan.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Psych 3400

A statistics book and tutorial about statistics

Author(s): Dr Matt Crump

Type of resources: Data Set, Full Course, Lecture, Lecture Notes, Lesson, Lesson Plan, Module, Reading, Student Guide, Syllabus, Teaching/Learning Strategy, Tutorial, Textbook

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Statistical Book, Tutorial

Link to resource

Psychological testing and psychological assessment: A review of evidence and issues.

This article summarizes evidence and issues associated with psychological assessment. Data from more than 125 meta-analyses on test …

Author(s): Meyer et al.

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science

Tag(s):

Link to resource

Psychologists Are Open to Change, yet Wary of Rules

Psychologists must change the way they conduct and report their research—this notion has been the topic of much debate in recent years. …

Author(s): Heather M. Fuchs, Mirjam Jenny, Susann Fiedler

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Psychology’s renaissance

In 2010–2012, a few largely coincidental events led experimental psychologists to realize that their approach to collecting, analyzing, …

Author(s): Leif D. Nelson, Joseph Simmons, and Uri Simonsohn

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Psychology, Science, and Knowledge Construction: Broadening Perspectives from the Replication Crisis

Psychology advances knowledge by testing statistical hypotheses using empirical observations and data. The expectation is that most …

Author(s): Patrick E. Shrout and Joseph L. Rodgers

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Psychology’s Replication Crisis and the Grant Culture: Righting the Ship

The past several years have been a time for soul searching in psychology, as we have gradually come to grips with the reality that some …

Author(s): Scott O. Lilienfeld

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Publication bias and the canonization of false facts.

Science is facing a “replication crisis” in which many experimental findings cannot be replicated and are likely to be false. Does this …

Author(s): Nissen, S. B., Magidson, T., Gross, K., & Bergstrom, C. T.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Publication bias and the limited strength model of self-control: has the evidence for ego depletion been overestimated?

Few models of self-control have generated as much scientific interest as has the limited strength model. One of the entailments of this …

Author(s): Evan C. Carter and Michael E. McCullough

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Publication Bias in Psychology: A Diagnosis Based on the Correlation between Effect Size and Sample Size

Background: The p value obtained from a significance test provides no information about the magnitude or importance of the underlying …

Author(s): Anton Kuhberger, Astrid Fritz, Thomas Scherndl

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Publication bias in the social sciences: Unlocking the file drawer

We studied publication bias in the social sciences by analyzing a known population of conducted studies—221 in total—in which there is …

Author(s): Annie Franco, Neil Malhotra, Gabor Simonovits

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Publication Decisions and their Possible Effects on Inferences Drawn from Tests of Significance—or Vice Versa

There is some evidence that in fields where statistical tests of significance are commonly used, research which yields nonsignificant …

Author(s): Theodore D. Sterling

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics, Social Science

Tag(s):

Link to resource

Publication Prejudices: An Experimental Study of Confirmatory Bias in the Peer Review System

Confirmatory bias is the tendency to emphasize and believe experiences which support one’s views and to ignore or discredit those …

Author(s): Michael J. Mahoney

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Quality Uncertainty Erodes Trust in Science

When consumers of science (readers and reviewers) lack relevant details about the study design, data, and analyses, they cannot …

Author(s): S. Vazire

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Questionable Research Practices Revisited

The current discussion of questionable research practices (QRPs) is meant to improve the quality of science. It is, however, important …

Author(s): Klaus Fiedler and Norbert Schwarz

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

R Graphics Cookbook: Practical Recipes for Visualizing Data

This practical guide provides more than 150 recipes to help you generate high-quality graphs quickly, without having to comb through …

Author(s): Winston Chang

Type of resources: Textbook

Primary user(s): Student, Teacher, Researcher

Subject area(s): Applied Science, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Book, Software, R

Link to resource

Raise standards for preclinical cancer research

C. Glenn Begley and Lee M. Ellis propose how methods, publications and incentives must change if patients are to benefit.

Author(s): C. Glenn Begley and Lee M. Ellis

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Life Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Randomization Does Not Help Much, Comparability Does

According to R.A. Fisher, randomization “relieves the experimenter from the anxiety of considering innumerable causes by which the data …

Author(s): Uwe Saint-Mont

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s):

Link to resource

Recommendations for increasing replicability in psychology

Replicability of findings is at the heart of any empirical science. The aim of this article is to move the current replicability debate …

Author(s): Asendorpf et al.

Type of resources: Primary Source, Reading, Chapter

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Reconceptualizing replication as a sequence of different studies: A replication typology

In contrast to the truncated view that replications have only a little to offer beyond what is already known, we suggest a broader …

Author(s): Joachim Hüffmeier, Jens Mazei and Thomas Schultze

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Redefine statistical signifcance

We propose to change the default P-value threshold for statistical signifcance from 0.05 to 0.005 for claims of new discoveries.

Author(s): Daniel J. Benjamin

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Registered replication report: Hart & Albarracín (2011).

Language can be viewed as a complex set of cues that shape people’s mental representations of situations. For example, people think of …

Author(s): Eerland, A.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Transparency, Open Science, Registered Reports

Link to resource

Registered Replication Report: Rand, Greene, and Nowak (2012).

In an anonymous 4-person economic game, participants contributed more money to a common project (i.e., cooperated) when required to …

Author(s): Bouwmeester, S., Verkoeijen, P. P., Aczel, B., Barbosa, F., Bègue, L., Brañas-Garza, P., ... & Evans, A. M.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Registered replication report: Schooler and engstler-schooler (1990).

Trying to remember something now typically improves your ability to remember it later. However, after watching a video of a simulated …

Author(s): Alogna, V. K et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Transparency, Open Science, Registered Reports

Link to resource

Registered Replication Report: Strack, Martin, & Stepper (1988)

According to the facial feedback hypothesis, people’s affective responses can be influenced by their own facial expression (e.g., …

Author(s): Wagenmakers et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Registered Replication Report: Study 1 From Finkel, Rusbult, Kumashiro, & Hannon (2002).

Finkel, Rusbult, Kumashiro, and Hannon (2002, Study 1) demonstrated a causal link between subjective commitment to a relationship and …

Author(s): Cheung, I., Campbell, L., LeBel, E. P., Ackerman, R. A., Aykutoğlu, B., Bahník, Š., ... & Carcedo, R. J.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Registered reports : a method to increase the credibility of published results

Registered reports a Method to Increase the Credibility of Published Results

Author(s): Brian Nosek and Daniel Lakens

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Registered Reports: A New Publishing Initiative at Cortex

An article about registered Reports: A new publishing initiative at Cortex

Author(s): Christopher Chambers

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Replacing p-values with Bayes-Factors: A Miracle Cure for the Replicability Crisis in Psychological Science

A blog post about choosing Bayes Factor over p-value

Author(s): Ulrich Schimmack

Type of resources: Reading, Student Guide

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Blog

Link to resource

Replicating Studies in Which Samples of Participants Respond to Samples of Stimuli

In a direct replication, the typical goal is to reproduce a prior experimental result with a new but comparable sample of participants …

Author(s): Jacob Westfall, Charles M. Judd, David A. Kenny

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution

Link to resource

Replication and Open Science for Undergraduates

Blog post going over the replication crisis and how it has led to the open science movement.

Author(s): David Funder

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Life Science, Math & Statistics, Social Science

Tag(s): Blog, Open Science, Reproducibility Crisis and Credibility Revolution

Link to resource

Replication and Robustness in Developmental Research

Replications and robustness checks are key elements of the scientific method and a staple in many disciplines. However, leading …

Author(s): Greg J Duncan, Mimi Engel, Amy Claessens, Chantelle J Dowsett

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Replication and the Manufacture of Scientific Inferences: A Formal Approach

The field of replication studies remains a controversial, misunderstood, and unappreciated piñata of 18 replication typologies spanning …

Author(s): Fernando Martel García

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Replication in Psychology: a historical perspective

The reproducibility of psychological findings has generated much discussion of late. However, the question of replication is not a new …

Author(s): Michael Pettit

Type of resources: Interactive

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Blog, Reproducibility Knowledge

Link to resource

Replication, Communication, and the Population Dynamics of Scientific Discovery

Many published research results are false (Ioannidis, 2005), and controversy continues over the roles of replication and publication …

Author(s): Richard McElreath, Paul E. Smaldino

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Replications in Psychology Research: How Often Do They Really Occur?

Recent controversies in psychology have spurred conversations about the nature and quality of psychological research. One topic …

Author(s): Matthew C. Makel, Jonathan A. Plucker, and Boyd Hegarty

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Reply to Uri Simonsohn’s Critique of Default Bayesian Tests

A blog that summarises a reply to Uri Simonsohn’s Critique of Default Bayesian Tests

Author(s): Jeff Rouder

Type of resources: Reading

Primary user(s): Student, Teacher, Researcher

Subject area(s): Life Science, Math & Statistics, Social Science

Tag(s): Blog

Link to resource

Reporting Effect Sizes in Original Psychological Research: A Discussion and Tutorial

Statistical practice in psychological science is undergoing reform which is reflected in part by strong recommendations for reporting …

Author(s): Jolynn Pek and David B Flora

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

ReproducibiliTea

Serving mugs of Reproducibili☕️: Blends include transparency, openness and robustness + a spoonful of science.

Author(s): Sophia Cruwell, Amy Orben and Sam Parsons

Type of resources: Student Guide, Unit of Study

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Podcast, Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Reproducible research in sport and exercise psychology: The role of sample sizes.

Objectives: We aim to introduce the discussion on the crisis of confidence to sport and exercise psychology. We focus on an important …

Author(s): Schweizer, G., & Furley, P.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Social Science

Tag(s):

Link to resource

Research method syllabi

A syllabus about evaluating research methods

Author(s): Morton-Ann Gernsbacher

Type of resources: Syllabus

Primary user(s): Teacher

Subject area(s): Applied Science, Education, Social Science

Tag(s): Syllabus

Link to resource

Research Practices That Can Prevent an Inflation of False-Positive Rates

Recent studies have indicated that research practices in psychology may be susceptible to factors that increase false-positive rates, …

Author(s): Kou Murayama, Reinhard Pekrun, Klaus Fiedler

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Research preregistration 101.

A blog about pre-registration

Author(s): Lindsay, D.S., Simons, D.J., Lilienfeld, S.O.

Type of resources: Blog

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Blog

Link to resource

Researchers’ Intuitions About Power in Psychological Research

Many psychology studies are statistically underpowered. In part, this may be because many researchers rely on intuition, rules of …

Author(s): Bakker et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s):

Link to resource

Resources (tutorials, papers, analysis scripts, utilities) for testing moderation and mediation

A website about moderation and mediation

Author(s): Kristopher J. Preacher

Type of resources: Lesson

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Website

Link to resource

Response to Comment on “Estimating the reproducibility of psychological science”

Gilbert et al. conclude that evidence from the Open Science Collaboration’s Reproducibility Project: Psychology indicates high …

Author(s): Christopher J. Anderson et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Retraction Watch: Michael LaCour archives

How easy is it to change people’s minds? In 2014, a Science study suggested that a short conversation could have a lasting impact on …

Author(s): Alison McCook

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Education, Math & Statistics, Physical Science, Social Science

Tag(s): Blog

Link to resource

Reviewer Bias Against Replication Research

Social science journal reviewers (N=8) responded to questionnaires regarding their reviewing history, and attitudes towards and …

Author(s): Neuliep, J W; Crandall, R.

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Rewarding Replications: A Sure and Simple Way to Improve Psychological Science

Although replications are vital to scientific progress, psychologists rarely engage in systematic replication efforts. In this article, …

Author(s): Sander L Koole, Daniël Lakens

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

RIPOSTE: A Framework for Improving the Design and Analysis of Laboratory-Based Research

Lack of reproducibility is an ongoing problem in some areas of the biomedical sciences. Poor experimental design and a failure to …

Author(s): Masca et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Safeguard Power as a Protection Against Imprecise Power Estimates

An essential first step in planning a confirmatory or a replication study is to determine the sample size necessary to draw …

Author(s): Marco Perugini, Marcello Gallucci, and Giulio Costantini

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Sailing From the Seas of Chaos Into the Corridor of Stability: Practical Recommendations to Increase the Informational Value of Studies

Recent events have led psychologists to acknowledge that the inherent uncertainty encapsulated in an inductive science is amplified by …

Author(s): Daniel Lakens and Ellen R K Evers

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Sample Size in Psychological Research Over the Past 30 Years

The American Psychological Association (APA) Task Force on Statistical Inference was formed in 1996 in response to a growing body of …

Author(s): Jacob M Marszalek, Carolyn Barber, Julie Kohlhart, Cooper B Holmes

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Sample Size Planning for Statistical Power and Accuracy in Parameter Estimation

This review examines recent advances in sample size planning, not only from the perspective of an individual researcher, but also with …

Author(s): Scott E. Maxwell, Ken Kelley and Joseph R. Rausch

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Sample Size Planning for the Standardized Mean Difference: Accuracy in Parameter Estimation via Narrow Confidence Intervals

Methods for planning sample size (SS) for the standardized mean difference so that a narrow confidence interval (CI) can be obtained …

Author(s): Ken Kelley 1, Joseph R Rausch

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics, Social Science

Tag(s):

Link to resource

Sample-size planning for more accurate statistical power: A method adjusting sample effect sizes for publication bias and uncertainty.

The sample size necessary to obtain a desired level of statistical power depends in part on the population value of the effect size, …

Author(s): Anderson, S. F., Kelley, K., & Maxwell, S. E.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Sangnier, M., & Zylberberg, Y. (2016). Star wars: The empirics strike back. American Economic Journal: Applied Economics, 8(1), 1-32.

Using 50,000 tests published in the AER, JPE, and QJE, we identify a residual in the distribution of tests that cannot be explained …

Author(s): Sangnier, M., & Zylberberg, Y.

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Scanning the horizon: towards transparent and reproducible neuroimaging research

Functional neuroimaging techniques have transformed our ability to probe the neurobiological basis of behaviour and are increasingly …

Author(s): Russell A. Poldrack, Chris I. Baker, Joke Durnez, Krzysztof J. Gorgolewski, Paul M. Matthews, Marcus R. Munafò, Thomas E. Nichols, Jean-Baptiste Poline, Edward Vul & Tal Yarkoni

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Life Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Schmidt, S. (2009). Shall we really do it again? The powerful concept of replication is neglected in the social sciences. Review of General Psychology, 13(2), 90–100.

Replication is one of the most important tools for the verification of facts within the empirical sciences. A detailed examination of …

Author(s): Stefan Schmidt

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Science and Pseudoscience BBC Radio Talk

A podcast about science and pseudoscience

Author(s): London School of Economics and Political Science

Type of resources: Student Guide, Teaching/Learning Strategy, Podcast

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Podcast, Reproducibility Crisis and Credibility Revolution

Link to resource

Science or art? How aesthetic standards grease the way through the publication bottleneck but undermine science.

The current crisis in psychological research involves issues of fraud, replication, publication bias, and false positive results. I …

Author(s): Giner-Sorolla, R.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

SCIENTIFIC APOPHENIA IN STRATEGIC MANAGEMENT RESEARCH: SIGNIFICANCE TESTS & MISTAKEN INFERENCE

This article uses distributional matching and posterior predictive checks to estimate the extent of false and inflated findings in …

Author(s): BRENT GOLDFARB and ANDREW A. KING

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Scientific utopia II. Restructuring incentives and practices to promote truth over publishability.

An academic scientist’s professional success depends on publishing. Publishing norms emphasize novel, positive results. As such, …

Author(s): Nosek, B. A., Spies, J. R., & Motyl, M.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Scientific Utopia: I. Opening Scientific Communication

Existing norms for scientific communication are rooted in anachronistic practices of bygone eras making them needlessly inefficient. We …

Author(s): Brian A. Nosek & Yoav Bar-Anan

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Scientists’ Reputations Are Based on Getting It Right, Not Being Right

Replication is vital for increasing precision and accuracy of scientific claims. However, when replications “succeed” or “fail,” they …

Author(s): Charles R. Ebersole, Jordan R. Axt, Brian A. Nosek

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Short R script to plot effect sizes (Cohen’s d) and share overlapping are

This blog describes how to plot effect sizes (Cohen’s d) and shade overlapping area with R scripts

Author(s): Kristoffer Magnusson

Type of resources: Reading, R code

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Blog, Tutorial

Link to resource

Short, Sweet, and Problematic? The Rise of the Short Report in Psychological Science

Our field has witnessed a rapid increase in the appeal and prevalence of the short report format over the last two decades. In this …

Author(s): Alison Ledgerwood, Jeffrey W. Sherman

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Significance

Visual comics about significance testing

Author(s): Randall Munroe

Type of resources: Diagram/Illustration

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Life Science, Math & Statistics, Social Science

Tag(s): Comic

Link to resource

Significance tests have their place

Null-hypothesis significance tests (NHST), properly used, tell us whether we have sufficient evidence to be confident of the sign of …

Author(s): Richard J. Harris

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Small telescopes Detectability and the evaluation of replication results.

This article introduces a new approach for evaluating replication results. It combines effect-size estimation with hypothesis testing, …

Author(s): Uri Simonsohn

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Social, Behavioral, and Economic Sciences Perspectives on Robust and Reliable Science

A paper about social, behavioural economic perspective on reproducible science

Author(s): Cacioppo, J. T., Kaplan, R. M., Krosnick, J. A., Olds, J. L., & Dean, H.

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

SPSP experts - open science

A video about open science, pre-registration etc.

Author(s): Society for Social and Personality Psychology

Type of resources: Teaching/Learning Strategy, Video

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Video, Reproducibility Knowledge

Link to resource

Spurious Correlations

A website detailing spurious correlations

Author(s): Tyler Vigen

Type of resources: Teaching/Learning Strategy

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Education, Math & Statistics, Social Science

Tag(s): Website

Link to resource

Standard Operating Procedures: A Safety Net for Pre-Analysis Plans

Across the social sciences, growing concerns about research transparency have led to calls for pre-analysis plans (PAPs) that specify …

Author(s): Winston Lin and Donald P.Green

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Stat 545

This site is about everything that comes up during data analysis except for statistical modelling and inference. This might strike you …

Author(s): Jenny Bryan

Type of resources: Reading, Student Guide, Tutorial

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Tutorial

Link to resource

Statistical Evidence: A Likelihood Paradigm

Interpreting statistical data as evidence, Statistical Evidence: A Likelihood Paradigm focuses on the law of likelihood, fundamental to …

Author(s): Richard Royall

Type of resources: Textbook

Primary user(s): Student, Teacher, Researcher

Subject area(s): Math & Statistics

Tag(s): Book

Link to resource

Statistical methods in psychology journals: Guidelines and explanations.

In the light of continuing debate over the applications of significance testing in psychology journals and following the publication of …

Author(s): Wilkinson, L., & Task Force on Statistical Inference, American Psychological Association, Science Directorate.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Statistical Modeling, Causal Inference, and Social Science

A blog about statistics and open science

Author(s): Andrew Gelman

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Blog, Reproducibility Knowledge

Link to resource

Statistical power and optimal design in experiments in which samples of participants respond to samples of stimuli

Researchers designing experiments in which a sample of participants responds to a sample of stimuli are faced with difficult questions …

Author(s): Jacob Westfall, David A Kenny, Charles M Judd

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Statistical Power and the Testing of Null Hypotheses: A Review of Contemporary Management Research and Recommendations for Future Studies

The purpose of this study is to determine how well contemporary management research fares on the issue of statistical power with regard …

Author(s): Luke H. Cashen, Scott W. Geiger

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Statistical power in operations management research

This paper discusses the need and importance of statistical power analysis in field-based empirical research in Production and …

Author(s): Rohit Verma and John C. Goodale

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Statistical Power of Psychological Research: What Have We Gained in 20 Years?

Power was calculated for 6,155 statistical tests in 221 journal articles published in the 1982 volumes of the Journal of Abnormal …

Author(s): J S Rossi

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Statistical Power Problems with Moderated Multiple Regression in Management Research

Due to the increasing importance of moderating (i.e., interaction) effects, the use of moderated multiple regression (MMR) has become …

Author(s): Herman Aguinis

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Statistical Power, Sample Size, and Their Reporting in Randomized Controlled Trials

Objective. —To describe the pattern over time in the level of statistical power and the reporting of sample size calculations in …

Author(s): David Moher, MSc; Corinne S. Dulberg, PhD, MPH; George A. Wells, PhD

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics, Social Science

Tag(s):

Link to resource

Statistical Procedures and the Justification of Knowledge in Psychological Science

Justification, in the vernacular language of philosophy of science, refers to the evaluation, defense, and confirmation of claims of …

Author(s): Rosnow, R.L., & Rosenthal, R.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Statistical Reporting Errors and Collaboration on Statistical Analyses in Psychological Science

Statistical analysis is error prone. A best practice for researchers using statistics would therefore be to share data among …

Author(s): Coosje L. S. Veldkamp, 1, * Michèle B. Nuijten, Linda Dominguez-Alvarez, Marcel A. L. M. van Assen, and Jelte M. Wicherts

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s):

Link to resource

Statistical Rethinking Winter 2015

A video about Statistical Rethinking: A Bayesian Course with R

Author(s): Richard McElreath

Type of resources: Lecture, Teaching/Learning Strategy, Video

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Video, Bayesian

Link to resource

Statistical Significance and the Dichotomization of Evidence

In light of recent concerns about reproducibility and replicability, the ASA issued a Statement on Statistical Significance and …

Author(s): Blakeley B. McShane & David Gal

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Statistical significance in psychological research.

MOST THEORIES IN THE AREAS OF PERSONALITY, CLINICAL, AND SOCIAL PSYCHOLOGY PREDICT ONLY THE DIRECTION OF A CORRELATION, GROUP …

Author(s): Lykken

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s):

Link to resource

Statistical significance testing and cumulative knowledge in psychology: implications for training researchers.

Data analysis methods in psychology still emphasize statistical significance testing, despite numerous articles demonstrating its …

Author(s): F.L. Schmidt

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Statistical tests, p values, confidence intervals, and power: a guide to misinterpretations.

Misinterpretation and abuse of statistical tests, confidence intervals, and statistical power have been decried for decades, yet remain …

Author(s): Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Stereotype threat

A podcast about stereotype threat and replication

Author(s): Simon Adler, Amanda Aronczyk and Dan Engber

Type of resources: Teaching/Learning Strategy, Podcast

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Podcast, Reproducibility Knowledge, Reproducibility Crisis and Credibility Revolution

Link to resource

Strong inference

certain systematic methods of scientific thinking may produce much more rapid progress than others.

Author(s): John R Platt

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Surrogate Science: The Idol of a Universal Method for Scientific Inference

The application of statistics to science is not a neutral act. Statistical tools have shaped and were also shaped by its objects. In …

Author(s): Gerd Gigerenzer and Julian N. Marewski

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s):

Link to resource

SWIRL

Swirl teaches you R programming and data science interactively, at your own pace, and right in the R console!

Author(s): Anon

Type of resources: Activity/Lab, Homework/Assignment, Student Guide, Teaching/Learning Strategy, Tutorial

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics

Tag(s): Tutorial

Link to resource

teaching psych science

A collection of activities to teach APA writing and statistics

Author(s): Lewandowski, Jr., Ciarocco and Strohmetz

Type of resources: Activity/Lab, Asssessment

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics, Social Science

Tag(s):

Link to resource

Teaching replication

Replication is held as the gold standard for ensuring the reliability of published scientific literature. But conducting direct …

Author(s): Michael C. Frank and Rebecca Saxe

Type of resources: Primary Source, Reading, Paper

Primary user(s): Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution

Link to resource

Teaching Replication in Psychology: A Guide for Teachers and Students

This symposium explores the “replication crisis” from the perspective of teachers and students. Presenters will describe the major …

Author(s): Bradford Wiggins

Type of resources: Lecture, Lecture Notes, Syllabus

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): OSF Project

Link to resource

Teaching reproducible science hack-a-thon from 2018 SIPS

Participants will develop materials for teaching replicability and reproducible science. Possible materials to be generated include …

Author(s): Lane et al.

Type of resources: Lesson, Teaching/Learning Strategy

Primary user(s): Student

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Reproducibility Knowledge

Link to resource

Teaching resources spreadsheet

An excel spreadsheet about collection of open science items

Author(s): Courtney Soderberg

Type of resources: Reading

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Reproducibility Knowledge

Link to resource

Ten Simple Rules for Effective Statistical Practice

A paper about Ten Simple Rules for Effective Statistical Practice

Author(s): Robert E.Kass et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Ten Simple Rules for the Care and Feeding of Scientific Data

A paper about ten Simple Rules for the Care and Feeding of Scientific Data

Author(s): Goodman, Alyssa, et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The (mis)reporting of statistical results in psychology journals

In order to study the prevalence, nature (direction), and causes of reporting errors in psychology, we checked the consistency of …

Author(s): Marjan Bakker & Jelte M. Wicherts

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s):

Link to resource

The 52 symptoms of major depression: Lack of content overlap among seven common depression scales

Depression severity is assessed in numerous research disciplines, ranging from the social sciences to genetics, and used as a dependent …

Author(s): Eiko I.Fried

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The Amazing Significo: why researchers need to understand poker

A post that describes significance values

Author(s): Dorothy Bishop

Type of resources: Reading

Primary user(s): Student, Teacher, Researcher

Subject area(s): Education, Life Science, Math & Statistics, Social Science

Tag(s): Blog, Reproducibility Crisis and Credibility Revolution

Link to resource

The appropriate use of null hypothesis testing.

The many criticisms of null hypothesis testing suggest when it is not useful and what is should not be used for. This article explores …

Author(s): R.W. Frick

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics

Tag(s):

Link to resource

The ASA Statement on p-Values: Context, Process, and Purpose

An editorial about p value

Author(s): Ronald L. Wasserstein & Nicole A. Lazar

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

The Baby Factory: Difficult Research Objects, Disciplinary Standards, and the Production of Statistical Significance

Science studies scholars have shown that the management of natural complexity in lab settings is accomplished through a mixture of …

Author(s): David Peterson

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The Bayes Factor

In this episode JP and Alex interview Zoltan Dienes. They discuss Zoltan’s passion for the martial arts, why Bayesian inference …

Author(s): Prof. JP De Ruiter

Type of resources: Student Guide, Teaching/Learning Strategy, Podcast

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Podcast

Link to resource

The Bayesfactor blog

Blog about Bayesfactor and statistics

Author(s): Richard Morey

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Blog

Link to resource

The Bayesian Reproducibility Project

An abstract about bayesian reproducibility project

Author(s): Alexander Etz

Type of resources: Data Set, Reading, Simulation, R code

Primary user(s): Student, Teacher, Researcher

Subject area(s): Applied Science, Life Science, Math & Statistics, Social Science

Tag(s): Blog, R code

Link to resource

The Black Goat

Three psychologists talk about doing science. Hosted by Sanjay Srivastava, Alexa Tullett, and Simine Vazire.

Author(s): Sanjay Srivastava, Alexa Tullett, and Simine Vazire

Type of resources: Student Guide, Unit of Study, Podcast

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Podcast, Reproducibility Crisis and Credibility Revolution

Link to resource

The case against statistical significance testing.

In recent years the use of traditional statistical methods in educational research has increasingly come under attack. In this article, …

Author(s): Ronald P Carver

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

The Chrysalis Effect: How Ugly Initial Results Metamorphosize Into Beautiful Articles

The issue of a published literature not representative of the population of research is most often discussed in terms of entire studies …

Author(s): Ernest Hugh O’Boyle, Jr., George Christopher Banks, Erik Gonzalez-Mulé

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The crisis of confidence in social psychology.

Notes that social psychologists' early enthusiasm has been replaced by serious doubts about the future of their field. Difficulties in …

Author(s): A.C. Elms

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The default bayesian test is prejudiced against small effects

When considering any statistical tool I think it is useful to answer the following two practical questions: 1. “Does it give …

Author(s): Uri Simonsohn

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Blog

Link to resource

The earth is round (p < .05).

After 4 decades of severe criticism, the ritual of null hypothesis significance testing (mechanical dichotomous decisions around a …

Author(s): Jacob Cohen

Type of resources: Primary Source, Reading

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s):

Link to resource

The effect of horizontal eye movements on free recall: A preregistered adversarial collaboration.

A growing body of research has suggested that horizontal saccadic eye movements facilitate the retrieval of episodic memories in free …

Author(s): Matzke et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science, Transparency

Link to resource

The empirical benefits of conceptual rigor: Systematic articulation of conceptual hypotheses can reduce the risk of non-replicable results (and facilitate novel discoveries too)

Most discussions of rigor and replication focus on empirical practices (methods used to collect and analyze data). Typically overlooked …

Author(s): Mark Schaller

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The Empirical March: Making Science Better at Self-Correction

Psychology has been criticized recently for a range of research quality issues. The current article organizes these problems around the …

Author(s): Matthew Makel

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The existence of publication bias and risk factors for its occurrence.

Publication bias is the tendency on the parts of investigators, reviewers, and editors to submit or accept manuscripts for publication …

Author(s): Kay Dickersin

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The Experiment Experiment

A few years back, a famous psychologist published a series of studies that found people could predict the future — not all the time, …

Author(s): Planet Money

Type of resources: Student Guide, Teaching/Learning Strategy, Podcast

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Podcast, Reproducibility Knowledge, Reproducibility Crisis and Credibility Revolution

Link to resource

The Extent and Consequences of P-Hacking in Science.

A focus on novel, confirmatory, and statistically significant results leads to substantial bias in the scientific literature. One type …

Author(s): Head, M. L., Holman, L., Lanfear, R., Kahn, A. T., & Jennions, M. D.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

The fickle P value generates irreproducible results

The reliability and reproducibility of science are under scrutiny. However, a major cause of this lack of repeatability is not being …

Author(s): Lewis G Halsey, Douglas Curran-Everett, Sarah L Vowler & Gordon B Drummond

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

The file drawer problem and tolerance for null results.

For any given research area, one cannot tell how many studies have been conducted but never reported. The extreme view of the …

Author(s): R.Rosenthal

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The frequency of excess success for articles in Psychological Science.

Recent controversies have questioned the quality of scientific practice in the field of psychology, but these concerns are often based …

Author(s): Gregory Francis

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The garden of forking paths: Why multiple comparisons can be a problem, even when there is no “fishing expedition” or “p-hacking” and the research hypothesis was posited ahead of time

Data-dependent analysis—a “garden of forking paths”— explains why many statistically significant comparisons don’t hold up.

Author(s): Andrew Gelman and Eric Loken

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

The Generalizability of Survey Experiments

Survey experiments have become a central methodology across the social sciences. Researchers can combine experiments’ causal power with …

Author(s): Kevin J. Mullinix et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The GRIM Test: A Simple Technique Detects Numerous Anomalies in the Reporting of Results in Psychology

We present a simple mathematical technique that we call granularity-related inconsistency of means (GRIM) for verifying the summary …

Author(s): Nicholas J. L. Brown, James A. J. Heathers

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The Hardest Science

Blogposts about psychology, reproducibility, replication etc.

Author(s): Sanjay Srivastava

Type of resources: Reading

Primary user(s): Student, Researcher

Subject area(s): Math & Statistics, Social Science

Tag(s): Blog, Reproducibility Knowledge

Link to resource

The harm done by tests of significance.

Three historical episodes in which the application of null hypothesis significance testing (NHST) led to the mis-interpretation of data …

Author(s): Ezra Hauer

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s):

Link to resource

The Hidden Brain

A podcast about replication crisis

Author(s): Shankar Vedantam and Maggie Penman

Type of resources: Student Guide, Teaching/Learning Strategy, Podcast

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Podcast, Reproducibility Knowledge, Reproducibility Crisis and Credibility Revolution

Link to resource

The ironic effect of significant results on the credibility of multiple-study articles.

Cohen (1962) pointed out the importance of statistical power for psychology as a science, but statistical power of studies has not …

Author(s): Schimmack U

Type of resources: Primary Source, Reading, Reading

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

The meaning of “significance” for different types of research

Adrianus Dingeman de Groot (1914-2006) was one of the most influential Dutch psychologists. He became famous for his work …

Author(s): De Groot, A. D. translated and annotated by Eric-Jan Wagenmakers, Denny Borsboom, Josine Verhagen, Rogier Kievit, Marjan Bakker, Angelique Cramer, Dora Matzke, Don Mellenbergh, and Han LJ van der Maas

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science, Transparency

Link to resource

The Missing Semester of Your CS Education

Course on computer sciences skills needed for all scientific research

Author(s): MIT

Type of resources: Full Course

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Career and Technical Education, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Computer Sciences

Link to resource

The N-Pact Factor: Evaluating the Quality of Empirical Journals with Respect to Sample Size and Statistical Power

The authors evaluate the quality of research reported in major journals in social-personality psychology by ranking those journals with …

Author(s): R. Chris Fraley, Simine Vazire

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

The natural selection of bad science.

Poor research design and data analysis encourage false-positive findings. Such poor methods persist despite perennial calls for …

Author(s): Smaldino, P. E., & McElreath, R.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The New Statistics: Confidence Intervals, NHST, and p Values (Workshop Part 1)

A video about Confidence Intervals, NHST, and p Values

Author(s): Psychological Science/Geoff Cumming

Type of resources: Student Guide, Teaching/Learning Strategy, Video

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Video

Link to resource

The New Statistics: Effect Sizes and Confidence Intervals (Workshop Part 3)

A video about effect sizes and confidence intervals

Author(s): Psychological Science/Geoff Cumming

Type of resources: Student Guide, Teaching/Learning Strategy, Video

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics, Social Science

Tag(s): Video

Link to resource

The New Statistics: Meta-Analysis and Meta-Analytic Thinking (workshop Part 6)

A video about meta analysis and meta-analytical thinking

Author(s): Psychological Science/Geoff Cumming

Type of resources: Student Guide, Teaching/Learning Strategy, Video

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s): Video

Link to resource

The New Statistics: Planning, Power, and Precision (Workshop Part 5)

A video about power analysis and precision

Author(s): Psychological Science/Geoff Cumming

Type of resources: Student Guide, Teaching/Learning Strategy, Video

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Math & Statistics

Tag(s): Video

Link to resource

The New Statistics: Research Integrity & the New Statistics (Workshop Part 2)

A video about Research Integrity & the New Statistics

Author(s): Psychological Science/Geoff Cumming

Type of resources: Student Guide, Teaching/Learning Strategy, Video

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Life Science, Math & Statistics, Social Science

Tag(s): Video, Reproducibility Knowledge

Link to resource

The New Statistics: The New Statistics in Action (Workshop Part 4)

A video about New statistics in action

Author(s): Psychological Science/Geoff Cumming

Type of resources: Student Guide, Teaching/Learning Strategy, Video

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Video

Link to resource

The new statistics: Why and how.

We need to make substantial changes to how we conduct research. First, in response to heightened concern that our published research …

Author(s): Geoff Cumming

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

The Null Hypothesis Significance-Testing Debate and Its Implications for Personality Research

A chapter about null hypothesis significance testing in personality research

Author(s): R. Chris Fraley and Michael J. Marks

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The Null Ritual What You Always Wanted to Know About Significance Testing but Were Afraid to Ask

A chapter about significance testing

Author(s): Gerd Gigerenzer, Stefan Krauss, and Oliver Vitouch

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Social Science

Tag(s):

Link to resource

The Open Science Training Handbook

A collection about open science

Author(s): Sonja Bezjak, Philipp Conzett, Pedro L. Fernandes, Edit Görögh, Kerstin Helbig, Bianca Kramer, Ignasi Labastida, Kyle Niemeyer, Fotis Psomopoulos, Tony Ross-Hellauer, René Schneider, Jon Tennant, Ellen Verbakel

Type of resources: Activity/Lab, Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Life Science, Social Science

Tag(s):

Link to resource

The Peer Reviewers’ Openness Initiative: incentivizing open research practices through peer review.

Openness is one of the central values of science. Open scientific practices such as sharing data, materials and analysis scripts …

Author(s): Richard Morey et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Transparency, Peer-Review

Link to resource

The Persistence of Underpowered Studies in Psychological Research: Causes, Consequences, and Remedies

Underpowered studies persist in the psychological literature. This article examines reasons for their persistence and the effects on …

Author(s): S.E. Maxwell

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The pipeline project: Pre-publication independent replications of a single laboratory’s research pipeline

This crowdsourced project introduces a collaborative approach to improving the reproducibility of scientific research, in which …

Author(s): Martin Schweinsberg et al.

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The poor availability of psychological research data for reanalysis.

The origin of the present comment lies in a failed attempt to obtain, through e-mailed requests, data reported in 141 empirical …

Author(s): Wicherts, Jelte M., Borsboom, Denny, Kats, Judith, Molenaar, Dylan

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The poor availability of syntaxes of structural equation modeling

The syntax or codes used to fit Structural Equation Models (SEMs) convey valuable information on model specifications and the manner in …

Author(s): Jelte M. Wicherts and Elise A. V. Crompvoets

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

The preregistration revolution

Progress in science relies in part on generating hypotheses with existing observations and testing hypotheses with new observations. …

Author(s): Brian A. Nosek, Charles R. Ebersole, Alexander C. DeHaven, and David T. Mellor

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s):

Link to resource

The prevalence of statistical reporting errors in psychology (1985–2013)

This study documents reporting errors in a sample of over 250,000 p-values reported in eight major psychology journals from 1985 until …

Author(s): Michèle B. Nuijten, Chris H. J. Hartgerink, Marcel A. L. M. van Assen, Sacha Epskamp & Jelte M. Wicherts

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

The relation between statistical power and inference in fMRI.

Statistically underpowered studies can result in experimental failure even when all other experimental considerations have been …

Author(s): Cremers, H. R., Wager, T. D., & Yarkoni, T.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Life Science, Math & Statistics

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The Replication Crisis in Psychology

In science, replication is the process of repeating research to determine the extent to which findings generalize across time and …

Author(s): Edward Diener and Robert Biswas-Diener

Type of resources: Primary Source, Reading, Blog

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The Replication Recipe: What makes for a convincing replication?

Psychological scientists have recently started to reconsider the importance of close replications in building a cumulative knowledge …

Author(s): Brandt et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The Reputational Consequences of Failed Replications and Wrongness Admission among Scientists

Scientists are dedicating more attention to replication efforts. While the scientific utility of replications is unquestionable, the …

Author(s): Adam K. Fetterman and Kai Sassenberg

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The rules of the game called psychological science

If science were a game, a dominant rule would probably be to collect results that are statistically significant. Several reviews of the …

Author(s): Marjan Bakker, Annette van Dijk, Jelte M. Wicherts

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The State of Social and Personality Science: Rotten to the Core, Not So Bad, Getting Better, or Getting Worse?

The scientific quality of social and personality psychology has been debated at great length in recent years. Despite research on the …

Author(s): Motyl et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The statistical power of abnormal-social psychological research: A review.

An article about statistical power of abnormal and social psychology

Author(s): Jacob Cohen

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Life Science, Social Science

Tag(s):

Link to resource

The Superego, the Ego, and the Id in Statistical Reasoning

Statistical reasoning is an art and so demands both mathematical knowledge and informed judgment. When it is mechanized, as with the …

Author(s): Gerd Gigerenzer

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Math & Statistics, Social Science

Tag(s):

Link to resource

The t-distritbution and its normal approximation

I just published a new interactive visualization in my series of basic statistical concepts and techniques. This time I am trying to …

Author(s): Kristoffer Magnusson

Type of resources: Interactive, Simulation, Student Guide, Teaching/Learning Strategy

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Blog, Interaction, Simulation, Tutorial

Link to resource

The Test of Insufficient Variance (TIVA): A New Tool for the Detection of Questionable Research Practices

A blog about The Test of Insufficient Variance (TIVA): A New Tool for the Detection of Questionable Research Practices

Author(s): Shimmack, U.

Type of resources: Blog

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The value of direct replication.

Reproducibility is the cornerstone of science. If an effect is reliable, any competent researcher should be able to obtain it when …

Author(s): Daniel Simons

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The what, why, and how of born-open data

Although many researchers agree that scientific data should be open to scrutiny to ferret out poor analyses and outright fraud, most …

Author(s): Jeffrey N. Rouder

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s):

Link to resource

Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology.

Theories in “soft” areas of psychology (e.g., clinical, counseling, social, personality, school, and community) lack the …

Author(s): Meehl, P. E

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Theory Construction and Model-Building Skills: A Practical Guide for Social Scientists

Meeting a crucial need for graduate students and newly minted researchers, this innovative text provides hands-on tools for generating …

Author(s): James Jaccard, Jacob Jacoby

Type of resources: Textbook

Primary user(s): Student, Teacher, Researcher

Subject area(s): Applied Science, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Book

Link to resource

Theory-Testing in Psychology and Physics: A Methodological Paradox

Because physical theories typically predict numerical values, an improvement in experimental precision reduces the tolerance range and …

Author(s): Paul E.Meehl

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Things I have learned (so far).

This is an account of what I have learned (so far) about the application of statistics to psychology and the other sociobiomedical …

Author(s): Jacob Cohen

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics, Social Science

Tag(s):

Link to resource

Tijdink, J. K., Verbeke, R., & Smulders, Y. M. (2014). Publication pressure and scientific misconduct in medical scientists. Journal of Empirical Research on Human Research Ethics, 9(5), 64-71.

There is increasing evidence that scientific misconduct is more common than previously thought. Strong emphasis on scientific …

Author(s): Joeri K. Tijdink, Reinout Verbeke, Yvo M. Smulders

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Too true to be bad: When sets of studies with significant and nonsignificant findings are probably true

Psychology journals rarely publish nonsignificant results. At the same time, it is often very unlikely (or “too good to be true”) that …

Author(s): Lakens, D., & Etz, A. J.

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Tools for De-Identification of Personal Health Information

This report identifies useful and available tools and techniques for the deidentification of personal information from interoperable …

Author(s): Ross Fraser and Don Willison

Type of resources: Reading

Primary user(s): Administrator, Researcher

Subject area(s): Life Science, Social Science

Tag(s):

Link to resource

Tracking replicability as a method of post-publication open evaluation.

Recent reports have suggested that many published results are unreliable. To increase the reliability and accuracy of published papers, …

Author(s): Hartshorne, J. K., & Schachner, A.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Transparent and Open Social Science Research course

Demand is growing for evidence-based policy making, but there is also growing recognition in the social science community that limited …

Author(s): Berkeley Initiative for Transparency in the Social Sciences

Type of resources: Full Course, Lecture, Module, Video

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Course, Video, Reproducibility Knowledge

Link to resource

Transparent science: A more credible, reproducible, and publishable way to do science

This is an exciting time to be a psychological scientist. There is a major new movement that seeks to promote the credibility and …

Author(s): David Mellor, Simine Vazire and D. Lindsay

Type of resources: Primary Source, Reading, Chapter

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Trust your science? Open your data and code.

A paper about Open Your Data and Code

Author(s): Victoria Stodden

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Tutorial/R code for creating funnel/forest plots

Meta-analyses are often accompanied by two popular forms of data visualization: forest plots and funnel plots. In this post, I’ll show …

Author(s): John K. Sakaluk

Type of resources: Reading, R code

Primary user(s): Student, Teacher

Subject area(s): Life Science, Math & Statistics, Social Science

Tag(s): Blog, Tutorial

Link to resource

Tutorial/R code for creating plots for 2-way interactions

ggplot2, as I’ve already made clear, is one of my favourite packages for R. And since that original post about ggplot2 remains one of …

Author(s): John K. Sakaluk

Type of resources: Reading, R code

Primary user(s): Student, Teacher

Subject area(s): Life Science, Math & Statistics, Social Science

Tag(s): Blog, Tutorial

Link to resource

Tutorial/R code for creating scree/parallel analysis plots

With this post, I’m going to be showing how you can use the psych package in conjunction with ggplot2 in order to create a prettier …

Author(s): John K Sakaluk

Type of resources: Reading, R code

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Blog, Tutorial

Link to resource

Underreporting in Psychology Experiments: Evidence from a Study Registry.

Many scholars have raised concerns about the credibility of empirical findings in psychology, arguing that the proportion of false …

Author(s): Annie Franco, Neil Malhotra, and Gabor Simonovits

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Understanding Bayes: A Look at the Likelihood

A blog about bayesian statistics

Author(s): Alexander Etz

Type of resources: Reading

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Blog

Link to resource

Understanding Bayes: Visualization of the Bayes Factor

An abstract about Understanding Bayes and visualising Bayes Factor

Author(s): Alexander Etz

Type of resources: Diagram/Illustration, Reading, Student Guide, Teaching/Learning Strategy, R code

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Blog, R code

Link to resource

Understanding psychology as a science: An introduction to scientific and statistical inference

How can we objectively define categories of truth in scientific thinking? How can we reliably measure the results of research? In this …

Author(s): Zoltan Dienes

Type of resources: Textbook

Primary user(s): Student, Teacher, Researcher

Subject area(s): Applied Science, Life Science, Math & Statistics, Social Science

Tag(s): Book

Link to resource

Understanding Statistical Power and Significance Testing: an interactive visualization

Much has been said about significance testing – most of it negative. Methodologists constantly point out that researchers misinterpret …

Author(s): Kristoffer Magnusson

Type of resources: Interactive, Simulation, Student Guide, Teaching/Learning Strategy

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Blog, Interaction, Simulation, Tutorial

Link to resource

Unlikely Results

Why most published scientific research is probably false

Author(s): The Economist

Type of resources: Student Guide, Teaching/Learning Strategy, Video

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Video

Link to resource

Using OSF to Share Data: A Step-by-Step Guide

Sharing data, materials, and analysis scripts with reviewers and readers is valued in psychological science. To facilitate this …

Author(s): Courtney K. Soderberg

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Using prediction markets to estimate the reproducibility of scientific research

Concerns about a lack of reproducibility of statistically significant results have recently been raised in many fields, and it has been …

Author(s): Anna Drebner et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Using R and lme/lmer to fit different two- and three- level longitudinal models

I often get asked how to fit different multilevel models (or individual growth models, hierarchical linear models or linear …

Author(s): Kristoffer Magnusson

Type of resources: Reading, R code

Primary user(s): Student, Teacher, Researcher

Subject area(s): Education, Life Science, Math & Statistics, Social Science

Tag(s): Blog, Tutorial

Link to resource

Using science and psychology to improve the dissemination and evaluation of scientific work

Here I outline some of what science can tell us about the problems in psychological publishing and how to best address those problems. …

Author(s): Brett T. Buttliere

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Visualizing a One-Way ANOVA using D3.js

A blog post and tutorial on visualizing one-way ANOVA

Author(s): Kristoffer Magnusson

Type of resources: Interactive, Reading, Simulation, Student Guide, Teaching/Learning Strategy

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Blog, Interactive, Tutorial

Link to resource

We have to break up

Three mostly positive developments in academic psychology—the cognitive revolution, the virtual requirement for multiple study reports …

Author(s): Robert B. Cialdini

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

We Knew the Future All Along: Scientific Hypothesizing Is Much More Accurate Than Other Forms of Precognition-A Satire in One Part

A critique about Daryl Bem’s (2011) paper and reproducibility

Author(s): Arina K Bones

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Welcoming Quality in Non-Significance and Replication Work, but Moving Beyond the p-Value: Announcing New Editorial Policies for Quantitative Research in JOAA

The self-correcting nature of psychological and educational science has been seriously questioned. Recent special issues of …

Author(s): Matthew T. McBee, Michael S. Matthews

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

What is a p-value?

Blog post going over the p value, misnomers and what p < .05 means

Author(s): Daniel Lakens

Type of resources: Reading, R Code

Primary user(s): Student, Teacher, Researcher

Subject area(s): Applied Science, Education, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Blog, Code, Reproducibility Crisis and Credibility Revolution

Link to resource

WHAT YOU SEE IS WHAT YOU GET? ENHANCING METHODOLOGICAL TRANSPARENCY IN MANAGEMENT RESEARCH

We review the literature on evidence-based best practices on how to enhance methodological transparency, which is the degree of detail …

Author(s): Herman Aguinis, Ravi S. Ramani and Nawaf Alabduljader

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

What’s wrong with Psychology, anyway?

This chapter considers various factors that have been responsible for the comparatively slow development of psychology into a …

Author(s): David T. Lykken

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science, Transparency

Link to resource

What’s wrong with statistical tests – and where do we go from here? In R. B. Kline, Beyond significance testing: Reforming data analysis methods in behavioral research

This chapter considers problems with null hypothesis significance testing (NHST). The literature in this area is quite large. D. …

Author(s): R.B. Kline

Type of resources: Primary Source, Reading, Chapter

Primary user(s): Student

Subject area(s): Social Science

Tag(s):

Link to resource

When power analyses based on pilot data are biased: Inaccurate effect size estimators and follow-up bias

When designing a study, the planned sample size is often based on power analyses. One way to choose an effect size for power analyses …

Author(s): Casper Albers and Daniel Lakens

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Which is the correct statistical test to use?

This paper explains how to select the correct statistical test for a research project, clinical trial, or other investigation. The …

Author(s): Evie McCrum-Gardner

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Why an Entire Field of Psychology Is in Trouble

A video about psychology being trouble

Author(s): SciShow/Michael Aranda

Type of resources: Student Guide, Teaching/Learning Strategy, Video

Primary user(s): Student, Teacher

Subject area(s): Social Science

Tag(s): Video

Link to resource

Why Most Discovered True Associations Are Inflated

Newly discovered true (non-null) associations often have inflated effects compared with the true effect sizes. I discuss here the main …

Author(s): John P A Ioannidis

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Why most of psychology is statistically unfalsifiable

Low power in experimental psychology is an oft-discussed problem. We show in the context of the Replicability Project: Psychology (Open …

Author(s): Richard Morey and Daniel Lakens

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Social Science

Tag(s):

Link to resource

Why most published research findings are false

There is increasing concern that most current published research findings are false. The probability that a research claim is true may …

Author(s): John Ioannidis

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

why p = .048 should be rare (and why this feels counterintuitive)

This post discusses the why p value around .048 should be rare

Author(s): Simine Vazire

Type of resources: Reading

Primary user(s): Student, Teacher, Researcher

Subject area(s): Math & Statistics

Tag(s): Blog

Link to resource

Why psychologists must change the way they analyze their data: The case of psi: Comment on Bem (2011).

Does psi exist? D. J. Bem (2011) conducted 9 studies with over 1,000 participants in an attempt to demonstrate that future events …

Author(s): Wagenmakers, E.-J., Wetzels, R., Borsboom, D., & van der Maas, H. L. J.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Why Psychologists’ Food Fight Matters

“Important findings” haven’t been replicated, and science may have to change its ways.

Author(s): Michelle N.Meyer and Christopher Chabris

Type of resources: Blog

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Blog, Open Science, Reproducibility Crisis and Credibility Revolution

Link to resource

WHY SUMMARIES OF RESEARCH ON PSYCHOLOGICAL THEORIES ARE OFTEN UNINTERPRETABLE

Null hypothesis testing of correlational predictions from weak substantive theories in soft psychology is subject to the influence of …

Author(s): Paul Meehl

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Willingness to share research data is related to the strength of the evidence and the quality of reporting of statistical results

Background: The widespread reluctance to share published research data is often hypothesized to be due to the authors’ fear that …

Author(s): Jelte M. Wicherts, Marjan Bakker, Dylan Molenaar

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Life Science, Social Science

Tag(s):

Link to resource

You are not so smart

Psychology is working on the hardest problems in all of science. Physics, astronomy, geology — those are easy, by comparison. …

Author(s): David McRaney

Type of resources: Student Guide, Teaching/Learning Strategy, Podcast

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Podcast, Reproducibility Crisis and Credibility Revolution

Link to resource

You Cannot Step Into the Same River Twice: When Power Analyses Are Optimistic

Statistical power depends on the size of the effect of interest. However, effect sizes are rarely fixed in psychological research: …

Author(s): Blakeley B McShane, Ulf Böckenholt

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

‘‘Positive’’ Results Increase Down the Hierarchy of the Sciences

The hypothesis of a Hierarchy of the Sciences with physical sciences at the top, social sciences at the bottom, and biological sciences …

Author(s): Daniele Fanelli

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Missing a resource?

Fill out the form below or here is the direct link to open the Google form in a new window