Curated resources

There are more than 1000 curated resources in our database. These resources were crowdsourced by our community and, more recently, our database was increased by over 60% thanks to the Open Science Knowledge Base( see also link to the OER commons page), a Center for Open Science educational initiative, which graciously chose FORRT to host their curated Open Science resources. COS, OSKB, and FORRT forged a partnership to promote open and reproducible research practices, enabling the scientific community to build upon each other’s work and advance knowledge collaboratively. Learn more about it FORRT’s and OSKB’s partnership in the COS’s announcement.

Contribute

If you notice there is an educational resource, research article or pedagogical tool missing in our database, please consider adding it below in our on submission form or directly into a separate g-forms.

FORRT’s Curated Resources



Loading search data...

*

(divorcing) WHITE SUPREMACY CULTURE

This website is a long-time dream finally bearing fruit, a needed remix of the widely circulated article WHITE SUPREMACY CULTURE …

Author(s): Tema Okun

Type of resources: Reading

Primary user(s): Teacher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): White Supremacy, Antiracist

Link to resource

1,500 scientists lift the lid on reproducibility

Survey sheds light on the ‘crisis’ rocking research.

Author(s): Monya Baker

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

10 Things for Curating Reproducible and FAIR Research

10 Things for Curating Reproducible and FAIR Research (CURE-FAIR) provides standards-based guidelines to follow CURE-FAIR best …

Author(s): RDA CURE-FAIR WG (Consortium)

Type of resources: Primary Source, Teaching/Learning Strategy

Primary user(s): Librarian

Subject area(s): Social Science

Tag(s): Reproducibility; Curation; Open Data; Open Science

Link to resource

1Economics 270D: Research Transparency Methods in the Social Sciences

This course coversa range of approachesthat aim to enhance the transparency and reproducibility of social science research. It is …

Author(s): Edward Miguel

Type of resources: Syllabus

Primary user(s): Teacher

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

7 Easy Steps to Open Science: An Annotated Reading List

The Open Science movement is rapidly changing the scientific landscape. Because exact definitions are often lacking and reforms are …

Author(s): Alexander Etz, Amy Orben, Hannah Moshontz, Jesse Niebaum, Johnny van Doorn, Matthew Makel, Michael Schulte-Mecklenbeck, Sam Parsons, Sophia Crüwell

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Data, Materials, Open Scholarship Guidelines, Policy, Publishing, Reproducibility, Researchers

Link to resource

A 21 word solution.

One year after publishing “False-Positive Psychology,” we propose a simple implementation of disclosure that requires but …

Author(s): Simmons, Joseph P. and Nelson, Leif D. and Simonsohn, Uri, A

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

A Bayesian Perspective on the Reproducibility Project: Psychology

We revisit the results of the recent Reproducibility Project: Psychology by the Open Science Collaboration. We compute Bayes factors—a …

Author(s): Alexander Etz and Joachim Vandekerckhove

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

A Case For Data Dashboards: First Steps with R Shiny

Dashboards for data visualisation, such as R Shiny and Tableau, allow an interactive exploration of data by means of drop-down lists …

Author(s): Pablo Bernabeu

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Data, Librarians, Open Data, Open Source Software, Publishers, Researchers, Visualization

Link to resource

A cautionary note on aggregation in educational psychology and beyond

This article addresses aggregation as a fundamental practice in educational psychology and ties it into the idiographic/nomothetic …

Author(s): Georg Krammer

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian, researcher

Subject area(s): Education

Tag(s): Aggregation, Best Practice, Educational Psychology, Idiographic, Nomothetic, Quantitative Psychology

Link to resource

A checklist is associated with increased quality of reporting preclinical biomedical research: A systematic review

Irreproducibility of preclinical biomedical research has gained recent attention. It is suggested that requiring authors to complete a …

Author(s): Doris M. Rubio, Janet S. Lee, Jill Zupetic, John P. Pribis, Joo Heung Yoon, Kwonho Jeong, Kyle M. Holleran, Nader Shaikh, SeungHye Han, Tolani F. Olonisakin

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Health, Medicine and Nursing, Biology

Tag(s): Animal Studies, Clinical Trials, Genetically Modified Animals, Inbred Strains, Medical Journals, Randomized Controlled Trials, Reproducibility, Research Reporting Guidelines, Scientific Publishing, Systematic Reviews

Link to resource

A collaborative approach to infant research: Promoting reproducibility, best practices, and theory-building.

The ideal of scientific progress is that we accumulate measurements and integrate these into theory, but recent discussion of …

Author(s): Frank et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

A community-endorsed open-source lexicon for contrast agent–based perfusion MRI: A consensus guidelines report from the ISMRM Open Science Initiative for Perfusion Imaging (OSIPI)

This manuscript describes the ISMRM OSIPI (Open Science Initiative for Perfusion Imaging) lexicon for dynamic contrast-enhanced and …

Author(s): Ben R. Dickie, Zaki Ahmed, Jonathan Arvidsson, Laura C. Bell, David L. Buckley, Charlotte Debus, Andrey Fedorov, Ralf Floca, Ingomar Gutmann, Rianne A. van der Heijden, Petra J. van Houdt, Steven Sourbron, Michael J. Thrippleton, Chad Quarles, Ina N. Kompan, on behalf of The Perfusion Study Group of the ISMRM

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): CAPLEX, DCE-MRI, DSC-MRI, ISMRM OSIPI, Perfusion, Standardization, Lexicon

Link to resource

A consensus-based transparency checklist

We present a consensus-based checklist to improve and document the transparency of research reports in social and behavioural research. …

Author(s): Agneta Fisher, Alexandra M. Freund, Alexandra Sarafoglou, Alice S. Carter, Andrew A. Bennett, Andrew Gelman, Balazs Aczel, Barnabas Szaszi, Benjamin R. Newell, Brendan Nyhan, Candice C. Morey, Charles Clifton, Christopher Beevers, Christopher D. Chambers, Christopher Sullivan, Cristina Cacciari, Daniel Benjamin, Daniel J. Simons, David R. Shanks, Debra Lieberman, Derek Isaacowitz, Dolores Albarracin, Don P. Green, D. Stephen Lindsay, Eric-Jan Wagenmakers, Eric Johnson, Eveline A. Crone, Fernando Hoces de la Guardia, Fiammetta Cosci, George C. Banks, Gordon D. Logan, Hal R. Arkes, Harold Pashler, Janet Kolodner, Jarret Crawford, Jeffrey Pollack, Jelte M. Wicherts, John Antonakis, John Curtin, John P. Ioannidis, Joseph Cesario, Kai Jonas, Lea Moersdorf, Lisa L. Harlow, Marcus Munafò, Mark Fichman, M. Gareth Gaskell, Mike Cortese, Mitja D. Back, Morton A. Gernsbacher, Nelson Cowan, Nicole D. Anderson, Pasco Fearon, Randall Engle, Robert L. Greene, Roger Giner-Sorolla, Ronán M. Conroy, Scott O. Lilienfeld, Simine Vazire, Simon Farrell, Šimon Kucharský, Stavroula Kousta, Ty W. Boyer, Wendy B. Mendes, Wiebke Bleidorn, Willem Frankenhuis, Zoltan Kekecs

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Psychology

Tag(s): Inside Your Classroom, Publishing

Link to resource

A Duty to Describe: Better the Devil You Know Than the Devil You Don’t

Although many researchers have discussed replication as a means to facilitate self-correcting science, in this article, we identify …

Author(s): Sacha D Brown, David Furrow, Daniel F Hill, Jonathon C Gable, Liam P Porter, W Jake Jacobs

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

A funder-imposed data publication requirement seldom inspired data sharing

Growth of the open science movement has drawn significant attention to data sharing and availability across the scientific community. …

Author(s): Colette L. Ward, Gavin McDonald, Jessica L. Couture, Rachael E. Blake

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Health, Medicine and Nursing, Biology, Social Science

Tag(s): Communications, Data, Data Acquisition, Data Management, Language, Open Data, Open Science, Policy, Public Policy, Research Funding, Research Grants, Science Policy

Link to resource

A future for digital public goods for monitoring SDG indicators

Digital public goods (DPGs), if implemented with effective policies, can facilitate the realization of the United Nations Sustainable …

Author(s): Dong Liang, Huadong Guo, Stefano Nativi, Markku Kulmala, Zeeshan Shirazi, Fang Chen, Gretchen Kalonji, Dongmei Yan, Jianhui Li, Robert Duerler, Lei Luo, Qunli Han, Siming Deng, Yuanyuan Wang, Lingyi Kong, Thorsten Jelinek

Type of resources: Reading

Primary user(s): Student, Teacher, Administrator, Librarian

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Environmental Impact, Policy

Link to resource

A Guide to Data Visualization: Best Practices for Communicating Open Educational Data

This applied webinar explores best practices for communicating open educational data with a wide audience. Topics include different …

Author(s): Daniel Anderson

Type of resources: Lesson

Primary user(s):

Subject area(s): Education

Tag(s): Data Communication, Data Visualization, Open Education Research

Link to resource

A Guide to Supporting Early Career Researchers in Open Science

Supporting early career researchers in their open science journey is critical for the future of the field. The development of open …

Author(s): Jesse Fleming, Sarah Emily Wilson, Ummul Kathawalla

Type of resources: Lecture

Primary user(s):

Subject area(s): Education

Tag(s): #earlycareerresearchers #openscience #education

Link to resource

A manifesto for reproducible science

Improving the reliability and efficiency of scientific research will increase the credibility of the published scientific literature …

Author(s): Munafo, M. R., et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

A multilab preregistered replication of the ego-depletion effect.

Good self-control has been linked to adaptive outcomes such as better health, cohesive personal relationships, success in the workplace …

Author(s): Hagger, M. S., Chatzisarantis, N. L., Alberts, H., Anggono, C. O., Batailler, C., Birt, A. R., ... & Calvillo, D. P.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

A Plea for Preregistration in Personality Disorders Research: The Case of Psychopathy

In response to a crisis of confidence, several methodological initiatives have been launched to improve the robustness of psychological …

Author(s): Bruno Verschuere, Franziska M. Yasrebi-de Kom, Iza van Zelm, Scott O. Lilienfeld

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Social Science

Tag(s): Personality Disorders, Preregistration, Clinical Psychology, Psychopathology

Link to resource

A Power Primer

One possible reason for the continued neglect of statistical power analysis in research in the behavioral sciences is the …

Author(s): Jacob Cohen

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

A Powerful Nudge? Presenting Calculable Consequences of Underpowered Research Shifts Incentives Toward Adequately Powered Designs

If psychologists have recognized the pitfalls of underpowered research for decades, why does it persist? Incentives, perhaps: …

Author(s): Will M. Gervais, Jennifer A. Jewell, Maxine B. Najle, and Ben K. L. Ng

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

A primer for choosing, designing and evaluating registered reports for qualitative methods

Registered reports are a publication format that involves peer reviewing studies both before and after carrying out research …

Author(s): Veli-Matti Karhulahti, Peter Branney, Miia Siutila, Moin Syed

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Social Science

Tag(s): Qualitative, Registered Reports

Link to resource

A Primer on Systematic Review and Meta-analysis in Diabetes Research

A systematic review is a rigorous process that involves identifying, selecting, and synthesizing available evidence pertaining to an a …

Author(s): Deirdre K. Tobias, Stefania Papatheodorou, Jennifer M. Yamamoto, Frank B. Hu

Type of resources: Reading, Student Guide

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Systematic Review, Meta Analysis, Diabetes

Link to resource

A proposal for a new editorial policy in the social sciences.

“… there’s this desert prison, see, with an old prisoner, resigned to his life, and a young one just arrived. The young one talks …

Author(s): G. William Walster &T. Anne Cleary

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s):

Link to resource

A proposal for the future of scientific publishing in the life sciences

Science advances through rich, scholarly discussion. More than ever before, digital tools allow us to take that dialogue online. To …

Author(s): Bodo M. Stern, Erin K. O’Shea

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Biology

Tag(s): Bibliometrics, Careers, Citation Analysis, Internet, Peer Review, Publishing, Quality Control, Research Funding, Scientific Publishing, Scientists

Link to resource

A Reliability-Generalization Study of Journal Peer Reviews: A Multilevel Meta-Analysis of Inter-Rater Reliability and Its Determinants

Background: This paper presents the first meta-analysis for the inter-rater reliability (IRR) of journal peer reviews. IRR is defined …

Author(s): Lutz Bornmann, Rüdiger Mutz and Hans-Dieter Daniel

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

A reputation economy: how individual reward considerations trump systemic arguments for open access to data

Open access to research data has been described as a driver of innovation and a potential cure for the reproducibility crisis in many …

Author(s): Benedikt Fecher, Marcel Hebing, Sascha Friesike, Stephanie Linek

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Information Science

Tag(s): Data Sharing, Open Data, Publishing, Reproducibility

Link to resource

A Short (Personal) Future History of Revolution 2.0

Crisis of replicability is one term that psychological scientists use for the current introspective phase we are in—I argue instead …

Author(s): Barbara A. Spellman

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

A Short Introduction to the Reproducibility Debate in Psychology

The Journal of European Psychology Students (JEPS) is an open-access, double-blind, peer-reviewed journal for psychology students …

Author(s): Cedric Galetzka

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Psychology

Tag(s): Publishing

Link to resource

A Social Priming Data Set With Troubling Oddities

A recent paper by Chatterjee, Rose, and Sinha (2013) reported impressively large “money priming” effects: incidental exposure to …

Author(s): Harold Pashler, Doug Rohrer, Ian Abramson, Tanya Wolfson &Christine R. Harris

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

A Social Psychological Model of Scientific Practices: Explaining Research Practices and Outlining the Potential for Successful Reforms

A crescendo of incidents have raised concerns about whether scientific practices in psychology may be suboptimal, sometimes leading to …

Author(s): Jon A. Krosnick, Lee Jussim, Sean T. Stevens, Stephanie M. Anglin

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Psychology

Tag(s): Meta-Xcience, Questionable Research Practices, Replication Crisis, Reproducibility, Science Reform

Link to resource

A Software Tool for Removing Patient Identifying Information from Clinical Documents

We created a software tool that accurately removes all patient identifying information from various kinds of clinical data documents, …

Author(s): Friedlin, F. J., & McDonald, C. J.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s):

Link to resource

A step-by-step guide on preregistration and effective data sharing for psychopathology research

Data analysis in psychopathology research typically entails multiple stages of data preprocessing (e.g., coding of physiological …

Author(s): Angelos-Miltiadis Krypotos, Irene Klugkist, Gaëtan Mertens, Iris M. Engelhard

Type of resources: Student Guide

Primary user(s): Student, Teacher

Subject area(s): Social Science

Tag(s): Preregistration, Psychopathology, Clinical Psychology, Data Sharing

Link to resource

A study of the impact of data sharing on article citations using journal policies as a natural experiment

This study estimates the effect of data sharing on the citations of academic articles, using journal policies as a natural experiment. …

Author(s): Allan Dafoe, Andrew K. Rose, Don A. Moore, Edward Miguel, Garret Christensen

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Economics

Tag(s): Citation Analysis, Data, Instrumental Variable Analysis, Policy, Political Science, Publishing, Science Policy, Science Policy and Economics, Scientific Publishing, Scientists, Statistical Data

Link to resource

A survey of the statistical power of research in behavioral ecology and animal behavior.

We estimated the statistical power of the first and last statistical test presented in 697 papers from 10 behavioral journals. First …

Author(s): Jennions, M. D., & Møller, A. P.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Life Science, Math & Statistics

Tag(s):

Link to resource

A survey on how preregistration affects the research workflow: better science but more work

The preregistration of research protocols and analysis plans is a main reform innovation to counteract confirmation bias in the social …

Author(s): Alexandra Sarafoglou, Marton Kovacs, Bence Bakos, Eric-Jan Wagenmakers, Balazs Aczel

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Open Science, Metascience, Replication Crisis, Preregistration

Link to resource

A systematic review of statistical power in software engineering experiments.

Statistical power is an inherent part of empirical studies that employ significance testing and is essential for the planning of …

Author(s): Dybå, T., Kampenes, V. B., & Sjøberg, D. I.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

A tale of two papers

An abstract about transparency and robustness for two papers

Author(s): Michael Inzlicht

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian, Researcher

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Blog, Reproducibility Crisis and Credibility Revolution

Link to resource

A test of the diffusion model explanation for the worst performance rule using preregistration and blinding

People with higher IQ scores also tend to perform better on elementary cognitive-perceptual tasks, such as deciding quickly whether an …

Author(s): Alexander Ly, Andreas Pedroni, Dora Matzke, Eric-Jan Wagenmakers, Gilles Dutilh, Joachim Vandekerckhove, Jörg Rieskamp, Renato Frey

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Psychology

Tag(s): Reproducibility

Link to resource

A tutorial on cognitive modeling for cognitive aging research

Cognitive aging researchers are interested in understanding how cognitive processes change in old age, but the relationship between …

Author(s): Nathaniel R. Green

Type of resources: Student Guide

Primary user(s): Student, Teacher

Subject area(s): Life Science, Social Science

Tag(s): Cognitive Aging, Cognitive Modeling, Replication

Link to resource

A Unified Framework to Quantify the Credibility of Scientific Findings

Societies invest in scientific studies to better understand the world and attempt to harness such improved understanding to address …

Author(s): Etienne P. LeBel, Randy J. McCarthy, Brian D. Earp, Malte Elson and Wolf Vanpaemel

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Transparency, Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

A Vast Graveyard of Undead Theories: Publication Bias and Psychological Science’s Aversion to the Null

Publication bias remains a controversial issue in psychological science. The tendency of psychological science to avoid publishing null …

Author(s): Christopher J. Ferguson and Moritz Heene

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

A. Solomon Kurz website

A blog posts about Bayesian statistics

Author(s): A.Solomon Kurz

Type of resources: Reading

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s): Bayesian statistics

Link to resource

Abandon statistical significance.

We discuss problems the null hypothesis significance testing (NHST) paradigm poses for replication and more broadly in the biomedical …

Author(s): McShane, B. B., Gal, D., Gelman, A., Robert, C., & Tackett, J. L.

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Academic Research in the 21st Century: Maintaining Scientific Integrity in a Climate of Perverse Incentives and Hypercompetition

Over the last 50 years, we argue that incentives for academic scientists have become increasingly perverse in terms of competition for …

Author(s): Marc A. Edwards and Siddhartha Roy

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Accessibility Data Curation Primer

Data curators are uniquely positioned to help improve access not just to individual datasets, but to the world of research data at …

Author(s): Emily Oxford, Rachel Woodbrook

Type of resources: Student Guide

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Accessibility, Data Sharing, Data Management, Open Data, FAIR Data

Link to resource

Accuracy of effect size estimates from published psychological research

Monte-Carlo simulation was used to model the biasing of effect sizes in published studies. The findings from the simulation indicate …

Author(s): ANDREW BRAND, MICHAEL T. BRADLEY, LISA A. BEST, AND GEORGE STOICA

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Adapting open science and pre-registration to longitudinal research

Open science practices, such as pre-registration and data sharing, increase transparency and may improve the replicability of …

Author(s): Isaac T. Petersen, Keith S. Apfelbaum, Bob McMurray

Type of resources: Reading, Student Guide

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Development, Longitudinal, Open Science, Preregistration, Replication, Reproducibility

Link to resource

Adjusting for Publication Bias in MetaAnalysis. Perspectives on Psychological Science, 11(5), 730–749.

We review and evaluate selection methods, a prominent class of techniques first proposed by Hedges (1984) that assess and adjust for …

Author(s): McShane, B. B., Böckenholt, U., & Hansen, K. T

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Adopting Open Educational Resources as an Equity Strategy

Social determinants of learning (SDOL) impact students’ abilities to successfully complete their courses. An economic barrier for …

Author(s): Ernstmeyer, Kimberly E.; Christman, Elizabeth I.

Type of resources: Reading

Primary user(s): Teacher, Administrator

Subject area(s): Life Science

Tag(s): Equity in Nursing Education, Nursing Textbooks, Open Educational Resources, Social Determinants of Learning

Link to resource

Advocating for Change in How Science is Conducted to Level the Playing Field

Open science practices have the potential to greatly accelerate progress in scientific research if widely adopted, but individual …

Author(s):

Type of resources: Lecture

Primary user(s):

Subject area(s): Education

Tag(s): #centerforopenscience, #openpractice, #openscience, Reproducibility, Research, #TOPguidelines

Link to resource

All the Weight of Our Dreams On Living Racialized Autism

For those of us who are autistic and racialized, we often struggle to find representation in mass media, academic work about autism or …

Author(s): Lydia X. Z. Brown, E. Ashkenazy and Morénike Giwa Onaiwu

Type of resources: Reading

Primary user(s): Student, Teacher, Administrator, Parent, Librarian

Subject area(s): Applied Science, Education, Social Science

Tag(s): Diversity, Equity and Inclusion

Link to resource

Always use Welch t-test

This is a blogpost describing why we should use Welch t-test instead of Student t-test

Author(s): Daniel Lakens

Type of resources: Reading, R Code

Primary user(s): Student, Teacher, Researcher

Subject area(s): Applied Science, Education, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Blog, Code, Reproducibility Crisis and Credibility Revolution

Link to resource

An Agenda for Purely Confirmatory Research

The veracity of substantive research claims hinges on the way experimental data are collected and analyzed. In this article, we discuss …

Author(s): Wagenmakers, E.-J., Wetzels, R., Borsboom, D., van der Mass, H. L. J., & Kievit, R. A.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

An assessment of the magnitude of effect sizes: Evidence from 30 years of meta-analysis in management.

This study compiles information from more than 250 meta-analyses conducted over the past 30 years to assess the magnitude of reported …

Author(s): Paterson, T. A., Harms, P. D., Steel, P., & Credé, M

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

An excess of positive results: Comparing the standard Psychology literature with Registered Reports

When studies with positive results that support the tested hypotheses have a higher probability of being published than studies with …

Author(s): Anne M. Scheel, Daniel Lakens, Mitchell Schijen

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Psychology

Tag(s): Publication Bias, Publishing

Link to resource

An exploratory test for an excess of significant findings

Background The published clinical research literature may be distorted by the pursuit of statistically significant results. Purpose: We …

Author(s): John PA Ioannidis and Thomas A Trikalinos

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s):

Link to resource

An Introduction to Registered Replication Reports at Perspectives on Psychological Science

An article about an Introduction to Registered Replication Reports at Perspectives on Psychological Science

Author(s): Daniel J. Simons, Alex O. Holcombe, Barbara A. Spellman

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

An Introduction to Registered Reports for the Research Funder Community

In this webinar, Doctors David Mellor (Center for Open Science) and Stavroula Kousta (Nature Human Behavior) discuss the Registered …

Author(s): Center for Open Science

Type of resources: Lecture

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science

Tag(s): Analysis, Data, Education, Materials, Open Scholarship Tools and Technologies, OSF, Publishing, Reproducibility, Research Data Management Tools, Researchers

Link to resource

An open investigation of the reproducibility of cancer biology research

It is widely believed that research that builds upon previously published findings has reproduced the original work. However, it is …

Author(s): Brian A Nosek, Elizabeth Iorns, Fraser Elisabeth Tan, Joelle Lomax, Timothy M Errington, William Gunn

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Health, Medicine and Nursing, Biology

Tag(s): Data, Methodology, Open Science, Replication, Reproducibility, Reproducibility Project: Cancer Biology, Research Methods

Link to resource

An Open Science MRI Database of over 100 Synaesthetic Brains and Accompanying Deep Phenotypic Information

We provide a neuroimaging database consisting of 102 synaesthetic brains using state-of-the-art 3 T MRI protocols from the Human …

Author(s): Chris Racey, Christina Kampoureli, Oscar Bowen-Hill, Mathilde Bauer, Ivor Simpson, Charlotte Rae, Magda del Rio, Julia Simner, Jamie Ward

Type of resources: Data Set

Primary user(s): Student, Teacher

Subject area(s): Life Science, Social Science

Tag(s): Cognitive Neuroscience, Human Behaviour

Link to resource

An Open Science Primer for Social Scientists

“Open Science” has become a buzzword in academic circles. However, exactly what it means, why you should care about it, and – most …

Author(s): Eike Mark Rinke

Type of resources: Lesson

Primary user(s): student, teacher

Subject area(s): Social Science

Tag(s): Data, Materials, Open Access Policies, Open Scholarship Guidelines, Policy, Publishing, Reproducibility, Researchers

Link to resource

An open source pharma roadmap

In an Essay, Matthew Todd and colleagues discuss an open source approach to drug development. This Essay outlines how open source …

Author(s): Els Torreele, Jaykumar Menon, John McKew, John Wilbanks, Manica Balasegaram, Matthew H. Todd, Peter Kolb, Piero Olliaro, Tomasz Sablinski, Zakir Thomas

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Health, Medicine and Nursing, Biology

Tag(s): Drug Discovery, Drug Research and Development, Finance, Global Health, Open Source Drug Discovery, Open Source Software, Public and Occupational Health, Tuberculosis

Link to resource

Analysis of Open Data and Computational Reproducibility in Registered Reports in Psychology

Ongoing technological developments have made it easier than ever before for scientists to share their data, materials, and analysis …

Author(s): Daniel Lakens, Jaroslav Gottfried, Nicholas Alvaro Coles, Pepijn Obels, Seth Ariel Green

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Psychology

Tag(s): Analysis, Data, Materials, Open Data, Publishing, Registered Reports, Reproducibility

Link to resource

Analytic Review as a Solution to the Misreporting of Statistical Results in Psychological Science

In this article, we propose analytic review (AR) as a solution to the problem of misreporting statistical results in psychological …

Author(s): John Sakaluk, Alexander Williams, Monica Biernat

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Analytical code sharing practices in biomedical research

Data-driven computational analysis is becoming increasingly important in biomedical research, as the amount of data being generated …

Author(s): Nitesh Kumar Sharma, Ram Ayyala, Dhrithi Deshpande, Yesha M Patel, Viorel Munteanu, Dumitru Ciorba, Andrada Fiscutean, Mohammad Vahed, Aditya Sarkar, Ruiwei Guo, Andrew Moore, Nicholas Darci-Maher, Nicole A Nogoy, Malak S. Abedalthagafi, Serghei Mangul

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Life Science

Tag(s): Open Code, Open Data, Biomedical Research, Replication, Reproducibility

Link to resource

Analyzing Education Data with Open Science Best Practices, R, and OSF

This workshop demonstrates how using R can advance open science practices in education. We focus on R and RStudio because it is an …

Author(s): Cynthia D'Angelo, Joshua Rosenberg

Type of resources: Teaching/Learning Strategy

Primary user(s):

Subject area(s): Computer Science, Education

Tag(s): Data Analysis Tools, Education Research, Open Data, Open Science, Open Science Framework, OSF, Osfr, R for Researchers, Rstats, Statistical Programming

Link to resource

Análisis y visualización de datos usando Python

Python es un lenguaje de programación general que es útil para escribir scripts para trabajar con datos de manera efectiva y …

Author(s): Alejandra Gonzalez-Beltran, April Wright, chekos, Christopher Erdmann, Enric Escorsa O'Callaghan, Erin Becker, Fernando Garcia, Hely Salgado, Juan Martín Barrios, Juan M. Barrios, Katrin Leinweber, Laura Angelone, Leonardo Ulises Spairani, LUS24, Maxim Belkin, Miguel González, monialo2000, Nicolás Palopoli, Nohemi Huanca Nunez, Paula Andrea Martinez, Raniere Silva, Rayna Harris, rzayas, Sarah Brown, Silvana Pereyra, Spencer Harris, Stephan Druskat, Trevor Keller, Wilson Lozano

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science, Measurement and Data

Tag(s): Analysis, Data, Education, Jupyter Notebooks, Open Scholarship Tools and Technologies, Python, Reproducibility, Research Data Management Tools, Researchers

Link to resource

Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences

This classic text on multiple regression is noted for its nonmathematical, applied, and data-analytic approach. Readers profit from its …

Author(s): Jacob Cohen, Patricia Cohen, Stephen G. West, Leona S. Aiken

Type of resources: Textbook

Primary user(s): Student, Teacher, Researcher

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Book

Link to resource

Applied Open Science Talk GSA 2020

Applied Open Science Talk GSA 2020

Author(s): Jennifer Lodi-smith, Yunhwan Lee

Type of resources: Lecture Notes

Primary user(s):

Subject area(s): Life Science, Social Science

Tag(s): Aging, Aging Science, Gerontology, Open Science

Link to resource

Are choices based on conditional or conjunctive probabilities in a sequential risk-taking task?

In this study, we examined participants' choice behavior in a sequential risk-taking task. We were especially interested in the extent …

Author(s): Peter Haffke, Ronald Hübner

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Psychology

Tag(s): Columbia Card Task (CCT), Conditional Probability, Conjunctive Probability, Data, Dependent Events, Sequential Risk-Taking

Link to resource

Are Preregistration and Registered Reports Vulnerable to Hacking?

Preregistration of study hypotheses has been proclaimed as the “revolution”1 to curb publication bias and data dredging such as …

Author(s): Jiang Bian, Jae S. Min, Mattia Prosperi, Mo Wang

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Preregistration, Registered Reports, Hacking, Editorial

Link to resource

Are Psychology Journals Anti-replication? A Snapshot of Editorial Practices

Recent research in psychology has highlighted a number of replication problems in the discipline, with publication bias – the …

Author(s): G. N. Martin, Richard M. Clarke

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Psychology

Tag(s): JOURNAL EDITORIAL PRACTICES, P-Hacking, Psychology, Publication Bias, Replication

Link to resource

Are We Wasting a Good Crisis? The Availability of Psychological Research Data after the Storm

To study the availability of psychological research data, we requested data from 394 papers, published in all issues of four APA …

Author(s): Wolf Vanpaemel, Maarten Vermorgen, Leen Deriemaecker and Gert Storms

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Arrested Theory Development: The Misguided Distinction Between Exploratory and Confirmatory Research

Science progresses by finding and correcting problems in theories. Good theories are those that help facilitate this process by being …

Author(s): Aba Szollosi, Chris Donkin

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Philosophy of Science, Theory Development, Confirmatory Research, Exploratory Research, Preregistration, Direct Replication

Link to resource

ARRIVE has not ARRIVEd: Support for the ARRIVE (Animal Research: Reporting of in vivo Experiments) guidelines does not improve the reporting quality of papers in animal welfare, analgesia or anesthesia

Poor research reporting is a major contributing factor to low study reproducibility, financial and animal waste. The ARRIVE (Animal …

Author(s): Daniel S. J. Pang, Frédérik Rousseau-Blass, Guy Beauchamp, Vivian Leung

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Health, Medicine and Nursing, Life Science

Tag(s): Adverse Events, Analgesics, Anesthetics, Animal Slaughter, Animal Welfare, Finance, Publishing, Reporting Bias, Reporting Guidelines

Link to resource

Artificial intelligence and responsibility gaps: What is the problem?

Recent decades have witnessed tremendous progress in artificial intelligence and in the development of autonomous systems that rely on …

Author(s): Peter Königs

Type of resources: Reading

Primary user(s): Student, Teacher, Administrator, Librarian

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Artificial Intelligence, Responsibility, Ethics, Just in Bello

Link to resource

ASL lexicon and reporting recommendations: A consensus report from the ISMRM Open Science Initiative for Perfusion Imaging (OSIPI)

The 2015 consensus statement published by the International Society for Magnetic Resonance in Medicine (ISMRM) Perfusion Study Group …

Author(s): Yuriko Suzuki, Patricia Clement, Weiying Dai, Sudipto Dolui, Maria A. Fernández-Seara, Thomas Lindner, Henk J. M. M. Mutsaerts, Jan Petr, Xingfeng Shao, Manuel Taso, David L. Thomas, ISMRM Perfusion Study Group

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Life Science

Tag(s): Arterial Spin Labeling, Interoperability, Noninvasive, Perfusion Imaging, Reproducibility

Link to resource

Assessing data availability and research reproducibility in hydrology and water resources

There is broad interest to improve the reproducibility of published research. We developed a survey tool to assess the availability of …

Author(s): Adel M. Abdallah, David E. Rosenberg, Hadia Akbar, James H. Stagge, Nour A. Attallah, Ryan James

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Information Science, Physical Science, Hydrology

Tag(s): Data, Reproducibility

Link to resource

Association between trial registration and treatment effect estimates: a meta-epidemiological study

To increase transparency in research, the International Committee of Medical Journal Editors required, in 2005, prospective …

Author(s): Agnès Dechartres, Carolina Riveros, Ignacio Atal, Isabelle Boutron, Philippe Ravaud

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Health, Medicine and Nursing

Tag(s): Bias, ClinicalTrials.gov, Meta-analysis, Meta-epidemiology, Metascience, Meta-Science, Preregistration, Publishing, Randomised Controlled Trial, Registration, Reproducibility

Link to resource

At what sample size do correlations stabilize?

Sample correlations converge to the population value with increasing sample size, but the estimates are often inaccurate in small …

Author(s): Felix D. Schonbrodt and Marco Perugini

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Attitudes towards animal study registries and their characteristics: An online survey of three cohorts of animal researchers

Objectives Prospective registration of animal studies has been suggested as a new measure to increase value and reduce waste in …

Author(s): André Bleich, Daniel Strech, Emily S. Sena, Hans Laser, René Tolba, Susanne Wieschowski

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Health, Medicine and Nursing, Biology

Tag(s): Animal Studies, Chi Square Tests, Medical Journals, Psychological Attitudes, Publication Ethics, Publishing, Survey Research, Surveys, Theft

Link to resource

Authorization of Animal Experiments Is Based on Confidence Rather than Evidence of Scientific Rigor

Accumulating evidence indicates high risk of bias in preclinical animal research, questioning the scientific validity and …

Author(s): Christina Nathues, Hanno Würbel, Lucile Vogt, Thomas S. Reichlin

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Biology

Tag(s): Experimental Design, Language, Mammals, Peer Review, Reproducibility, Research Reporting Guidelines, Research Validity, Rodents

Link to resource

Authorship practices must evolve to support collaboration and open science

Journal authorship practices have not sufficiently evolved to reflect the way research is now done. Improvements to support teams, …

Author(s): Veronique Kiermer

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Research Assessment, Open Science, Taxonomy, Medical Journals, Metadata, Centrality, Citation Analysis, Computer Software

Link to resource

Autistic community and the Neurodiversity movement: Stories from the Frontline

A book about the autistic community and the neurodivergent movement

Author(s): Steven K. Kapp

Type of resources: Reading

Primary user(s): Student, Teacher, Parent

Subject area(s): Education, Social Science

Tag(s): Diversity, Equity, Inclusion, Neurodiversity, Autism

Link to resource

Automation and Make

A Software Carpentry lesson to learn how to use Make Make is a tool which can run commands to read files, process these files in some …

Author(s): Adam Richie-Halford, Ana Costa Conrado, Andrew Boughton, Andrew Fraser, Andy Kleinhesselink, Andy Teucher, Anna Krystalli, Bill Mills, Brandon Curtis, David E. Bernholdt, Deborah Gertrude Digges, François Michonneau, Gerard Capes, Greg Wilson, Jake Lever, Jason Sherman, John Blischak, Jonah Duckles, Juan F Fung, Kate Hertweck, Lex Nederbragt, Luiz Irber, Matthew Thomas, Michael Culshaw-Maurer, Mike Jackson, Pete Bachant, Piotr Banaszkiewicz, Radovan Bast, Raniere Silva, Rémi Emonet, Samuel Lelièvre, Satya Mishra, Trevor Bekolay

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science, Measurement and Data

Tag(s): Analysis, Data, Education, Make, Open Scholarship Tools and Technologies, Reproducibility, Research Data Management Tools, Researchers

Link to resource

Awesome Open Science Resources

Scientific data and tools should, as much as possible, be free as in beer and free as in freedom. The vast majority of science today is …

Author(s): Austin Soplata

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Data, Materials, Policy, Publishing, Reproducibility, Researchers

Link to resource

Bad Science blog on the Guardian webpages

Articles about Bad Science

Author(s): Ben Goldacre

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Blog, Reproducibility Knowledge, Open Science

Link to resource

Bad Science log, post-Guardian

Blogs about bad science

Author(s): Ben Goldacre

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Math & Statistics

Tag(s): Blog

Link to resource

Badges for sharing data and code at Biostatistics: an observational study

Background: The reproducibility policy at the journal Biostatistics rewards articles with badges for data and code sharing. This study …

Author(s): Adrian G. Barnett, Anisa Rowhani-Farid

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Psychology

Tag(s): Data

Link to resource

Badges to Acknowledge Open Practices: A Simple, Low-Cost, Effective Method for Increasing Transparency

Beginning January 2014, Psychological Science gave authors the opportunity to signal open data and materials if they qualified for …

Author(s): Mallory C. Kidwell et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Bargain Basement Bayes

A blog about bayesian Statistics

Author(s): David Funder

Type of resources: Reading

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Blog

Link to resource

Bayes Factor

Bayes factors are somewhat essential to Bayesian statistics. Tony O’Hagan explains their basics

Author(s): O'Hagan

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Bayes for Beginners: Probability and Likelihood

A blog about Bayes for Beginners: Probability and Likelihood

Author(s): C. Randy Gallistel

Type of resources: Blog

Primary user(s): Student

Subject area(s): Math & Statistics, Social Science

Tag(s):

Link to resource

Bayes Rules! An Introduction to Bayesian Modelling in R

Bayesian statistics?! Once an obscure term outside specialized industry and research circles, Bayesian methods are enjoying a …

Author(s): Alicia A. Johnson, Miles Ott, Mine Dogucu

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Bayesian statistics

Link to resource

Bayesian Cognitive Modelling

This site is dedicated to the book “Bayesian Cognitive Modeling: A Practical Course”, published by Cambridge University Press.

Author(s): Michael Lee and Eric-Jan Wagenmakers

Type of resources: Reading

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s): Bayesian statistics

Link to resource

Bayesian inference

This blog post is about a new statistical visualization. This time I’ve tried to illustrate the logic of bayesian updating and …

Author(s): Kristoffer Magnusson

Type of resources: Interactive, Student Guide, Teaching/Learning Strategy

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Blog, Interaction

Link to resource

Bayesian inference for psychology. Part II: Example applications with JASP

Bayesian hypothesis testing presents an attractive alternative to p value hypothesis testing. Part I of this series outlined several …

Author(s): Akash Raj, Alexander Etz, Alexander Ly, Alexandra Sarafoglou, Bruno Boutin, Damian Dropmann, Don van den Bergh, Dora Matzke, Eric-Jan Wagenmakers, Erik-Jan van Kesteren, Frans Meerhoff, Helen Steingroever, Jeffrey N. Rouder, Johnny van Doorn, Jonathon Love, Josine Verhagen, Koen Derks, Maarten Marsman, Martin Šmíra, Patrick Knight, Quentin F. Gronau, Ravi Selker, Richard D. Morey, Sacha Epskamp, Tahira Jamil, Tim de Jong

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Psychology

Tag(s): Analysis, Data, Statistics

Link to resource

Being a Reviewer or Editor for Registered Reports

Experienced Registered Reports editors and reviewers come together to discuss the format and best practices for handling submissions. …

Author(s): Center for Open Science

Type of resources: Lecture

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science

Tag(s): Analysis, Data, Education, Materials, Open Scholarship Tools and Technologies, OSF, Publishing, Reproducibility, Research Data Management Tools, Researchers

Link to resource

Ben Goldacre: Battling Bad Science

A video about battling bad science

Author(s): TED/Ben Goldacre

Type of resources: Video

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Video

Link to resource

Benefits of Open and High-Powered Research Outweigh Costs

Several researchers recently outlined unacknowledged costs of open science practices, arguing these costs may outweigh benefits and …

Author(s): Etienne P. LeBel, Lorne Campbell and Timothy J. Loving

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Berlin|Oxford Summer School 2020

This repository contains materials and talks for the Berlin|Oxford Summer School 2020 from 28th September to 1st of October

Author(s): Toelch et al.

Type of resources: Lesson, Module

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Open Research

Link to resource

Best practices in data analysis and sharing in neuroimaging using MRI

Given concerns about the reproducibility of scientific findings, neuroimaging must define best practices for data analysis, results …

Author(s): Thomas E. Nichols et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Life Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Best research practices in psychology: Illustrating epistemological and pragmatic considerations with the case of relationship science.

In recent years, a robust movement has emerged within psychology to increase the evidentiary value of our science. This movement, which …

Author(s): Finkel, E. J., Eastwick, P. W., & Reis, H. T.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Better P-curves: Making P-curve Analysis More Robust to Errors, Fraud, and Ambitious P-hacking, a Reply to Ulrich and Miller (2015)

When studies examine true effects, they generate right-skewed p-curves, distributions of statistically significant results with more …

Author(s): Uri Simonsohn, Joseph P Simmons, Leif D Nelson

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s):

Link to resource

Beyond kindness: A proposal for the flourishing of science and scientists

We argue that many of the crises currently afflicting science can be associated with a present failure of science to sufficiently …

Author(s): Flourishing Science Think Tank

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Academia, Flourishing, Ethics, Culture, Management

Link to resource

Beyond Power Calculations: Assessing Type S (Sign) and Type M (Magnitude) Errors

Statistical power analysis provides the conventional approach to assess error rates when designing a research study. However, power …

Author(s): Andrew Gelman1 and John Carlin

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Math & Statistics, Social Science

Tag(s):

Link to resource

Big Correlations in Little Studies: Inflated fMRI Correlations Reflect Low Statistical Power-Commentary on Vul Et Al. (2009)

Vul, Harris, Winkielman, and Pashler (2009), (this issue) argue that correlations in many cognitive neuroscience studies are grossly …

Author(s): Tal Yarkoni

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Life Science, Social Science

Tag(s):

Link to resource

Big team science initiatives: A catalyst for trustworthy advancements in IO psychology

Keener et al. (2023) raise concerns about the trustworthiness of Industrial/Organizational (IO) Psychology research and related fields …

Author(s): Liana M. Kreamer, Haley R. Cobb, Christopher Castille, Joshua Cogswell

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Social Science

Tag(s): Big Team Science Initiatives, Big Team Science, Large Collaborations, Team Science

Link to resource

Biomedical publishing: Past historic, present continuous, future conditional

Academic journals have been publishing the results of biomedical research for more than 350 years. Reviewing their history reveals that …

Author(s): Richard Sever

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Life Science, Social Science

Tag(s): Impact Factor, Publishing, Open Access, Preprints

Link to resource

Bite-Size Science and Its Undesired Side Effects

Short and rapid publication of research findings has many advantages. However, there is another side of the coin that needs careful …

Author(s): Marco Bertamini, Marcus R. Munafò

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Boas práticas em ciência

Reprodutibilidade e replicabilidade de estudos científicos. Randomização, cegamento, planejamento amostral e pré-registro de projeto. …

Author(s): Ana Paula Herrmann and Mailton França de Vasconcelos

Type of resources: Syllabus

Primary user(s): Teacher

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Bradley Efron: Frequentist accuracy of Bayesian estimates

A video about Frequentist accuracy of Bayesian estimates

Author(s): RoyalStatSoc/Bradley Efron

Type of resources: Student Guide, Teaching/Learning Strategy, Video

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Video, Bayesian

Link to resource

Breaking free: How preregistration hurts scholars and science

Pre-registration has become an increasingly popular proposal to address concerns regarding questionable research practices. Yet …

Author(s): Rose McDermott

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Preregistration, Critique

Link to resource

Bringing institutional multidisciplinary repositories into the FAIR picture

The FAIR Principles are a set of good practices to improve the reproducibility and quality of data in an Open Science context. …

Author(s): Fernando Aguilar Gómez, Isabel Bernal

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Information Technology, Scientific Data, Software

Link to resource

Bringing the (pre)registration revolution to graduate training

Preregistration, which involves documentation of hypotheses, methods, and plans for data analysis prior to data collection or analysis, …

Author(s): Jennifer Tackett, Cassandra Brandes, Elizabeth Dworak, Allison Shields

Type of resources: Lesson Plan, Teaching/Learning Strategy

Primary user(s): Teacher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Preregistration, Graduate Training

Link to resource

Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs

Effect sizes are the most important outcome of empirical studies. Most articles on effect sizes highlight their importance to …

Author(s): Daniel Lakens

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s):

Link to resource

Calculating the overlap of two normal distributions using monte carlo intergration

I read this post over at the blog Cartesian Faith about Probability and Monte Carlo methods. The post describe how to numerically …

Author(s): Kristoffer Magnusson

Type of resources: Reading, Student Guide, Teaching/Learning Strategy, R code

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Blog, Tutorial

Link to resource

Campbell Open and Reproducible Science Syllabus

The purpose of this course is to acquaint students with recent developments in open science and reproducibility of the research …

Author(s): Lorne Campbell

Type of resources: Syllabus

Primary user(s): Teacher

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution; Open Science; Conceptual and statistical knowledge

Link to resource

Capstone: Advanced General

How do psychologists determine what is true and what is false about human behavior, affect, and cognition? The question encompasses …

Author(s): Katie Corker

Type of resources: Syllabus

Primary user(s): Teacher

Subject area(s): Social Science

Tag(s): Transparency, Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Cargo cult science.

some remarks on science, pseudoscience, and learning how to not fool yourself. Caltech’s 1974 commencement address

Author(s): Richard Feynman

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Carpentries Instructor Training

A two-day introduction to modern evidence-based teaching practices, built and maintained by the Carpentry community.

Author(s): Aleksandra Nenadic, Alexander Konovalov, Alistair John Walsh, Allison Weber, amoskane, Amy E. Hodge, Andrew B. Collier, Anita Schürch, AnnaWilliford, Ariel Rokem, Brian Ballsun-Stanton, Callin Switzer, Christian Brueffer, Christina Koch, Christopher Erdmann, Colin Morris, Dan Allan, DanielBrett, Danielle Quinn, Darya Vanichkina, davidbenncsiro, David Jennings, Eric Jankowski, Erin Alison Becker, Evan Peter Williamson, François Michonneau, Gerard Capes, Greg Wilson, Ian Lee, Jason M Gates, Jason Williams, Jeffrey Oliver, Joe Atzberger, John Bradley, John Pellman, Jonah Duckles, Jonathan Bradley, Karen Cranston, Karen Word, Kari L Jordan, Katherine Koziar, Katrin Leinweber, Kees den Heijer, Laurence, Lex Nederbragt, Maneesha Sane, Marie-Helene Burle, Mik Black, Mike Henry, Murray Cadzow, naught101, Neal Davis, Neil Kindlon, Nicholas Tierney, Nicolás Palopoli, Noah Spies, Paula Andrea Martinez, Petraea, Rayna Michelle Harris, Rémi Emonet, Rémi Rampin, Sarah Brown, Sarah M Brown, Sarah Stevens, satya-vinay, Sean, Serah Anne Njambi Kiburu, Stefan Helfrich, Stéphane Guillou, Steve Moss, Ted Laderas, Tiago M. D. Pereira, Toby Hodges, Tracy Teal, Yo Yehudi

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science, Education, Higher Education, Measurement and Data

Tag(s): Analysis, Data, Education, Open Scholarship Tools and Technologies, Reproducibility, Research Data Management Tools, Researchers

Link to resource

Challenges and opportunities in preregistration of coordinated data analysis: A tutorial and template

The credibility revolution in social science has led to the recommendation and adoption of practices to increase the replicability of …

Author(s): Emily C. Willroth, Eileen K. Graham, Daniel K. Mroczek

Type of resources: Student Guide

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Preregistration, Replicability, Template

Link to resource

Change starts with journal editors: In response to Makel (2014).

The editors of the Journal of Advanced Academics comment on Makel (2014). The replicability crisis in psychology is summarized in terms …

Author(s): Matthew T. McBee and Michael S. Matthews

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Changing the Culture of Peer Review for a More Inclusive and Equitable Psychological Science

Peer review is a core component of scientific progression. Although peer review ideally improves research and promotes rigor, it also …

Author(s): Reviewer Zero

Type of resources: Reading

Primary user(s): Researchers, journal editors

Subject area(s): Social Science

Tag(s): Peer review, open review, diversity, inclusion, equity

Link to resource

Chaos in the brickyard

A paper about open science and novelty

Author(s): Forscher, B. K.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Characteristics of Clinical Trials Registered in ClinicalTrials.gov, 2007-2010

Context Recent reports highlight gaps between guidelines-based treatment recommendations and evidence from clinical trials that …

Author(s): Robert M. Califf, Deborah A. Zarin, Judith M. Kramer, Rachel E. Sherman, Laura H. Aberle, Asba Tasneem

Type of resources: Reading

Primary user(s): Student, Teacher, Administrator, Librarian

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Clinical Trials, Preregistration

Link to resource

Choice of analysis pathway dramatically affects statistical outcomes in breaking continuous flash suppression

Breaking Continuous Flash Suppression (bCFS) has been adopted as an appealing means to study human visual awareness, but the literature …

Author(s): Guido Hesselmann, Isabell Wartenburger, James Allen Kerr, Philipp Sterzer, Romy Räling

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Psychology

Tag(s): Data

Link to resource

Choosing prediction over explanation in psychology: Lessons from machine learning.

Psychology has historically been concerned, first and foremost, with explaining the causal mechanisms that give rise to behavior. …

Author(s): Yarkoni, T., & Westfall, J.

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Chrome Book Data Science

Chromebook Data Science (CBDS) is a free, massive open online educational program offered through Leanpub to help anyone who can read, …

Author(s): Jeff Leek, Ashley Johnson, Shannon Ellis, Aboozar Hadavand, John Muschelli, Sean Kross, Leo Collado-Torres, Leah Jager, Sarah McClymont, Leslie Myint

Type of resources: Full Course, Lesson, Module

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Life Science, Math & Statistics, Social Science

Tag(s): Data Science Tool

Link to resource

Circle of Willis (episode with Sinime Vazire)

Welcome to Episode 8, where I talk to SIMINE VAZIRE, Associate Professor of Psychology at the University of California at Davis, about …

Author(s): Jim Coan

Type of resources: Student Guide, Teaching/Learning Strategy, Podcast

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Podcast, Reproducibility Crisis and Credibility Revolution

Link to resource

Citing and Being Cited: Data & Code Edition

Introduction to citations as a presentation. Citing data and code as well as getting citations for data and code.

Author(s): Vicky Steeves

Type of resources: Lesson

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Citation, Data, Materials, Open Scholarship Guidelines, Repositories, Research Data Management Tools, Researchers

Link to resource

Ciência aberta e reprodutível

Esta disciplina destina-se a alunos em níveis de Mestrado e Doutorado do Programade Pós-Graduação em Ciências da Saúde: Cardiologia e …

Author(s): Daniel Umpierre de Moraes

Type of resources: Syllabus

Primary user(s): Teacher

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Clinical trial registration and reporting: a survey of academic organizations in the United States

Many clinical trials conducted by academic organizations are not published, or are not published completely. Following the US Food and …

Author(s): Anthony Keyes, Audrey Omar, Carrie Dykes, Daniel E. Ford, Diane Lehman Wilson, Evan Mayo-Wilson, G. Caleb Alexander, Hila Bernstein, James Heyward, Jesse Reynolds, Keren Dunn, Leah Silbert, M. E. Blair Holbein, Nidhi Atri, Niem-Tzu (Rebecca) Chen, Sarah White, Yolanda P. Davis

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Health, Medicine and Nursing

Tag(s): Publishing, Reporting

Link to resource

Clinical trial registration: A statement from the International Committee of Medical Journal Editors

Altruism and trust lie at the heart of research on human subjects. Altruistic individuals volunteer for research because they trust …

Author(s): Catherine De Angelis, Jeffrey M Drazen, Frank A Frizelle, Charlotte Haug, John Hoey, Richard Horton, Sheldon Kotzin, Christine Laine, Ana Marusic, A John PM Overbeke, Torben V Schroeder, Hal C Sox, Martin B Van Der Weyden

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Preregistration, Medicine, Clinical Trial Registration

Link to resource

Clinical trial registration: Looking back and moving ahead

In 2005, the International Committee of Medical Journal Editors (ICMJE) initiated a policy requiring investigators to deposit …

Author(s): Christine Laine, Richard Horton, Catherine D DeAngelis, Jeffrey M Drazen, Frank A Frizelle, Fiona Godlee, Charlotte Haug, Paul C Hébert, Sheldon Kotzin, Ana Marusic, Peush Sahni, Torben V Schroeder, Harold C Sox, Martin B Van Der Weyden, Freek WA Verheugt

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Trial Registration, Medicine, Preregistration

Link to resource

Clinical trial registration: transparency is the watchword

All parties associated in clinical trials—patients, doctors, scientists, industry—share a common desire for a vigorous clinical …

Author(s): Ida Sim, An-Wen Chan, A Metin Gülmezoglu, Tim Evans, Tikki Pang

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Trial Registration, Medicine, Preregistration

Link to resource

Cognitive Science StackExchange site (for psychology Q&A)

A website about questions on psychology

Author(s): Anon

Type of resources: Student Guide, Teaching/Learning Strategy

Primary user(s): Student, Teacher, Adminstrator, Parent, Librarian

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Website

Link to resource

Common statistical tests are linear models (or: how to teach stats)

This document is summarised in the table below. It shows the linear models underlying common parametric and “non-parametric” tests. …

Author(s): Jonas Kristoffer Lindeløv

Type of resources: Reading

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s): Statistics

Link to resource

COMPare: a prospective cohort study correcting and monitoring 58 misreported trials in real time

Discrepancies between pre-specified and reported outcomes are an important source of bias in trials. Despite legislation, guidelines …

Author(s): Aaron Dale, Anna Powell-Smith, Ben Goldacre, Carl Heneghan, Cicely Marston, Eirion Slade, Henry Drysdale, Ioan Milosevic, Kamal R. Mahtani, Philip Hartley

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Health, Medicine and Nursing

Tag(s): Bias, Data, Preregistration, Registration, Reporting, Reporting Guidelines, Reproducibility

Link to resource

Comparing Analysis Blinding With Preregistration in the Many-Analysts Religion Project

In psychology, preregistration is the most widely used method to ensure the confirmatory status of analyses. However, the method has …

Author(s): Alexandra Sarafoglou, Suzanne Hoogeveen, Eric-Jan Wagenmakers

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Open Science, Metascience, Replication Crisis, Many Analysts, Open Data, Open Materials, Preregistration

Link to resource

Comparing Published Scientific Journal Articlesto Their Pre-print VersionsMart

Academic publishers claim that they add value to scholarly communications by coordinating reviews and contributing and enhancing text …

Author(s): Martin Klein, Peter Broadwell, Sharon E. Farb, Todd Grappone

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Knowledge

Link to resource

Comparison of Preregistration Platforms

Preregistration forces researchers to front-load a lot of decision-making to an early stage of a project. Choosing which …

Author(s): Steve Haroz

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Preregistration

Link to resource

Comparison of registered and published outcomes in randomized controlled trials: a systematic review

Clinical trial registries can improve the validity of trial results by facilitating comparisons between prospectively planned and …

Author(s): Christopher W. Jones, Lukas G. Keil, Melissa C. Caughey, Timothy F. Platts-Mills, Wesley C. Holland

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Health, Medicine and Nursing

Tag(s): Data, Publishing

Link to resource

Comparison of Registered and Published Primary Outcomes in Randomized Controlled Trials

Context: As of 2005, the International Committee of Medical Journal Editors required investigators to register their trials prior to …

Author(s): Sylvain Mathieu, Isabelle Boutron, David Moher, Douglas G Altman, Philippe Ravaud

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Life Science, Social Science

Tag(s):

Link to resource

Computational social science is growing up: why puberty consists of embracing measurement validation, theory development, and open science practices

Puberty is a phase in which individuals often test the boundaries of themselves and surrounding others and further define their …

Author(s): Timon Elmer

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Computational Social Science, Passive Measurement, Digital Trace Data, Validity, Open science Practices, Metascience

Link to resource

Conducting Research With People in Lower-Socioeconomic-Status Contexts

In recent years, the field of psychology has increasingly recognized the importance of conducting research with …

Author(s): Lydia F. Emery, David M. Silverman, Rebecca M. Carey

Type of resources: Reading, Student Guide

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Social Science

Tag(s): Diversity, Participants, Socioeconomic Status, SES

Link to resource

Connecting Research Tools to the Open Science Framework (OSF)

This webinar (recorded Sept. 27, 2017) introduces how to connect other services as add-ons to projects on the Open Science Framework …

Author(s): Center for Open Science

Type of resources: Lecture

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science

Tag(s): Analysis, Data, Education, Materials, Open Scholarship Tools and Technologies, OSF, Publishing, Reproducibility, Research Data Management Tools, Researchers

Link to resource

Consequences of Low Statistical Power

This video will go over three issues that can arise when scientific studies have low statistical power. All materials shown in the …

Author(s): Center for Open Science

Type of resources: Lecture

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science

Tag(s): Analysis, Data, Education, Materials, Open Scholarship Tools and Technologies, OSF, Publishing, Reproducibility, Research Data Management Tools, Researchers

Link to resource

Consequences of prejudice against the null hypothesis.

Examined the consequences of prejudice against accepting the null hypothesis through (a) a mathematical model intended to stimulate the …

Author(s): Anthony G. Greenwald

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Conservative tests under satisficing models of publication bias.

Publication bias leads consumers of research to observe a selected sample of statistical estimates calculated by producers of research. …

Author(s): Justin McCrary, Garret Christensen, Daniele Fanelli

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Constraints on generality statements are needed to define direct replication

Whether or not a replication attempt counts as “direct” often cannot be determined definitively after the fact as a result of …

Author(s): Daniel J. Simons, Yuichi Shoda and D. Stephen Lindsay

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Construct validation in social and personality research: Current practice and recommendations.

The verity of results about a psychological construct hinges on the validity of its measurement, making construct validation a …

Author(s): Flake, J. K., Pek, J., & Hehman, E.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Continuously Cumulating Meta-Analysis and Replicability

The current crisis in scientific psychology about whether our findings are irreproducible was presaged years ago by Tversky and …

Author(s): Sanford L. Braver, Felix J. Thoemmes, Robert Rosenthal

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Correcting for Bias in Psychology: A Comparison of Meta-Analytic Methods

Publication bias and questionable research practices in primary research can lead to badly overestimated effects in meta-analysis. …

Author(s): Evan C. Carter, Felix D. Schönbrodt, Will M. Gervais, and Joseph Hilgard

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Correlational Effect Size Benchmarks

Effect size information is essential for the scientific enterprise and plays an increasingly central role in the scientific process. We …

Author(s): Bosco, F. A., Aguinis, H., Singh, K., Field, J. G., & Pierce, C. A.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

COS Registered Reports Portal

Registered Reports: Peer review before results are known to align scientific values and practices.

Registered Reports is a publishing …

Author(s): Center for Open Science, David Mellor

Type of resources: Student Guide

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Preregistration, Publishing Models, Registered Reports, Reproducibility

Link to resource

Course materials | Project TIER

Learn about courses, in a wide range of fields at a variety of institutions, where principles and resources from Project TIER have been …

Author(s): Project TIER

Type of resources: Module, Syllabus

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution; Open Science; Conceptual and statistical knowledge

Link to resource

course syllabi for open and reproducible science

A collection of course syllabi from any discipline featuring content to examine or improve open and reproducible research practices. …

Author(s): Ball et al.

Type of resources: Reading, syllabus

Primary user(s): Student, Teacher, Researchers

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Collection, Reproducibility Knowledge

Link to resource

Credibility of preprints: an interdisciplinary survey of researchers

Preprints increase accessibility and can speed scholarly communication if researchers view them as credible enough to read and use. …

Author(s): Courtney K. Soderberg et al.

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Social Science

Tag(s): Preprints

Link to resource

CREP project

CREP’s mission is to provide training, support, and professional growth opportunities for students and instructors completing …

Author(s): Jon Grahe et al.

Type of resources: Primary Source, Reading, Project

Primary user(s): Student, Teacher, Researchers

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Reproducibility Knowledge, OSF Project

Link to resource

CREP tutorials

This playlist was created to help students and their instructors complete CREP projects.

Author(s): Dr Jordan Wagge

Type of resources: Videos

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility and replicability

Link to resource

Crisis research, fast and slow

A blog about the crisis of research

Author(s): Anna Scheel

Type of resources: Reading

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Open science

Link to resource

CrossValidated StackExchange site (for statistics Q&A)

A website about questions on statistics

Author(s): Anon

Type of resources: Student Guide, Teaching/Learning Strategy, Unit of Study, Website

Primary user(s): Student, Teacher, Librarian

Subject area(s): Math & Statistics

Tag(s): Website

Link to resource

Crowdsourced research: Many hands make tight work

Crowdsourcing research can balance discussions, validate findings and better inform policy, say Raphael Silberzahn and Eric L. Uhlmann.

Author(s): Silberzahn, R., & Uhlmann, E.L.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Curate Science

Curate Science is a unified curation system and platform to verify that research is transparent and credible. It will allow …

Author(s):

Type of resources: Data Set

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Funders, Open Scholarship Guidelines, Publishers, Reproducibility, Research Administration, Researchers, Research Integrity

Link to resource

Curating Research Assets: A Tutorial on the Git Version Control System

Recent calls for improving reproducibility have increased attention to the ways in which researchers curate, share, and collaborate on …

Author(s): Matti Vuorre and James P. Curley

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s):

Link to resource

Current Incentives for Scientists Lead to Underpowered Studies with Erroneous Conclusions

We can regard the wider incentive structures that operate across science, such as the priority given to novel findings, as an ecosystem …

Author(s): Andrew D. Higginson, Marcus R. Munafò

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Biology

Tag(s): Careers, Careers in Research, Drug Discovery, Ecosystems, Peer Review, Research Assessment, Research Errors, Scientists

Link to resource

Curtailing the Use of Preregistration: A Misused Term

Improving the usability of psychological research has been encouraged through practices such as prospectively registering research …

Author(s): Danielle B. Rice, David Moher

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Registration, Open Science, Preregistration, Registered Reports

Link to resource

DAGitty

DAGitty is a browser-based environment for creating, editing, and analyzing causal diagrams (also known as directed acyclic graphs or …

Author(s): Johannes Texor, Maciej Liśkiewicz and Benito van der Zander

Type of resources: Teaching/Learning Strategy, R code

Primary user(s): Researcher

Subject area(s): Applied Science, Life Science, Math & Statistics, Social Science

Tag(s): Website

Link to resource

Data Analysis and Visualization in Python for Ecologists

Python is a general purpose programming language that is useful for writing scripts to work effectively and reproducibly with data. …

Author(s): Maxim Belkin, Tania Allard

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science, Measurement and Data

Tag(s): Analysis, Data, Education, Open Scholarship Tools and Technologies, Python, Reproducibility, Research Data Management Tools, Researchers

Link to resource

Data Analysis and Visualization in R for Ecologists

Data Carpentry lesson from Ecology curriculum to learn how to analyse and visualise ecological data in R. Data Carpentry’s aim is to …

Author(s): Ankenbrand, Markus, Arindam Basu, Ashander, Jaime, Bahlai, Christie, Bailey, Alistair, Becker, Erin Alison, Bledsoe, Ellen, Boehm, Fred, Bolker, Ben, Bouquin, Daina, Burge, Olivia Rata, Burle, Marie-Helene, Carchedi, Nick, Chatzidimitriou, Kyriakos, Chiapello, Marco, Conrado, Ana Costa, Cortijo, Sandra, Cranston, Karen, Cuesta, Sergio Martínez, Culshaw-Maurer, Michael, Czapanskiy, Max, Daijiang Li, Dashnow, Harriet, Daskalova, Gergana, Deer, Lachlan, Direk, Kenan, Dunic, Jillian, Elahi, Robin, Fishman, Dmytro, Fouilloux, Anne, Fournier, Auriel, Gan, Emilia, Goswami, Shubhang, Guillou, Stéphane, Hancock, Stacey, Hardenberg, Achaz Von, Harrison, Paul, Hart, Ted, Herr, Joshua R., Hertweck, Kate, Hodges, Toby, Hulshof, Catherine, Humburg, Peter, Jean, Martin, Johnson, Carolina, Johnson, Kayla, Johnston, Myfanwy, Jordan, Kari L, K. A. S. Mislan, Kaupp, Jake, Keane, Jonathan, Kerchner, Dan, Klinges, David, Koontz, Michael, Leinweber, Katrin, Lepore, Mauro Luciano, Lijnzaad, Philip, Li, Ye, Lotterhos, Katie, Mannheimer, Sara, Marwick, Ben, Michonneau, François, Millar, Justin, Moreno, Melissa, Najko Jahn, Obeng, Adam, Odom, Gabriel J., Pauloo, Richard, Pawlik, Aleksandra Natalia, Pearse, Will, Peck, Kayla, Pederson, Steve, Peek, Ryan, Pletzer, Alex, Quinn, Danielle, Rajeg, Gede Primahadi Wijaya, Reiter, Taylor, Rodriguez-Sanchez, Francisco, Sandmann, Thomas, Seok, Brian, Sfn_brt, Shiklomanov, Alexey, Shivshankar Umashankar, Stachelek, Joseph, Strauss, Eli, Sumedh, Switzer, Callin, Tarkowski, Leszek, Tavares, Hugo, Teal, Tracy, Theobold, Allison, Tirok, Katrin, Tylén, Kristian, Vanichkina, Darya, Voter, Carolyn, Webster, Tara, Weisner, Michael, White, Ethan P, Wilson, Earle, Woo, Kara, Wright, April, Yanco, Scott, Ye, Hao

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science, Ecology, Measurement and Data

Tag(s): Analysis, Data, Education, Open Scholarship Tools and Technologies, R, Reproducibility, Research Data Management Tools, Researchers

Link to resource

Data Analysis and Visualization with Python for Social Scientists

Python is a general purpose programming language that is useful for writing scripts to work effectively and reproducibly with data. …

Author(s): Geoffrey Boushey, Stephen Childs

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science, Measurement and Data

Tag(s): Analysis, Data, Education, Open Scholarship Tools and Technologies, Python, Reproducibility, Research Data Management Tools, Researchers

Link to resource

Data availability, reusability, and analytic reproducibility: evaluating the impact of a mandatory open data policy at the journal Cognition

Access to data is a critical feature of an efficient, progressive and ultimately self-correcting scientific ecosystem. But the extent …

Author(s): Alicia Hofelich Mohr, Bria Long, Elizabeth Clayton, Erica J. Yoon, George C. Banks, Gustav Nilsonne, Kyle MacDonald, Mallory C. Kidwell, Maya B. Mathur, Michael C. Frank, Michael Henry Tessler, Richie L. Lenne, Sara Altman, Tom E. Hardwicke

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Information Science

Tag(s): Data, Policy, Reproducibility

Link to resource

Data Carpentry

Data Carpentry trains researchers in the core data skills for efficient, shareable, and reproducible research practices. We run …

Author(s): Data Carpentry Community

Type of resources: Full Course

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Analyses, Data, Reproducibility, Research Data Management, Researchers

Link to resource

Data Carpentry for Biologists

The Biology Semester-long Course was developed and piloted at the University of Florida in Fall 2015. Course materials include …

Author(s): Ethan White, Zachary Brym

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science, Biology, Measurement and Data

Tag(s): Analysis, Data, Education, Open Scholarship Tools and Technologies, R, Reproducibility, Research Data Management Tools, Researchers, SQL

Link to resource

Data Cleaning and Management Using OpenRefine

Course materials on using OpenRefine, a powerful tool for cleaning and transforming tabular data.

Author(s): Nick Wolf, Vicky Steeves

Type of resources: Activity/Lab

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Analysis, Data, Open Scholarship Tools and Technologies, Research Data Management Tools, Researchers

Link to resource

Data Cleaning with OpenRefine for Ecologists

A part of the data workflow is preparing the data for analysis. Some of this involves data cleaning, where errors in the data are …

Author(s): Cam Macdonell, Deborah Paul, Phillip Doehle, Rachel Lombardi

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science, Measurement and Data

Tag(s): Analysis, Data, Education, Open Scholarship Tools and Technologies, Reproducibility, Research Data Management Tools, Researchers

Link to resource

Data Intro for Archivists

This Library Carpentry lesson introduces archivists to working with data. At the conclusion of the lesson you will: be able to explain …

Author(s): James Baker, Jeanine Finn, Jenny Bunn, Katherine Koziar, Noah Geraci, Scott Peterson

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Applied Science, Information Science, Measurement and Data

Tag(s): Analysis, Data, Education, Librarians, Open Scholarship Tools and Technologies, Reproducibility, Research Data Management Tools

Link to resource

Data Is Present: Open Workshops and Hackathons

Original data has become more accessible thanks to cultural and technological advances. On the internet, we can find innumerable data …

Author(s): Pablo Bernabeu

Type of resources: Activity/Lab

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Data, Librarians, Open Data, Open Source Software, Publishers, Researchers, Visualization

Link to resource

Data management

A series of video of data management

Author(s): Patrick S. Forscher

Type of resources: Video

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s): Data management

Link to resource

Data Management & Reproducibility

Introduction to data management and reproducibility for researchers as a presentation.

Author(s): Vicky Steeves

Type of resources: Lesson

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Analysis, Data, Open Scholarship Tools and Technologies, Reproducibility, Research Data Management, Researchers

Link to resource

Data Management and Data Sharing in Psychological Science: Revision of the DGPs Recommendations

Providing access to research data collected as part of scientific publications and publicly funded research projects is now regarded as …

Author(s): Mario Gollwitzer, Andrea Abele-Brehm, Christian Fiebach, Roland Ramthun, Anne Scheel, Felix Schönbrodt, Ulf Steinberg

Type of resources: Reading

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Open data

Link to resource

Data Management for Psychological Science: A Crowdsourced Syllabus

Data management - including data preparation, cleaning, storage, and sharing - is critical to psychological research. Despite its …

Author(s): Debolt et al.

Type of resources: Syllabus

Primary user(s): Teacher

Subject area(s): Applied Science, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Data management, Reproducible Analyses

Link to resource

Data management plans as linked open data: exploiting ARGOS FAIR and machine actionable outputs in the OpenAIRE research graph

Background Open Science Graphs (OSGs) are scientific knowledge graphs representing different entities of the research lifecycle (e.g. …

Author(s): Elli Papadopoulou, Alessia Bardi, George Kakaletris, Diamadis Tziotzios, Paolo Manghi, Natalia Manola

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Research Data Management, Data Management Plans, FAIR DMPs, Machine Actionable, Research Graphs, Knowledge Graphs, Open Science

Link to resource

Data Management with SQL for Ecologists

Databases are useful for both storing and using data effectively. Using a relational database serves several purposes. It keeps your …

Author(s): Christina Koch, Donal Heidenblad, Katy Felkner, Rémi Rampin, Timothée Poisot

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science, Measurement and Data

Tag(s): Analysis, Data, Databases, Education, Open Scholarship Tools and Technologies, Reproducibility, Research Data Management Tools, Researchers, SQL

Link to resource

Data Management with SQL for Social Scientists

This is an alpha lesson to teach Data Management with SQL for Social Scientists, We welcome and criticism, or error; and will take your …

Author(s): Peter Smyth

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science, Measurement and Data, Social Science

Tag(s): Analysis, Data, Education, Open Scholarship Tools and Technologies, Reproducibility, Research Data Management Tools, Researchers, SQL

Link to resource

Data Organization in Spreadsheets for Ecologists

Good data organization is the foundation of any research project. Most researchers have data in spreadsheets, so it’s the place that …

Author(s): Christie Bahlai, Peter R. Hoyt, Tracy Teal

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science, Measurement and Data

Tag(s): Analysis, Data, Education, Open Scholarship Tools and Technologies, Reproducibility, Research Data Management Tools, Researchers, Spreadsheets

Link to resource

Data Organization in Spreadsheets for Social Scientists

Lesson on spreadsheets for social scientists. Good data organization is the foundation of any research project. Most researchers have …

Author(s): David Mawdsley, Erin Becker, François Michonneau, Karen Word, Lachlan Deer, Peter Smyth

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Information Science, Measurement and Data, Social Science

Tag(s): Analysis, Data, Education, Open Scholarship Tools and Technologies, Reproducibility, Research Data Management Tools, Researchers, Spreadsheets

Link to resource

Data overuse in aging research: Emerging issues and potential solutions

Aging and lifespan development researchers have been fortunate to have public access to many longitudinal datasets. These data are …

Author(s): Daniel K. Mroczek, Sara J. Weston, Eileen K. Graham, Emily C. Willroth

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Life Science, Social Science

Tag(s): Lifespan Development, Aging, Data Overuse, Bias

Link to resource

Data peeking without p-hacking

Blog post going over data peeking without p-hacking

Author(s): Daniel Lakens

Type of resources: Reading, R Code

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Education, Life Science, Math & Statistics, Social Science

Tag(s): Blog, Code, Open Science, Reproducibility Crisis and Credibility Revolution

Link to resource

Data policies of highly-ranked social science journals

By encouraging and requiring that authors share their data in order to publish articles, scholarly journals have become an important …

Author(s): Abigail Schwartz, Dessi Kirilova, Gerard Otalora, Julian Gautier, Mercè Crosas, Sebastian Karcher

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Social Science, Psychology

Tag(s): Data, Data-Sharing Policies, Funder Policies, Open Data, Open Science, Policy

Link to resource

Data reuse and the open data citation advantage

Background. Attribution to the original contributor upon reuse of published data is important both as a reward for data creators and to …

Author(s): Heather A. Piwowar and Todd J. Vision

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s):

Link to resource

Data Sharing by Scientists: Practices and Perceptions

Background Scientific research in the 21st century is more data intensive and collaborative than in the past. It is important to study …

Author(s): Arsev Umur Aydinoglu, Carol Tenopir, Eleanor Read, Kimberly Douglass, Lei Wu, Maribeth Manoff, Mike Frame, Suzie Allard

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Ecology, Social Science

Tag(s): Data, Data Acquisition, Data Management, Data Processing, Ecology and Environmental Sciences, Europe, Scientists, Social Sciences, Surveys

Link to resource

Data sharing in PLOS ONE: An analysis of Data Availability Statements

A number of publishers and funders, including PLOS, have recently adopted policies requiring researchers to share the data underlying …

Author(s): Alicia Livinski, Christopher W. Belter, Douglas J. Joubert, Holly Thompson, Lisa M. Federer, Lissa N. Snyders, Ya-Ling Lu

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Computer Science, Health, Medicine and Nursing, Information Science, Social Science

Tag(s): Data, Institutional Repositories, Open Data, Policy, Public Policy, Publishing, Randomized Controlled Trials, Reproducibility, Research Validity, Science Policy, Scientific Publishing

Link to resource

Data Sharing in Psychology: A Survey on Barriers and Preconditions

Despite its potential to accelerate academic progress in psychological science, public data sharing remains relatively uncommon. In …

Author(s): Houtkoop et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Data sharing practices in collaborative human genomic research in low- and middle-income countries: A systematic review protocol

Introduction The practice of creating large databases has become increasingly common by combining research participants’ data into …

Author(s): Deborah Ekusai-Sebatta, Moses Ocan, Shenuka Singh, David Kyaddondo, Dickens Akena, Loyce Nakalembe, Robert Apunyo, Alison Annet Kinengyere, Eve Namisango, Ekwaro A. Obuku, Erisa Mwaka

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Life Science, Social Science

Tag(s): Human Genomics, Low and Middle Income Countries, Systematic Reviews, Database Searching, Science Policy, Research Ethics, Research Design, Survey Research

Link to resource

Data tutorial: Power analysis using G*Power

A tutorial on a priori and sensitivity power analyses using G*power

Author(s): Andrew Livingstone

Type of resources: Video

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Power analysis

Link to resource

Data Wrangling and Processing for Genomics

Data Carpentry lesson to learn how to use command-line tools to perform quality control, align reads to a reference genome, and …

Author(s): Adam Thomas, Ahmed R. Hasan, Aniello Infante, Anita Schürch, dbmarchant, Dev Paudel, Erin Alison Becker, Fotis Psomopoulos, François Michonneau, Gaius Augustus, Gregg TeHennepe, Jason Williams, Jessica Elizabeth Mizzi, Karen Cranston, Kari L Jordan, Kate Crosby, Kevin Weitemier, Lex Nederbragt, Luis Avila, Peter R. Hoyt, Rayna Michelle Harris, Ryan Peek, Sheldon John McKay, Sheldon McKay, Taylor Reiter, Tessa Pierce, Toby Hodges, Tracy Teal, Vasilis Lenis, Winni Kretzschmar

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science, Genetics, Measurement and Data

Tag(s): Analysis, Data, Education, Genomics, Inside Your Classroom, Open Scholarship Tools and Technologies, Reproducibility, Research Data Management Tools, Researchers, Shell

Link to resource

Data: Sharing Is Caring

Data sharing promotes scientific progress by permitting replication of prior scientific analyses and by increasing the return on the …

Author(s): Margaret C. Levenstein and Jared A. Lyle

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s):

Link to resource

Databases and SQL

Software Carpentry lesson that teaches how to use databases and SQL In the late 1920s and early 1930s, William Dyer, Frank Pabodie, and …

Author(s): Amy Brown, Andrew Boughton, Andrew Kubiak, Avishek Kumar, Ben Waugh, Bill Mills, Brian Ballsun-Stanton, Chris Tomlinson, Colleen Fallaw, Daniel Suess, Dan Michael Heggø, Dave Welch, David W Wright, Deborah Gertrude Digges, Donny Winston, Doug Latornell, Erin Alison Becker, Ethan Nelson, Ethan P White, François Michonneau, George Graham, Gerard Capes, Gideon Juve, Greg Wilson, Ioan Vancea, Jake Lever, James Mickley, John Blischak, JohnRMoreau@gmail.com, Jonah Duckles, Jonathan Guyer, Joshua Nahum, Kate Hertweck, Kevin Dyke, lorra, Louis Vernon, Luc Small, Luke William Johnston, Maneesha Sane, Mark Stacy, Matthew Collins, Matty Jones, Mike Jackson, Morgan Taschuk, Patrick McCann, Paula Andrea Martinez, Pauline Barmby, Piotr Banaszkiewicz, Raniere Silva, Ray Bell, Rayna Michelle Harris, Rémi Emonet, Rémi Rampin, Seda Arat, Sheldon John McKay, Sheldon McKay, slimlime, Stephen Davison, Thomas Guignard, Trevor Bekolay

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science, Measurement and Data

Tag(s): Analysis, Data, Education, Open Scholarship Tools and Technologies, Reproducibility, Research Data Management Tools, Researchers, SQL

Link to resource

DEBATE-statistical analysis plans for observational studies

Background All clinical research benefits from transparency and validity. Transparency and validity of studies may increase by …

Author(s): Bart Hiemstra, Christian Gluud, Frederik Keus, Iwan C. C. van der Horst, Jørn Wetterslev

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Health, Medicine and Nursing

Tag(s): Authorship, Data, Data Management, Publishing, Randomized Trial, Research Ethics

Link to resource

Declaration of common standards for the preregistration of animal research—speeding up the scientific progress

Preregistration of studies is a recognized tool in clinical research to improve the quality and reporting of all gained results. In …

Author(s): Céline Heinl, Anna M D Scholman-Végh, David Mellor, Gilbert Schönfelder, Daniel Strech, Steven Chamuleau, Bettina Bert

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Open Science, Preregistration, Animal Research, Preclinical Research, 3R, Research Methods

Link to resource

Deep Dive into Open Scholarship: Collaboration and Replication

This deep dive session on replications and large-scale collaborations introduces a glossary of relevant terms, the problems these …

Author(s): Erin Miller, Jay Carter, Matt Makel, Scott Peters

Type of resources: Lesson

Primary user(s): other

Subject area(s): Education

Tag(s): Open Science, Replication, Research Best Practices, Research Collaboration, Research Replication

Link to resource

Deep Dive into Open Scholarship: Data, Materials, and Code Transparency

In this deep dive session, Dr. Willa van Dijk discusses how transparency with data, materials, and code is beneficial for educational …

Author(s): Willa van Dijk

Type of resources: Lesson

Primary user(s): other

Subject area(s): Education

Tag(s): Code Transparency, Educational Research, Education Research, Inside Your Classroom, Open Code, Open Data, Open Materials, Open Scholarship, Open Science, Research Best Practices

Link to resource

Deep Dive into Open Scholarship: Preprints and OA

In this deep dive session, we discuss the current model of scholarly publishing, and highlight the challenges and limitations of this …

Author(s): Bryan Cook, Stacy Shaw

Type of resources: Lesson

Primary user(s): other

Subject area(s): Education

Tag(s): Open Access, Open Scholarship, Open Science, Preprints, Scholarly Publishing

Link to resource

Deep Dive into Open Scholarship: Preregistration and Registered Reports

In this deep dive session, Amanda Montoya (UCLA) and Karen Rambo-Hernandez (Texas A&M University) introduce the basics of …

Author(s): Amanda Montoya, Karen Rambo-Hernandez

Type of resources: Lesson

Primary user(s): other

Subject area(s): Education

Tag(s): Open Scholarship, Open Science, Preregistration, Registered Reports, Research Best Practices

Link to resource

Deep Dive on Open Practices: Understanding Data Sharing with Sara Hart

As sharing data openly becomes more and more the norm, and not just because of mandates for federal funding, more researchers may …

Author(s): Sara Hart

Type of resources: Lecture

Primary user(s):

Subject area(s): Education

Tag(s): Center for Open Science, Data Sharing, Education, Sara Hart, Unconference 2022

Link to resource

Deep Dive on Open Practices: Understanding Preregistration with Scott Peters & Karen Rambo-Hernandez

In this deep dive session, we introduce the basics of pre-registration: a method for creating a permanent record of a research plan …

Author(s): Karen Rambo-Hernandez, Scott Peters

Type of resources: Lecture

Primary user(s):

Subject area(s): Education

Tag(s): Center for Open Science, Education, Karen Rambo-Hernandez, Open Science, Preregistration, Pre-registration in Education Research, Pre-registration in Science, Scott Peters, Unconference 2022

Link to resource

Deep Dive on Open Practices: Understanding Registered Reports in Education Research

Deep Dive on Open Practices: Understanding Registered Reports in Education Research with Amanda Montoya and Betsy McCoach - Registered …

Author(s): Amanda Montoya, Betsy McCoach

Type of resources: Lecture

Primary user(s):

Subject area(s): Education

Tag(s): Amanda Montoya, Betsy McCoach, Center for Open Science, Education, Open Science, Registered Reports, Registered Reports in Education Research, Unconference 2022

Link to resource

Deep Dive on Open Practices: Understanding Replication in Education Research with Matt Makel

Deep Dive on Open Practices: Understanding Replication in Education Research with Matt Makel - In this deep dive session, we introduce …

Author(s): Matt Makel

Type of resources: Lesson

Primary user(s):

Subject area(s): Education

Tag(s): Center for Open Science, Education, Matt Makel, Open Science, Replication in Education Research, Unconference 2022

Link to resource

Defining and distinguishing validity: Interpretations of score meaning and justifications of test use.

The concept of validity has suffered because the term has been used to refer to 2 incompatible concerns: the degree of support for …

Author(s): G.J. Cizek

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s):

Link to resource

Defining and Growing the Field of Metascience

In this talk, Professor Fidler argues how the field of metascience contrasts with many scientific disciplines because it works in …

Author(s): Fiona Fidler

Type of resources: Lecture

Primary user(s):

Subject area(s): Education

Tag(s): Research

Link to resource

Degrees of Freedom in Planning, Running, Analyzing, and Reporting Psychological Studies: A Checklist to Avoid p-Hacking

The designing, collecting, analyzing, and reporting of psychological studies entail many choices that are often arbitrary. The …

Author(s): Jelte M. Wicherts, Coosje L. S. Veldkamp, Hilde E. M. Augusteijn, Marjan Bakker, Robbie C. M. van Aert, and Marcel A. L. M. van Assen

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Degrees of Freedom Tutorial

A lot of researchers seem to be struggling with their understanding of the statistical concept of degrees of freedom. Most do not …

Author(s): Ron Dotsch

Type of resources: Reading, Student Guide, Teaching/Learning Strategy

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Education, Life Science, Math & Statistics, Social Science

Tag(s): Blog, Tutorial

Link to resource

Dementia, Big Data and Open Science

Although there is clear potential to improve science and innovation systems through big data and open science, barriers still remain …

Author(s): OECD

Type of resources: Reading

Primary user(s):

Subject area(s): Life Science, Biology

Tag(s): Aging Science, Big Data, Consumer Policy, Dementia, E-commerce, E-Government, Health, Innovation, OECD, Openness, Open Science, Science, Security

Link to resource

Der Umgang mit Forschungsdaten im Fach Psychologie: Konkretisierung der DFG-Leitlinien. [Data Management in Psychological Science: Specification of the DFG Guidelines].

Calls for public access to research data have been ongoing for some time. For instance, in their “Recommendations for Secure Storage …

Author(s): Schönbrodt, F., Gollwitzer, M., & Abele-Brehm, A.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Designing for Preregistration: A User-Centered Perspective

The replication crisis—a failure to replicate foundational studies—has sparked a conversation in psychology, HCI, and …

Author(s): Xiaoying Pu, Licheng Zhu, Matthew Kay, Frederick Conrad

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Social Science

Tag(s): Human-Centered Computing, Human Computer Interaction (HCI), Empirical Studies in HCI, Interaction design

Link to resource

Detecting and avoiding likely false-positive findings: A practical guide

Recently there has been a growing concern that many published research findings do not hold up in attempts to replicate them. We argue …

Author(s): Wolfgang Forstmeier, Eric-Jan Wagenmakers and Timothy H. Parker

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Life Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Detecting fraud in social science

He’s been called a “Data vigilante.” In this episode, Prof. Uri Simonsohn describes how he detects fraudulent work in …

Author(s): Julia Galef/Uri Simonsohn

Type of resources: Teaching/Learning Strategy, Unit of Study, Podcast

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Podcast, Reproducibility Knowledge, Reproducibility Crisis and Credibility Revolution

Link to resource

Disclosing and Managing Non-Financial Conflicts of Interest in Scientific Publications

In the last decade, there has been increased recognition of the importance of disclosing and managing non-financial conflicts of …

Author(s): David B Resnik

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian, Journal Editors

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Conflicts of Interest, Financial, Non-Financial, Bias, Ethics, Policy, Journals

Link to resource

Discrepancies in the Registries of Diet vs Drug Trials

This cross-sectional study examines discrepancies between registered protocols and subsequent publications for drug and diet trials …

Author(s): Cara B. Ebbeling, David S. Ludwig, Steven B. Heymsfield

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Health, Medicine and Nursing

Tag(s): Bias, Preregistration, Registration, Reporting, Reporting Guidelines, Reproducibility

Link to resource

Discrepancy review: A feasibility study of a novel peer review intervention to reduce undisclosed discrepancies between registrations and publications

Undisclosed discrepancies often exist between study registrations and their associated publications. Discrepancies can increase risk of …

Author(s): TARG Meta-Research Group and Collaborators

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Peer Review, Outcome Switching, Selective Reporting, Pre-egistration, Trial Registration, Metaresearch

Link to resource

Dissemination and publication of research findings: an updated review of related biases

Objectives To identify and appraise empirical studies on publication and related biases published since 1998; to assess methods to deal …

Author(s): Aj Sutton, C Hing, C Pang, Cs Kwok, F Song, I Harvey, J Ryder, L Hooper, S Parekh, Yk Loke

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Health, Medicine and Nursing

Tag(s): Data, Open Data, Preregistration, Publication Bias, Publishing, Reporting, Systematic Reviews

Link to resource

Distribution of p-values when comparing two groups

An interactive visualisation of the distribution of p-values when comparing two groups

Author(s): Kristoffer Magnusson

Type of resources: Interactive, Reading, Simulation, Student Guide, Teaching/Learning Strategy

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Blog, Interaction, Simulation, Tutorial

Link to resource

Diversifying Research Methods Syllabi

This is a collection of research methodology articles, which are first- or senior-authored by women, to promote diverse perspectives in …

Author(s): Kane et al.

Type of resources: Syllabus

Primary user(s): Teacher

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution; Open Science; Conceptual and statistical knowledge

Link to resource

Do p values lose their meaning in exploratory analyses? It depends how you define the familywise error rate

Several researchers have recently argued that p values lose their meaning in exploratory analyses due to an unknown inflation of the …

Author(s): Mark Rubin

Type of resources: Reading

Primary user(s): Teacher, Librarian, research

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Confirmatory Research, Exploratory Research, Familywise Error Rate, P Values, Type I Errors

Link to resource

Do Pre-analysis Plans Hamper Publication?

Scholars assert that pre-analysis plans (PAPs) generate boring, lab-report style papers and thus hamper publication. We test this claim …

Author(s): George K. Ofosu, Daniel N. Posner

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Social Science

Tag(s): Preregistration, Pre-analysis Plans

Link to resource

Do Pre-Registration and Pre-Analysis Plans Reduce p-Hacking and Publication Bias?

Randomized controlled trials (RCTs) are increasingly prominent in economics, with pre-registration and pre-analysis plans (PAPs) …

Author(s): Abel Brodeur, Nikolai Cook, Jonathan Hartley, Anthony Heyes

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Social Science

Tag(s): Pre-analysis Plan, Pre-registration, p-Hacking, Publication Bias, Research Credibility

Link to resource

Do Statistical Reporting Standards Affect What Is Published? Publication Bias in Two Leading Political Science Journals

We examine the APSR and the AJPS for the presence of publication bias due to reliance on the 0.05 significance level. Our analysis …

Author(s): Alan Gerber and Neil Malhotra

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s):

Link to resource

Do studies of statistical power have an effect on the power of studies?

The long-term impact of studies of statistical power is investigated using J. Cohen’s (1962) pioneering work as an example. We …

Author(s): Sedlmeier, P. & Gigerenzer, G.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Does preregistration improve the credibility of research findings?

Preregistration entails researchers registering their planned research hypotheses, methods, and analyses in a time-stamped document …

Author(s): Mark Rubin

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Forking Paths, HARKing, Multiple Testing, Optional Stopping, P-hacking, Preregistration, Publication Bias

Link to resource

Does use of the CONSORT Statement impact the completeness of reporting of randomised controlled trials published in medical journals? A Cochrane reviewa

Background The Consolidated Standards of Reporting Trials (CONSORT) Statement is intended to facilitate better reporting of randomised …

Author(s): David Moher, Douglas G Altman, Kenneth F Schulz, Larissa Shamseer, Lucy Turner

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Health, Medicine and Nursing

Tag(s): Bias, Inside Your Classroom, Publishing, Reporting, Reporting Bias, Reporting Guidelines

Link to resource

Draft dated 2016-01-11; please see cuLearn for possible updatesp.1PSYC 5601:Contemporary Research in Personality

This course will provide a survey of current personality research and theorywith an emphasis on recent discussions around methods and …

Author(s): John M. Zelenski

Type of resources: Syllabus

Primary user(s): Student, Teacher

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Dyadic Data Analysis (Methodology in the Social Sciences)

Interpersonal phenomena such as attachment, conflict, person perception, helping, and influence have traditionally been studied by …

Author(s): David A. Kenny, Deborah A. Kashy, William L. Cook

Type of resources: Textbook

Primary user(s): Student, Teacher, Researcher

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Book

Link to resource

Easy preregistration will beneft any research

An article about easy preregistration will benefit any research.

Author(s): David T. Mellor and Brian Nosek

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s):

Link to resource

Economic Statistics with Calculus

A syllabus about economics and statistics

Author(s): Richard Ball

Type of resources: Syllabus

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Economics, statistics

Link to resource

Economics Lesson with Stata

A Data Carpentry curriculum for Economics is being developed by Dr. Miklos Koren at Central European University. These materials are …

Author(s): Andras Vereckei, Arieda Muço, Miklós Koren

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science, Measurement and Data, Economics

Tag(s): Analysis, Data, Education, Open Scholarship Tools and Technologies, Reproducibility, Research Data Management Tools, Researchers

Link to resource

Editorial Bias Against Replication Research

why aren’t more replications published? While there are many possible reasons, one simple one could be that journals prefer not …

Author(s): Neuliep, James W. and Rick Crandall

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Educational Psychologist - Educational Psychology in the Open Science Era

Special Issue of Educational Psychologist - Educational Psychology in the Open Science Era

Recently, scholars have noted how …

Author(s): OSKB Admin

Type of resources: Reading

Primary user(s):

Subject area(s): education

Tag(s): Research

Link to resource

Effect of open peer review on quality of reviews and on reviewers’recommendations: a randomised trial

Objectives: To examine the effect on peer review of asking reviewers to have their identity revealed to the authors of the paper. …

Author(s): Susan van Rooyen et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Transparency, Open Science

Link to resource

Effect of Population Heterogenization on the Reproducibility of Mouse Behavior: A Multi-Laboratory Study

In animal experiments, animals, husbandry and test procedures are traditionally standardized to maximize test sensitivity and minimize …

Author(s): Benjamin Zipser, Berry Spruijt, Britta Schindler, Chadi Touma, Christiane Brandwein, David P. Wolfer, Hanno Würbel, Johanneke van der Harst, Joseph P. Garner, Lars Lewejohann, Niek van Stipdonk, Norbert Sachser, Peter Gass, Sabine Chourbaji, S. Helene Richter, Vootele Võikar

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Health, Medicine and Nursing

Tag(s): Animal Behavior, Experimental Design, Factorial Design, Field Tests, Inbred Strains, Mice, Reproducibility, Research Methods, Study Design, White Light

Link to resource

Effect Size and Power in Assessing Moderating Effects of Categorical Variables Using Multiple Regression: A 30-Year Review

The authors conducted a 30-year review (1969-1998) of the size of moderating effects of categorical variables as assessed using …

Author(s): Herman Aguinis, James C Beaty, Robert J Boik and Charles A Pierce

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Effect Size Estimates: Current Use, Calculations, and Interpretation

The Publication Manual of the American Psychological Association (American Psychological Association, 2001, 2010) calls for the …

Author(s): Catherine O. Fritz et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Effect Size Estimation in Neuroimaging.

A central goal of translational neuroimaging is to establish robust links between brain measures and clinical outcomes. Success hinges …

Author(s): Marianne C. Reddan, Martin A. Lindquist and Tor D. Wager

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Effect size guidelines for individual differences researchers

Individual differences researchers very commonly report Pearson correlations between their variables of interest. Cohen (1988) provided …

Author(s): Gilles E.Gignac and Eva T.Szodorai

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Effect sizes and p values: What should be reported and what should be replicated?

Despite publication of many well-argued critiques of null hypothesis testing (NHT), behavioral science researchers continue to rely …

Author(s): ANTHONY G. GREENWALD, RICHARD GONZALEZ, RICHARD J. HARRIS and DONALD GUTHRIE

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics

Tag(s):

Link to resource

Effect Sizes: Why, When, and How to Use Them

The effect size (ES) is the magnitude of a study outcome or research finding, such as the strength of the relationship obtained between …

Author(s): Rosnow, R. L., & Rosenthal, R.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Eight common but false objections to the discontinuation of significance testing in the analysis of research data.

Logically and conceptually, the use of statistical significance testing in the analysis of research data has been thoroughly …

Author(s): Schmidt, J. E. H. F. L., Hunter, J. E., Harlow, L., Mulaik, S., & Steiger, J.

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

El Control de Versiones con Git

Software Carpentry lección para control de versiones con Git Para ilustrar el poder de Git y GitHub, usaremos la siguiente historia …

Author(s): Alejandra Gonzalez-Beltran, Amy Olex, Belinda Weaver, Bradford Condon, butterflyskip, Casey Youngflesh, Daisie Huang, Dani Ledezma, dounia, Francisco Palm, Garrett Bachant, Heather Nunn, Hely Salgado, Ian Lee, Ivan Gonzalez, James E McClure, Javier Forment, Jimmy O'Donnell, Jonah Duckles, Katherine Koziar, Katrin Leinweber, K.E. Koziar, Kevin Alquicira, Kevin MF, Kurt Glaesemann, LauCIFASIS, Leticia Vega, Lex Nederbragt, Mark Woodbridge, Matias Andina, Matt Critchlow, Mingsheng Zhang, Nelly Sélem, Nima Hejazi, Nohemi Huanca Nunez, Olemis Lang, Paula Andrea Martinez, Peace Ossom Williamson, P. L. Lim, Rayna M Harris, Romualdo Zayas-Lagunas, Sarah Stevens, Saskia Hiltemann, Shirley Alquicira, Silvana Pereyra, Tom Morrell, Valentina Bonetti, Veronica Ikeshoji-Orlati, Veronica Jimenez

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science, Measurement and Data

Tag(s): Analysis, Data, Education, Git, Open Scholarship Tools and Technologies, Reproducibility, Research Data Management Tools, Researchers

Link to resource

Eleven Strategies for Making Reproducible Research and Open Science Training the Norm at Research Institutions

Across disciplines, researchers increasingly recognize that open science and reproducible research practices may accelerate scientific …

Author(s): Friederike E. Kohrs, Susann Auer, Alexandra Bannach-Brown, Susann Fiedler, Tamarinde Haven, Verena Heise, Constance Holman, Flavio Azevedo, René Bernard, Arnim Bleier, Nicole Bössel, Brian Cahill, Leyla Jael Castro, Adrian Ehrenhofer, Kristina Eichel, Maximilian Frank, Claudia Frick, Malte Friese, Anne Gärtner, Kerstin Gierend, David Joachim Grüning, Lena Hahn, Maren Hülsemann, Malika Ihle, Sabrina Illius, Laura König, Matthias König, Louisa Kulke, Anton Kutlin, Fritjof Lammers, David M.A. Mehler, Christoph Miehl, Anett Müller-Alcazar, Claudia Neuendorf, Helen Niemeyer, Florian Pargent, Aaron Peikert, Christina U. Pfeuffer, Robert Reinecke, Jan Philipp Röer, Jessica L. Rohmann, Alfredo Sánchez-Tójar, Stefan Scherbaum, Elena Sixtus, Lisa Spitzer, Vera Maren Straßburger, Marcel Weber, Clarissa Whitmire, Josephine Zerna, Dilara Zorbek, Philipp Zumstein, Tracey L. Weissgerber

Type of resources: Teaching/Learning Strategy

Primary user(s): Teacher, Librarian

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Curriculum Design, Higher Education, Hiring, Open Science, Reproducible Research, Research Institutions, Scientific Rigor, Teaching, Transparency

Link to resource

Elsevier Title Level Pricing: Dissecting the Bowl of Spaghetti

INTRODUCTION This study will explore the issue of pricing opacity associated with prices paid by academic libraries that have recently …

Author(s): Joel B. Thornton and Curtis Brundy

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Business and Communication, Career and Technical Education, Social Science

Tag(s): Journal; Subscription

Link to resource

Embedding open and reproducible science into teaching: A bank of lesson plans and resources

Recently, there has been a growing emphasis on embedding open and reproducible approaches into research. One essential step in …

Author(s): Alaa Aldoh, Catherine V. Talbot, Charlotte Rebecca Pennington, David Moreau, Flavio Azevedo, John J Shaw, Loukia Tzavella, Madeleine Pownall, Mahmoud Elsherif, Martin Rachev Vasilev, Matthew C. Makel, Meng Liu, Myrthe Vel Tromp, Natasha April Tonge, Olly Robertson, Ronan McGarrigle, Ruth Horry, Sam Parsons

Type of resources: Reading

Primary user(s):

Subject area(s): Education

Tag(s): Learning, Lesson Plans, Pedagogical Resources, Teaching

Link to resource

Emerging Scientific Research Practices

This course aims to introduce students to current controversies and new developments in recommended scientific practices. The course is …

Author(s): Rodica Damian

Type of resources: Syllabus

Primary user(s): Teacher

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Empirical assessment of published effect sizes and power in the recent cognitive neuroscience and psychology literature

We have empirically assessed the distribution of published effect sizes and estimated power by analyzing 26,841 statistical records …

Author(s): Denes Szucs, John P. A. Ioannidis

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Psychology

Tag(s): Analysis, Behavioral Neuroscience, Bibliometrics, Cognitive Neuroscience, Data, Neuroscience, Psychology, Publishing, Scientific Publishing, Statistical Data, Statistical Distributions, Statistics

Link to resource

Empirical Evaluation of Very Large Treatment Effects of Medical Interventions

Context Most medical interventions have modest effects, but occasionally some clinical trials may find very large effects for benefits …

Author(s): Tiago V. Pereira, PhD Ralph I. Horwitz, MD John P. A. Ioannidis, MD, DSc

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Empirical Study of Data Sharing by Authors Publishing in PLoS Journals

Background Many journals now require authors share their data with other investigators, either by depositing the data in a public …

Author(s): Andrew J. Vickers, Caroline J. Savage

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Health, Medicine and Nursing

Tag(s): Clinical Trials, Data, Data Acquisition, Genomic Libraries, Genomic Medicine, Genomics, Health Care Policy, Medicine and Health Sciences, Open Access, Open Access Publishing, Publishing, Scientific Publishing

Link to resource

Enabling Open-Science Initiatives in Clinical Psychology and Psychiatry Without Sacrificing Patients’ Privacy: Current Practices and Future Challenges

The psychological and psychiatric communities are generating data on an ever-increasing scale. To ensure that society reaps the …

Author(s): Walsh, C., Xia, W., Li, M., Denny, J., Harris, P., & Malin, B

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Enhancing Reproducibility through Rigor and Transparency

The information provided on this website is designed to assist the extramural community in addressing rigor and transparency in NIH …

Author(s): NIH

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Health, Medicine and Nursing

Tag(s): Reproducibility

Link to resource

Enhancing transparency of the research process to increase accuracy of findings: A guide for relationship researchers

The purpose of this paper is to extend to the field of relationship science, recent discussions and suggested changes in open research …

Author(s): LORNE CAMPBELL TIMOTHY J. LOVING ETIENNE P. LEBEL

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

ePlatypus: an ecosystem for computational analysis of immunogenomics data

Motivation The maturation of systems immunology methodologies requires novel and transparent computational frameworks capable of …

Author(s): Tudor-Stefan Cotet, Andreas Agrafiotis, Victor Kreiner, Raphael Kuhn, Danielle Shlesinger, Marcos Manero-Carranza, Keywan Khodaverdi, Evgenios Kladis, Aurora Desideri Perea, Dylan Maassen-Veeters, Wiona Glänzer, Solène Massery, Lorenzo Guerci, Kai-Lin Hong, Jiami Han, Kostas Stiklioraitis, Vittoria Martinolli D’Arcy, Raphael Dizerens, Samuel Kilchenmann, Lucas Stalder, Leon Nissen, Basil Vogelsanger, Stine Anzböck, Daria Laslo, Sophie Bakker, Melinda Kondorosy, Marco Venerito, Alejandro Sanz García, Isabelle Feller, Annette Oxenius, Sai T Reddy, Alexander Yermanos

Type of resources: Reading

Primary user(s): Student, Teacher, Researcher

Subject area(s): Life Science

Tag(s): Systems Immunology, Bioinformatics, Computational Immunology, Ligand-receptor Interactions, Structural Modeling, Simulations, Machine Learning, Graph Theory, Pseudotime, Spatial Transcriptomics, Phylogenetics

Link to resource

Equity, transparency, and accountability: Open science for the 21st century

Knowledge is essential to saving lives and improving wellbeing. The term open science has been applied to improving the transparency of …

Author(s): Margaret A Winker, Theodora Bloom, Sandersan Onie, James Tumwine

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Equity, Transparency, Accountability, Open Science, Misconduct, Accuracy

Link to resource

Equivalence Testing for Psychological Research: A Tutorial

Psychologists must be able to test both for the presence of an effect and for the absence of an effect. In addition to testing against …

Author(s): Anne Scheel, Daniel Lakens, Peder Isager

Type of resources: Reading

Primary user(s):

Subject area(s): Social Science, Psychology

Tag(s): Aging, Equivalence Testing, Falsification, Frequentist, Null-hypothesis, Null-hypothesis Significance Test, Open Materials, Open Science, Power

Link to resource

Equivalence Tests: A Practical Primer for t Tests, Correlations, and Meta-Analyses

Scientists should be able to provide support for the absence of a meaningful effect. Currently, researchers often incorrectly conclude …

Author(s): Daniël Lakens

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Psychology

Tag(s): Analysis, Reproducibility, Statistics

Link to resource

Erroneous analyses of interactions in neuroscience: a problem of significance.

In theory, a comparison of two experimental effects requires a statistical test on their difference. In practice, this comparison is …

Author(s): Nieuwenhuis, S., Forstmann, B. U., & Wagenmakers, E. J.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Estimating the prevalence of transparency and reproducibility-related research practices in psychology (2014-2017)

Psychological science is navigating an unprecedented period of introspection about the credibility and utility of its research. A …

Author(s): Jessica Elizabeth Kosie, john Ioannidis, Joshua D Wallach, Mallory Kidwell, Robert T. Thibault, Tom Elis Hardwicke

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Psychology

Tag(s): Data

Link to resource

Estimating the reproducibility of psychological science

Reproducibility is a defining feature of science, but the extent to which it characterizes current research is unknown. We conducted …

Author(s): Open Science Collaboration.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Evaluating Content-Related Validity Evidence Using a Text-Based Machine Learning Procedure

Validity evidence based on test content is critical to meaningful interpretation of test scores. Within high-stakes testing and …

Author(s): Daniel Anderson, Brock Rowley, Sondra Stegenga, P. Shawn Irvin, Joshua M. Rosenberg

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Machine Learning, Text-mining, Textual Congruence, Validity

Link to resource

Evaluating Registered Reports: A Naturalistic Comparative Study of Article Impact

Registered Reports (RRs) is a publishing model in which initial peer review is conducted prior to knowing the outcomes of the research. …

Author(s): Brian A. Nosek, Felix Singleton Thorn, Lilian T. Hummer, Timothy M. Errington

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Life Science, Social Science

Tag(s): Preregistration, Publishing, Registered Reports

Link to resource

Evaluating the R-Index and the P-Curve

This blog evaluates the R-Index and the P-Curve

Author(s): Jeff Hughes

Type of resources: Reading

Primary user(s): Student, Researcher

Subject area(s): Math & Statistics

Tag(s): Blog

Link to resource

Evaluation of Transparency and Openness Guidelines in Physical Therapy Journals

Objective The goals of this study were to evaluate the extent that physical therapy journals support open science research practices by …

Author(s): Jacqueline Plante, Leigh Langerwerf, Mareli Klopper, Daniel I Rhon, Jodi L Young

Type of resources: Reading

Primary user(s): Teacher, Administrator, Librarian

Subject area(s): Life Science

Tag(s): Openness, Reproducibility of Results, Research, Science, Transparency

Link to resource

Everything Hertz

A podcast about open science and psychology

Author(s): Sound Cloud

Type of resources: Student Guide, Teaching/Learning Strategy, Podcast

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Podcast, Reproducibility Crisis and Credibility Revolution

Link to resource

Evidence of insufficient quality of reporting in patent landscapes in the life sciences

Despite the importance of patent landscape analyses in the commercialization process for life science and healthcare technologies, the …

Author(s): Andrew J. Carr, David A. Brindley, Hannah Thomas, James A. Smith, Zeeshaan Arshad

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Engineering, Biology

Tag(s): Licensing, Reproducibility

Link to resource

Evolution of Reporting P Values in the Biomedical Literature, 1990-2015

Importance: The use and misuse of P values has generated extensive debates. Objective: To evaluate in large scale the P values reported …

Author(s): David Chavalarias, Joshua David Wallach, Alvin Ho Ting Li, John P A Ioannidis

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Excess Success for Psychology Articles in the Journal Science

This article describes a systematic analysis of the relationship between empirical data and theoretical conclusions for a set of …

Author(s): Gregory Francis, Jay Tanzman, William J. Matthews

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Expectations for Replications: Are Yours Realistic?

Failures to replicate published psychological research findings have contributed to a “crisis of confidence.” Several …

Author(s): David J Stanley 1, Jeffrey R Spence

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Experimentation and manipulation with preregistration

Preregistration requires scientists to describe the planned research activities before their project begins. Preregistration improves …

Author(s): Mike Felgenhauer

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Preregistration, Experimentation, Manipulation

Link to resource

Experimenter as automaton; experimenter as human: Exploring the position of the researcher in scientific research

The crisis of confidence in the social sciences has many corollaries which impact our research practices. One of these is a push …

Author(s): Sarahanne M. Field, Maarten Derksen

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Reflexivity, Subjectivity, Reproducibility

Link to resource

Experiments with More Than One Random Factor: Designs, Analytic Models, and Statistical Power.

Traditional methods of analyzing data from psychological experiments are based on the assumption that there is a single random factor …

Author(s): Judd, C. M., Westfall, J., & Kenny, D. A.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Exploratory Factor Analysis

This book provides a non-mathematical introduction to the underlying theory of Efa and reviews the key decisions that must be made in …

Author(s): Leandre R. Fabrigar, Dueane T. Wegener

Type of resources: Textbook

Primary user(s): Student, Teacher, Researcher

Subject area(s): Applied Science, Life Science, Math & Statistics, Social Science

Tag(s): Book

Link to resource

Exploratory hypothesis tests can be more compelling than confirmatory hypothesis tests

Preregistration has been proposed as a useful method for making a publicly verifiable distinction between confirmatory hypothesis …

Author(s): Mark Rubin, Chris Donkin

Type of resources: Reading

Primary user(s): Student, Parent, Librarian

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Accommodation, Exploratory Analyses, Confirmatory Analyses, Prediction, Preregistration, Hypothesis Testing

Link to resource

Exploring Pre-registration and Pre-analysis Plans for Qualitative Inference

In recent years, the discipline of political science has experienced demands and moves toward greater research transparency. While in …

Author(s): Florian G. Kern, Kristian Skrede Gleditsch

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Physical Science, Social Science

Tag(s): Transparency, Replication, Qualitative Inference, Pre-Analysis Plans, Preregistration

Link to resource

Facilitating open science practices for research syntheses: PreregRS guides preregistration

Researchers increasingly engage in adopting open science practices in the field of research syntheses, such as preregistration. …

Author(s): Jürgen Schneider, Iris Backfisch, Andreas Lachner

Type of resources: Student Guide

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Open Science, Preregistration, Rmarkdown, R Package

Link to resource

Facts Are More Important Than Novelty: Replication in the Education Sciences

Despite increased attention to methodological rigor in education research, the field has focused heavily on experimental design and not …

Author(s): Matthew C. Makel and Jonathan A. Plucker

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Faking Science: A True Story of Academic Fraud

A book about Academic Fraud

Author(s): Diederik Stapel/Translated by Nicholas J. L. Brown

Type of resources: Primary Source

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Textbook, Reproducibility Crisis and Credibility Revolution

Link to resource

False Positive Psychology

A lecture on when analysis goes wrong A look at false-positive psychology

Author(s): Dermot Lynott

Type of resources: Lecture, Lecture Notes, Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Life Science, Math & Statistics, Social Science

Tag(s): Lecture, Reproducibility Knowledge

Link to resource

False-Positive Citations

We describe why we wrote “False-Positive Psychology,” analyze how it has been cited, and explain why the integrity of experimental …

Author(s): Joseph P. Simmons, Leif D. Nelson, and Uri Simonsohn

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant

In this article, we accomplish two things. First, we show that despite empirical psychologists’ nominal endorsement of a low rate of …

Author(s): Joseph P. Simmons, Leif D. Nelson, Uri Simonsohn

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Falsely Reassuring: Analyses of ALL p-values

A blog post that describes analyses of all p-values

Author(s): Uri Simonsohn

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Blog

Link to resource

Falsifiability Is Not Optional

Finkel, Eastwick, and Reis (2016; FER2016) argued the post-2011 methodological reform movement has focused narrowly on replicability, …

Author(s): LeBel, E. P., Berger, D., Campbell, L., & Loving, T. J. (2017).

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Fearing the future of empirical psychology: Bem’s (2011) evidence of psi as a case study in deficiencies in modal research practice.

In this methodological commentary, we use Bem’s (2011) recent article reporting experimental evidence for psi as a case study for …

Author(s): LeBel, E.P., & Peters, K.R.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Feasibility of Emulating Clinical Trials Supporting US FDA Supplemental Indication Approvals of Drugs and Biologics

The US Food and Drug Administration (FDA) has developed a framework to use data gathered outside of clinical trials (eg, electronic …

Author(s): Guneet S. Janda, Joshua D. Wallach, Meera M. Dhodapkar, Reshma Ramachandran, Joseph S. Ross

Type of resources: Reading

Primary user(s): Teacher, Librarian, researcher

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Clinical Trials, Data, Drugs, Biologics

Link to resource

Feeling the Future: Experimental Evidence for Anomalous Retroactive Influences on Cognition and Affect

The term psi denotes anomalous processes of information or energy transfer that are currently unexplained in terms of known physical or …

Author(s): Daryl J Bem

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Fifty years of research on questionable research practises in science: Quantitative analysis of co-citation patterns

Questionable research practises (QRPs) have been the focus of the scientific community amid greater scrutiny and evidence highlighting …

Author(s): Michelle Jin Yee Neoh, Alessandro Carollo, Albert Lee, Gianluca Esposito

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Scientific Integrity, Ethics of Research, Questionable Research Practices

Link to resource

Finding & Evaluating Open Data

Introduction to finding and evaluating Open Data by NYU DataServices.

Author(s): Vicky Steeves

Type of resources: Activity/Lab

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Data, Licenses, Open Data, Researchers

Link to resource

Fishing, Commitment, and Communication: A Proposal for Comprehensive Nonbinding Research Registration

Social scientists generally enjoy substantial latitude in selecting measures and models for hypothesis testing. Coupled with …

Author(s): Macartan Humphreys, Raul Sanchez de la Sierra and Peter van der Windt

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Five selfish reasons to work reproducibly

And so, my fellow scientists: ask not what you can do for reproducibility; ask what reproducibility can do for you! Here, I present …

Author(s): Florian Markowetz

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Organizational Change, Reproducibility, Researchers

Link to resource

Foster Open Science

The FOSTER portal is an e-learning platform that brings together the best training resources addressed to those who need to know more …

Author(s): FOSTER Open Science

Type of resources: Full Course

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Analysis, Data, Education, Materials, Policy, Publishing

Link to resource

Four simple recommendations to encourage best practices in research software

Scientific research relies on computer software, yet software is not always developed following practices that ensure its quality and …

Author(s): Alejandra Gonzalez-Beltran, Allegra Via, Andrew Treloar, Bérénice Batut, Bernard Pope, Björn GrüningJonas Hagberg, Brane Leskošek, Carole Goble, Daniel S. Katz, Daniel Vaughan, David Mellor, Federico López Gómez, Ferran Sanz, Harry-Anton Talvik, Horst Pichler, Ilian Todorov, Jon Ison, Josep Ll. Gelpí, Leyla Garcia, Luis J. Oliveira, Maarten van Gompel, Madison Flannery, Manuel Corpas, Maria V. Schneider, Martin Cook, Mateusz Kuzak, Michelle Barker, Mikael Borg, Monther Alhamdoosh, Montserrat González Ferreiro, Nathan S. Watson-Haigh, Neil Chue Hong, Nicola Mulder, Petr Holub, Philippa C. Griffin, Radka Svobodová Vařeková, Radosław Suchecki, Rafael C. Jiménez, Robert Pergl, Rob Hooft, Rowland Mosbergen, Salvador Capella-Gutierrez, Simon Gladman, Sonika Tyagi, Steve Crouchc, Victoria Stodden, Xiaochuan Wang, Yasset Perez-Riverol

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science

Tag(s): Reproducibility

Link to resource

Free and low cost resources for graduate students, postdocs, and early career researchers (or really anyone else)

A list of free or cheap resources (e.g. free and open statistical software) that support the processes of science

Author(s): Dr Jaclyn Siegel

Type of resources: Reading, Resource list

Primary user(s): Student, Teacher, Librarian, Early career researchers

Subject area(s): Applied Science, Social Science

Tag(s): Open science

Link to resource

From policy to practice: Lessons learned from an open science funding initiative

In the past few years, there has been a notable shift in the open science landscape as more countries and international agencies …

Author(s): Sonya B. Dumanis, Kristen Ratan, Souad McIntosh, Hetal V. Shah, Matt Lewis, Timothy H. Vines, Randy Schekman, Ekemini A. Riley

Type of resources: Reading

Primary user(s): Student, Teacher, Administrator, Librarian

Subject area(s): Life Science

Tag(s): Computer Software, Open Science, Science Policy, Research Assessment, Software Tools, Parkinson Disease, Programming Languages, Reproducibility

Link to resource

From pre-registration to publication: A non-technical primer for conducting a meta-analysis to synthesize correlational data

Meta-analysis synthesizes a body of research investigating a common research question. Outcomes from meta-analyses provide a more …

Author(s): Daniel S. Quintana

Type of resources: Student Guide

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Meta-Analysis, Primer, Methods, Preregistration, Statistics, Publication Bias

Link to resource

From private incentives to public health need: rethinking research and development for pandemic preparedness

Pandemic preparedness and response have relied primarily on market dynamics to drive development and availability of new health …

Author(s): Els Torreele, Daniel Wolfe, Michel Kazatchkine, Amadou Sall, Kiat Ruxrungtham, Joseph Robert Anderson Fitchett, Joanne Liu, Gary Kobinger, Claudia Vaca-González, Carolina Gómez, Petro Terblanche, Soumya Swaminathan, Piero Olliaro, Helen Clark

Type of resources: Reading

Primary user(s): Teacher, Administrator, Librarian

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Research and Development, Pandemic Preparedness

Link to resource

Funder Data-Sharing Policies: Overview and Recommendations

This report covers funder data-sharing policies/practices, and provides recommendations to funders and others as they consider their …

Author(s): Stephanie Wykstra

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Health, Medicine and Nursing, Life Science, Social Science

Tag(s): Data, Data-Sharing Policies, Funder Policies, Open Data, Open Science, Policy

Link to resource

G*Power

GPower is a tool to compute statistical power analyses for many different t tests, F tests, χ2 tests, z tests and some exact tests. G …

Author(s): The G*Power Team

Type of resources: Reading, Simulation

Primary user(s): Student, Teacher, Researcher

Subject area(s): Math & Statistics

Tag(s): Reproducibility Knowledge, Power Analysis Tool

Link to resource

General Principles of Preclinical Study Design

Preclinical studies using animals to study the potential of a therapeutic drug or strategy are important steps before translation to …

Author(s): Andrew S. C. Rice, Jan Vollert, Nathalie Percie du Sert, Wenlong Huang

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Health, Medicine and Nursing

Tag(s): Experimental Bias, Hypothesis Generating, Hypothesis Testing, In Vivo Studies, Preclinical Research, Reproducibility

Link to resource

Genomics Workshop Overview

Workshop overview for the Data Carpentry genomics curriculum. Data Carpentry’s aim is to teach researchers basic concepts, skills, and …

Author(s): Amanda Charbonneau, Erin Alison Becker, François Michonneau, Jason Williams, Maneesha Sane, Matthew Kweskin, Muhammad Zohaib Anwar, Murray Cadzow, Paula Andrea Martinez, Taylor Reiter, Tracy Teal

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science, Genetics, Measurement and Data

Tag(s): Analysis, Data, Education, Genomics, Open Scholarship Tools and Technologies, Reproducibility, Research Data Management Tools, Researchers

Link to resource

Geospatial Workshop Overview

Data Carpentry’s aim is to teach researchers basic concepts, skills, and tools for working with data so that they can get more done in …

Author(s): Anne Fouilloux, Arthur Endsley, Chris Prener, Jeff Hollister, Joseph Stachelek, Leah Wasser, Michael Sumner, Michele Tobias, Stace Maples

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Information Science, Measurement and Data, Physical Science, Geology, Physical Geography, Social Science

Tag(s): Analysis, Data, Education, Geospatial, Open Scholarship Tools and Technologies, Reproducibility, Research Data Management Tools, Researchers

Link to resource

Getting Involved with TOP Factor

This webinar provides an overview of TOP Factor: its rationale, how it is being used, and how each of the TOP standards relate to …

Author(s):

Type of resources: Lecture

Primary user(s):

Subject area(s): Education

Tag(s): Center for Open Science, Data Citation, Journal Impact Factor, Open Science, Preregistration, Replication, Reproducibility, Research, Research Analysis, Research Best Practices, Research Design, Scientific Funding, Scientific Publishing, TOP Factor, TOP Guidelines

Link to resource

GGPlot Colors Best Tricks You Will Love

This article presents multiple great solutions you should know for changing ggplot colors.

Author(s): Alboukadel

Type of resources: Reading

Primary user(s): Student

Subject area(s): Math & Statistics, Social Science

Tag(s): Data visualisation

Link to resource

Giving Community Psychology Away: A case for open access publishing

Amidst increased pressure for transparency in science, researchers and community members are calling for open access to study stimuli …

Author(s): Crystal N. Steltenpohl, Amy J. Anderson, and Katherine M. Daniels

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Life Science, Social Science

Tag(s):

Link to resource

GLIMMPSE

Welcome to GLIMMPSE. The GLIMMPSE software calculates power and sample size for study designs with normally distributed outcomes. …

Author(s): Anon

Type of resources: Reading, Simulation

Primary user(s): Student, Teacher, Researcher

Subject area(s): Math & Statistics

Tag(s): Reproducibility Knowledge, Power Analysis Tool

Link to resource

Good clinical practice improves rigor and transparency: Lessons from the ACTIVE trial

Clinical trials are governed by principles of good clinical practice (GCP), which can strengthen the achievement of rigor, …

Author(s): Brad P. Taylor, George W. Rebok, Michael Marsiske

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Life Science, Social Science

Tag(s): Rigor, Transparency, Clinical Trials

Link to resource

Good enough practices in scientific computing

Computers are now essential in all branches of science, but most researchers are never taught the equivalent of basic lab skills for …

Author(s): Greg Wilson, Jennifer Bryan, Justin Kitzes, Karen Cranston, Lex Nederbragt, Tracy K. Teal

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Biology

Tag(s): Computer Software, Control Systems, Data Management, Data Processing, Programming Languages, Reproducibility, Software Tools, Source Code

Link to resource

Good Science, Bad Science

A syllabi about open science: good science and bad science.

Author(s): Eric-Jan Wagenmakers

Type of resources: Syllabus

Primary user(s): Teacher

Subject area(s): Applied Science, Education, Life Science, Math & Statistics, Social Science

Tag(s): Syllabus

Link to resource

Graduate Research Methods

Completion of this course will provide a foundation for the practice of science. We will wrestle with the fundamental issues for …

Author(s): Brian Nosek

Type of resources: Syllabus

Primary user(s): Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Transparency, Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Graphical Causal Models

This chapter discusses the use of directed acyclic graphs (DAGs) for causal inference in the observational social sciences. It focuses …

Author(s): Felix Elwert

Type of resources: Primary Source, Reading, Chapter

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Guide Your Students to Become Better Research Consumers

Blog post going over making undergraduate students better consumers of research

Author(s): Beth Morling

Type of resources: Reading, Teaching/Learning Strategy

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Education, Life Science, Social Science

Tag(s): Blog, Open Science

Link to resource

Guidelines for Evaluating the Comparability of Down-Sampled GWAS Summary Statistics

Proprietary genetic datasets are valuable for boosting the statistical power of genome-wide association studies (GWASs), but their use …

Author(s): Camille M. Williams, Holly Poore, Peter T. Tanksley, Hyeokmoon Kweon, Natasia S. Courchesne-Krak, Diego Londono-Correa, Travis T. Mallard, Peter Barr, Philipp D. Koellinger, Irwin D. Waldman, Sandra Sanchez-Roige, K. Paige Harden, Abraham A. Palmer, Danielle M. Dick & Richard Karlsson Linnér

Type of resources: Reading

Primary user(s): Student, Teacher, researcher

Subject area(s): Life Science

Tag(s): Genomic SEM, Summary Statistics, Data Removal, Down-sample, Leave-one-out, Meta-analysis, Genomics, Genome-wide Association Study

Link to resource

Habits and perceptions regarding open science by researchers from Spanish institutions

The article describes the results of the online survey on open science (OS) carried out on researchers affiliated with universities and …

Author(s): Candela Ollé, Alexandre López-Borrull, Remedios Melero, Juan-José Boté-Vericad, Josep-Manuel Rodríguez-Gairín, Ernest Abadal

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Open Science, Science Policy, Open Access Publishing, Research Funding, Data Management, Open Peer Review, Open Data

Link to resource

Hack Your Way To Scientific Glory

You’re a social scientist with a hunch: The U.S. economy is affected by whether Republicans or Democrats are in office. Try to show …

Author(s): Anon

Type of resources: Activity/Lab, Interactive, Simulation, Teaching/Learning Strategy

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Demo, Reproducibility Knowledge

Link to resource

Hackathon: Encouraging Open Science Practices in Qualitative Education Research

This list of resources consists of resources for researchers, editors, and reviewers interested in practicing open science principles, …

Author(s): Crystal Steltenpohl, Rachel Renbarger, Sebastian Karcher, Sondra Stegenga, Thomas Lösch

Type of resources: Reading

Primary user(s):

Subject area(s): Education

Tag(s): Open Science, Qualitative Research, Research

Link to resource

Hail the impossible: p-values, evidence, and likelihood.

Significance testing based on p-values is standard in psychological research and teaching. Typically, research articles and textbooks …

Author(s): Johansson, T.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

HARK No More: On the Preregistration of CHI Experiments

Experimental preregistration is required for publication in many scientific disciplines and venues. When experimental intentions are …

Author(s): Andy Cockburn, Carl Gutwin, Alan Dix

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Human-Centered Computing, Human Computer Interaction, Preregistration

Link to resource

HARKing: How Badly Can Cherry-Picking and Question Trolling Produce Bias in Published Results?

The practice of hypothesizing after results are known (HARKing) has been identified as a potential threat to the credibility of …

Author(s): Kevin R. Murphy & Herman Aguinis

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

HARKing: Hypothesizing after the results are known

This article considers a practice in scientific communication termed HARKing (Hypothesizing After the Results are Known). HARKing is …

Author(s): Kerr, N. L.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Harry Potter and the Methods of Reproducibility

“Harry Potter and the Methods of Reproducibility – A brief Introduction to Open Science” gives a brief overview of …

Author(s): Mariella Paul

Type of resources: Lesson

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Reproducibility, Researchers, Students

Link to resource

History and Methods of Psychology Syllabus

This course is designed to expose students to the history of psychology through a study of the methods used in research over time, with …

Author(s): Moin Syed

Type of resources: Syllabus

Primary user(s): Teacher

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

History and Philosophy of Psychology

In this course, we will examine the on-going methodological controversies around psychology, cognitive science, and cognitive …

Author(s): Professor E.Machery

Type of resources: Syllabus

Primary user(s): Teacher

Subject area(s): Social Science

Tag(s): Philosophy, History of Psychology

Link to resource

History repeating: Guidelines to address common problems in psychedelic science

Research in the last decade has expressed considerable optimism about the clinical potential of psychedelics for the treatment of …

Author(s): Michiel van Elk, Eiko I. Fried

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Open Science, Psychedelics, Psychotherapy, Questionable Research Practices, Validity

Link to resource

How (and Whether) to Teach Undergraduates About the Replication Crisis in Psychological Science

Over the past 10 years, crises surrounding replication, fraud, and best practices in research methods have dominated discussions in the …

Author(s): William J. Chopik, Ryan H. Bremner, Andrew M. Defever, Victor N. Keller

Type of resources: Primary Source, Reading, Paper

Primary user(s): Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution

Link to resource

How can preregistration contribute to research in our field?

Comprehensive Results in Social Psychology (CRSP) is a novel journal for preregistered research (so-called registered reports, RR) in …

Author(s): Kai J. Jonas, Joseph Cesario

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Social Science

Tag(s): Preregistration, Registered Reports, Social Psychology

Link to resource

How racist policing took over American cities, explained by a historian

“The problem is the way policing was built,” historian Khalil Muhammad says.

Author(s): Anna North

Type of resources: Reading

Primary user(s): Student, Teacher, Administrator, Parent, Librarian

Subject area(s): Social Science

Tag(s): Diversity, Equity, Inclusion

Link to resource

How scientists can stop fooling themselves

Sampling simulated data can reveal common ways in which our cognitive biases mislead us.

Author(s): Professor Dorothy Bishop

Type of resources: Reading

Primary user(s): Student, Teacher, Researcher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

How significant are the public dimensions of faculty work in review, promotion and tenure documents?

Much of the work done by faculty at both public and private universities has significant public dimensions: it is often paid for by …

Author(s): Carol Muñoz Nieves, Erin C McKiernan, Gustavo E Fischman, Juan P Alperin, Lesley A Schimanski, Meredith T Niles

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Information Science

Tag(s): Academic Careers, Higher Education, Institutional Policy, Metrics, Open Access, Publishing, Scholarly Communications

Link to resource

How to classify, detect, and manage univariate and multivariate outliers, with emphasis on pre-registration

Researchers often lack knowledge about how to deal with outliers when analyzing their data. Even more frequently, researchers do not …

Author(s): Christophe Leys, Marie Delacre, Youri L. Mora, Daniel Lakens, Christophe Ley

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Psychology, Transparency, Outliers, Preregistration, Robust Detection, Malahanobis Distance, Median Absolute Deviation, Minimum Covariance Determinant

Link to resource

How to Crack Pre-registration: Toward Transparent and Open Science

The reproducibility problem that exists in various academic fields has been discussed in recent years, and it has been revealed that …

Author(s): Yuki Yamada

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Preregistration

Link to resource

How to make data open? Stop overlooking librarians

Digital archivists are already experts at tackling the complex challenges of making research data open and accessible. We can help to …

Author(s): Jessica Farrell

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Research Data, Research Management, Funding, Policy, Librarians, Digital Archiving, Open Data

Link to resource

How to Make More Published Research True

An essay about How to Make More Published Research True

Author(s): John Ioannidis

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

How to Use OSF as an Electronic Lab Notebook

This webinar outlines how to use the free Open Science Framework (OSF) as an Electronic Lab Notebook for personal work or private …

Author(s): Center for Open Science

Type of resources: Lecture

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science

Tag(s): Analysis, Data, Education, Electronic Lab Notebooks, Materials, Open Scholarship Tools and Technologies, OSF, Publishing, Reproducibility, Research Data Management Tools, Researchers

Link to resource

I Fooled Millions Into Thinking Chocolate Helps Weight Loss. Here’s How.

A blog about reproducibility crisis

Author(s): John Bohannon

Type of resources: Primary Source, Reading, Blog

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Identifying Participants in the Personal Genome Project by Name

We linked names and contact information to publicly available profiles in the Personal Genome Project. These profiles contain medical …

Author(s): Sweeney, Latanya, Akua Abu, and Julia Winn.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

IIT vs. GNWT and the meaning of evidence in consciousness science

This post follows one I wrote earlier in the summer, after ASSC. Since then I have been trying (in a first-person therapy sort of way) …

Author(s): Steve Fleming

Type of resources: Reading

Primary user(s): Student, Teacher, researcher

Subject area(s): Life Science, Social Science

Tag(s): Consciousness, IIT, GNWT, Accelerating Research on Consciousness, Higher Order Theories

Link to resource

Image Processing with Python

This lesson shows how to use Python and skimage to do basic image processing. With support from an NSF iUSE grant, Dr. Tessa Durham …

Author(s): Mark Meysenberg

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science, Measurement and Data

Tag(s): Analysis, Data, Education, Image Processing, Open Scholarship Tools and Technologies, Python, Reproducibility, Research Data Management Tools, Researchers

Link to resource

Impact of genetic background and experimental reproducibility on identifying chemical compounds with robust longevity effects

Limiting the debilitating consequences of ageing is a major medical challenge of our time. Robust pharmacological interventions that …

Author(s): Anna B. Crist, Anna C. Foulger, Anna L. Coleman-Hulbert, Brian Onken, Carolina Ibanez-Ventoso, Christina Chang, Christine A. Sedore, Daniel Edgar, Dipa Bhaumik, Elizabeth A. Chao, Erik Johnson, Esteban Chen, Girish Harinath, Gordon J. Lithgow, Jailynn Harke, Jason L Kish, Jian Xue, John H. Willis, June Hope, Kathleen J. Dumas, Manish Chamoli, Mark Lucanic, Mary Anne Royal, Max Guo, Michael P. Presley, Michelle K. Chen, Monica Driscoll, Patrick C. Phillips, Shaunak Kamat, Shobhna Patel, Suzanne Angeli, Suzhen Guo, Theo Garrett, W. Todd Plummer

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Biology

Tag(s): Analysis, Genetics, Reproducibility, Research Methods

Link to resource

Improving (Our) Science

The goal of science is to accumulate knowledge about nature. There are scientific values guiding how scientists should work, and …

Author(s): Brian Nosek

Type of resources: Syllabus

Primary user(s): Teacher

Subject area(s): Social Science

Tag(s): Transparency, Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Improving (Our) Science: Reproducibility, Reporting, and Openness

The goal of the course is to become a better scientist. You will learn about newest standards for scientific openness, and how they …

Author(s): Don Moore and Leif Nelson

Type of resources: Syllabus

Primary user(s): Teacher

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution; Open Science; Conceptual and statistical knowledge

Link to resource

Improving evidence-based practice through preregistration of applied research: Barriers and recommendations

Preregistration is the practice of publicly publishing plans on central components of the research process before access to, or …

Author(s): Thomas Rhys Evans, Peter Branney, Andrew Clements, Ella Hatton

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Preregistration, Academia, Industry, Consulting, Applied Research, Evidence-based Practices

Link to resource

Improving the Dependability of Research in Personality and Social Psychology: Recommendations for Research and Educational Practice

In this article, the Society for Personality and Social Psychology (SPSP) Task Force on Publication and Research Practices offers a …

Author(s): Funder et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s):

Link to resource

Improving transparency in observational social science research: A pre-analysis plan approach

Social science research has undergone a credibility revolution, but these gains are at risk due to problematic research practices. …

Author(s): Fiona Burlig

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Social Science

Tag(s): Preregistration, Social Science

Link to resource

Improving your statistical inferences

This course aims to help you to draw better statistical inferences from empirical research. First, we will discuss how to correctly …

Author(s): Daniel Lakens

Type of resources: Full Course

Primary user(s): Student

Subject area(s): Applied Science, Life Science, Math & Statistics, Social Science

Tag(s): Course

Link to resource

In Praise of Moderation: Suggestions for the Scope and Use of Pre-Analysis Plans for RCTs in Economics

Pre-Analysis Plans (PAPs) for randomized evaluations are becoming increasingly common in Economics, but their definition remains …

Author(s): Abhijit Banerjee, Esther Duflo, Amy Finkelstein, Lawrence F. Katz, Benjamin A. Olken, Anja Sautmann

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Business and Communication, Social Science

Tag(s): Preregistration, Pre-Analysis Plans, Economics

Link to resource

In Praise of the Null Hypothesis Statistical Test

Jacob Cohen (see record 1995-12080-001) raised a number of questions about the logic and information value of the null hypothesis …

Author(s): Richard L. Hagen

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Inappropriate fiddling with statistical analyses to obtain a desirable p-value: tests to detect its presence in published literature.

Much has been written regarding p-values below certain thresholds (most notably 0.05) denoting statistical significance and the …

Author(s): Gary L. Gadbury and David B. Allison

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Incorporating ecological momentary assessment into multimethod investigations of cognitive aging: Promise and practical considerations

Ecological momentary assessment (EMA) represents a promising approach to study cognitive aging. In contrast to laboratory-based …

Author(s): Jennifer L. Crawford, Tammy English, Todd S. Braver

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Life Science, Social Science

Tag(s): Ecological Momentary Assessment, Methods, Cognitive Aging, Robustness, Replicability, Replication, Generalizability, Transferability

Link to resource

Increasing efficiency of preclinical research by group sequential designs

Despite the potential benefits of sequential designs, studies evaluating treatments or experimental manipulations in preclinical …

Author(s): Alice Schneider, Andre Rex, Bob Siegerink, George Karystianis, Ian Wellwood, John P. A. Ioannidis, Jonathan Kimmelman, Konrad Neumann, Oscar Florez-Vargas, Sophie K. Piper, Ulrich Dirnagl, Ulrike Grittner

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Biology

Tag(s): Bayesian Method, Bayesian Statistics, Clinical Trials, Experimental Design, Medicine and Health Sciences, Probability Distribution, Research Design, Research Errors

Link to resource

Increasing the transparency of systematic reviews: Presenting a generalized registration form

This paper presents a generalized registration form for systematic reviews that can be used when currently available forms are not …

Author(s): Olmo R. van den Akker, Gjalt-Jorn Ygram Peters, Caitlin J. Bakker, Rickard Carlsson, Nicholas A. Coles, Katherine S. Corker, Gilad Feldman, David Moreau, Thomas Nordström, Jade S. Pickering, Amy Riegelman, Marta K. Topor, Nieky van Veggel, Siu Kit Yeung, Mark Call, David T. Mellor, Nicole Pfeiffer

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian, researcher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Registration, Systematic Reviews, Psychology, Economics, Law, Physics, Meta-Analysis

Link to resource

Increasing transparency through a multiverse analysis.

Empirical research inevitably includes constructing a data set by processing raw data into a form ready for statistical analysis. Data …

Author(s): Steegen, S., Tuerlinckx, F., Gelman, A., & Vanpaemel, W.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Increasing value and reducing waste: addressing inaccessible research

The methods and results of health research are documented in study protocols, full study reports (detailing all analyses), journal …

Author(s): An-Wen Chan, Fujian Song, Andrew Vickers, Tom Jefferson, Kay Dickersin, Peter C Gøtzsche, Harlan M Krumholz, Davina Ghersi, H Bart van der Worp

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Protocols, Preregistration, Open Access, Open Data, Funding

Link to resource

Instead of “playing the game” it is time to change the rules: Registered Reports at AIMS Neuroscience and beyond.

The last ten years have witnessed increasing awareness of questionable research practices (QRPs) in the life sciences [1,2], including …

Author(s): Chambers, C. D., Feredoes, E., Muthukumaraswamy, S. D., & Etchells, P.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Intellectual humility is central to science

Transparency is indispensable for accuracy and correction in science, and is discussed frequently in the credibility revolution. A less …

Author(s): Rink Hoekstra and Simine Vazire

Type of resources: Reading

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Internal conceptual replications do not increase independent replication success

Recently, many psychological effects have been surprisingly difficult to reproduce. This article asks why, and investigates whether …

Author(s): Richard Kunert

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Interpreting confidence intervals

A blog about the visualisation of Confidence intervals

Author(s): Kristoffer Magnusson

Type of resources: Interactive, Simulation, Student Guide, Teaching/Learning Strategy, R code

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Blog, Interaction, Simulation, Tutorial

Link to resource

Interpreting Correlations: an interactive visualization

Correlation is one of the most widely used tools in statistics. The correlation coefficient summarizes the association between two …

Author(s): Kristoffer Magnusson

Type of resources: Interactive, Reading, Simulation, Student Guide, Teaching/Learning Strategy

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Blog, Interaction, Simulation, Tutorial

Link to resource

Interpreting effect sizes: Toward a quantitative cumulative social psychology

Improved research practice is based on estimation of effect sizes rather than statistical significance. We discuss the challenging task …

Author(s): ARTHUR A. STUKAS AND GEOFF CUMMING

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Intro to Calculating Confidence Intervals

This video will introduce how to calculate confidence intervals around effect sizes using the MBESS package in R. All materials shown …

Author(s): Center for Open Science

Type of resources: Lecture

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science

Tag(s): Analysis, Data, Education, Materials, Open Scholarship Tools and Technologies, OSF, Publishing, Reproducibility, Research Data Management Tools, Researchers, Statistics

Link to resource

Intro to R and RStudio for Genomics

Welcome to R! Working with a programming language (especially if it’s your first time) often feels intimidating, but the rewards …

Author(s): Ahmed Moustafa, Alexia Cardona, Andrea Ortiz, Jason Williams, Krzysztof Poterlowicz, Naupaka Zimmerman, Yuka Takemon

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science, Biology, Measurement and Data

Tag(s): Analysis, Data, Education, Inside Your Classroom, Open Scholarship Tools and Technologies, R, Reproducibility, Research Data Management Tools, Researchers

Link to resource

Intro to the special issue

A paper about a special issue on cognitive modelling

Author(s): Kevin Glucka, Paul Bellob, Jerome Busemeyer

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s):

Link to resource

Introducing JASP

A blog about JASP to replace SPSS

Author(s): Jonas Haslbeck

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Blog, Reproducibility Knowledge

Link to resource

Introducing Preregistration of Research Design to Archaeology

Archaeology has an issue with “just-in-time” research, where insufficient attention is paid to articulating a research …

Author(s): Shawn Ross, Brian Ballsun-Stanton

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Arts and Humanities

Tag(s): Arts and Humanities, Digital Humanities, History, Philosophy, Philosophy of Science, Archaeology, Preregistration

Link to resource

Introduction materials for Reproducible Research Curriculum

Workshop goals

  • Why are we teaching this
  • Why is this important
  • For future and current you
  • For research as a whole
  • Lack of …

Author(s): Kristina Riemer, Mine Çetinkaya-Rundel, Pat Schloss, Paul Magwene

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Information Science

Tag(s): Organizing, Reproducibility

Link to resource

Introduction to Cloud Computing for Genomics

Data Carpentry lesson to learn how to work with Amazon AWS cloud computing and how to transfer data between your local computer and …

Author(s): Abigail Cabunoc Mayes, Adina Howe, Amanda Charbonneau, ammatsun, Bérénice Batut, Bob Freeman, Brittany N. Lasseigne, PhD, Caryn Johansen, Chris Fields, Darya Vanichkina, David Mawdsley, Erin Becker, François Michonneau, Greg Wilson, Jason Williams, Joseph Stachelek, Kari L. Jordan, PhD, Katrin Leinweber, Maxim Belkin, Michael R. Crusoe, Piotr Banaszkiewicz, Raniere Silva, Rémi Emonet, Renato Alves, Stephen Turner, Taylor Reiter, Thomas Morrell, Tracy Teal, vuw-ecs-kevin, William L. Close

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science, Measurement and Data

Tag(s): Analysis, Cloud Computing, Data, Education, Open Scholarship Tools and Technologies, Reproducibility, Research Data Management Tools, Researchers

Link to resource

Introduction to Geospatial Concepts

Data Carpentry lesson to understand data structures and common storage and transfer formats for spatial data. The goal of this lesson …

Author(s): Anne Fouilloux, Chris Prener, Dev Paudel, Ethan P White, Joseph Stachelek, Katrin Leinweber, Lauren O'Brien, Michael Koontz, Paul Miller, Tracy Teal, Whalen

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science, Measurement and Data

Tag(s): Analysis, Data, Education, Geospatial, Open Scholarship Tools and Technologies, R, Reproducibility, Research Data Management Tools, Researchers, Spatial

Link to resource

Introduction to Geospatial Raster and Vector Data with R

Data Carpentry lesson to open, work with, and plot vector and raster-format spatial data in R. The episodes in this lesson cover how to …

Author(s): Ana Costa Conrado, Angela Li, Anne Fouilloux, Brett Lord-Castillo, Ethan P White, Joseph Stachelek, Juan F Fung, Katrin Leinweber, Klaus Schliep, Kristina Riemer, Lachlan Deer, Lauren O'Brien, Marchand, Punam Amratia, Sergio Marconi, Stéphane Guillou, Tracy Teal, zenobieg

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science, Measurement and Data

Tag(s): Analysis, Data, Education, Geospatial, Open Scholarship Tools and Technologies, R, Reproducibility, Research Data Management Tools, Researchers, Spatial

Link to resource

Introduction to Git & GitHub

This workshop introduces the basic concepts of Git version control. Whether you’re new to version control or just need an …

Author(s): Vicky Steeves

Type of resources: Activity/Lab

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Analysis, Open Scholarship Tools and Technologies, Research Data Management Tools, Researchers

Link to resource

Introduction to Jupyter Notebooks

This class is designed for first-time and longer-term users of Jupyter Notebooks, a workspace for writing code. The class focuses on …

Author(s): Nick Wolf, Vicky Steeves

Type of resources: Activity/Lab

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Analysis, Open Scholarship Tools and Technologies, Research Data Management Tools, Researchers

Link to resource

Introduction to Mediation, Moderation, and Conditional Process Analysis, First Edition: A Regression-Based Approach (Methodology in the Social Sciences)

Explaining the fundamentals of mediation and moderation analysis, this engaging book also shows how to integrate the two using an …

Author(s): Andrew F. Hayes

Type of resources: Textbook

Primary user(s): Student, Teacher, Researcher

Subject area(s): Applied Science, Social Science

Tag(s): Book

Link to resource

Introduction to Meta-Analysis

A book about meta analyses

Author(s): Borenstein, M., Hedges, L.V., Higgins, J. P. T., & Rothstein, H. R. (2009).

Type of resources: Reading

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Introduction to Open Science principles and practices (in Health Economics)

This series of lecture slides were used during the “Advanced Simulation for Health Economic Analysis” course at the …

Author(s): Xavier G.L.V. Pouwels

Type of resources: Lecture

Primary user(s): Teacher

Subject area(s): Life Science

Tag(s): Health Economics; Open Science; Introduction

Link to resource

Introduction to power

This video is an introduction to power analyses to improve the reproducibility of your research.

Author(s): Center for Open Science

Type of resources: Lecture

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science

Tag(s): Analysis, Data, Education, Materials, Open Scholarship Tools and Technologies, OSF, Publishing, Reproducibility, Research Data Management Tools, Researchers, Statistics

Link to resource

Introduction to Power Analyses in R

This video will introduce how to calculate statistical power in R using the pwr package.

All materials shown in the video, as well as …

Author(s): Courtney Soderberg

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Information Science

Tag(s): Analysis, Reproducibility

Link to resource

Introduction to Preprints

This is a recording of a 45 minute introductory webinar on preprints. With our guest speaker Philip Cohen, we’ll cover what …

Author(s): Center for Open Science

Type of resources: Lecture

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science

Tag(s): Analysis, Data, Education, Materials, Open Scholarship Tools and Technologies, OSF, Publishing, Reproducibility, Research Data Management Tools, Researchers

Link to resource

Introduction to R for Geospatial Data

The goal of this lesson is to provide an introduction to R for learners working with geospatial data. It is intended as a pre-requisite …

Author(s): Anne Fouilloux, butterflyskip, Chris Prener, Claudia Engel, David Mawdsley, Erin Becker, François Michonneau, Ido Bar, Jeffrey Oliver, Juan Fung, Katrin Leinweber, Kevin Weitemier, Kok Ben Toh, Lachlan Deer, Marieke Frassl, Matt Clark, Miles McBain, Naupaka Zimmerman, Paula Andrea Martinez, Preethy Nair, Raniere Silva, Rayna Harris, Richard McCosh, Vicken Hillis

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science, Measurement and Data

Tag(s): Analysis, Data, Education, Geospatial, Open Scholarship Tools and Technologies, R, Reproducibility, Research Data Management Tools, Researchers

Link to resource

Introduction to Research Data Management

An introduction to the concepts and best practices of research data management.

Author(s): Nick Wolf, Vicky Steeves

Type of resources: Activity/Lab

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Open Scholarship Guidelines, Research Data Management, Researchers, ResearchersOpen Scholarship Guidelines

Link to resource

Introduction to simulations

Introduction to simulation - first of a series of short lectures given at University of Oxford in February 2021. Very basic for those …

Author(s): Professor Dorothy Bishop

Type of resources: Video

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics, Social Science

Tag(s): Simulation

Link to resource

Introduction to the Command Line for Economics

Command line interface (OS shell) and graphic user interface (GUI) are different ways of interacting with a computer’s operating …

Author(s): Andras Vereckei, Arieda Muço, Miklós Koren

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science, Measurement and Data

Tag(s): Analysis, Data, Education, Open Scholarship Tools and Technologies, Reproducibility, Research Data Management Tools, Researchers, Shell

Link to resource

Introduction to the Command Line for Genomics

Data Carpentry lesson to learn to navigate your file system, create, copy, move, and remove files and directories, and automate …

Author(s): Amanda Charbonneau, Amy E. Hodge, Anita Schürch, Bastian Greshake Tzovaras, Bérénice Batut, Colin Davenport, Diya Das, Erin Alison Becker, François Michonneau, Giulio Valentino Dalla Riva, Jessica Elizabeth Mizzi, Karen Cranston, Kari L Jordan, Mattias de Hollander, Mike Lee, Niclas Jareborg, Omar Julio Sosa, Rayna Michelle Harris, Ross Cunning, Russell Neches, Sarah Stevens, Shannon EK Joslin, Sheldon John McKay, Siva Chudalayandi, Taylor Reiter, Tobi, Tracy Teal, Tristan De Buysscher

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science, Genetics, Measurement and Data

Tag(s): Analysis, Data, Education, Open Scholarship Tools and Technologies, Reproducibility, Research Data Management Tools, Researchers, Shell

Link to resource

Introduction to web scraping

Web scraping is the process of extracting data from websites. Some data that is available on the web is presented in a format that …

Author(s): Belinda Weaver, Joshua Dull, Thomas Guignard

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science, Measurement and Data

Tag(s): Analysis, Data, Education, Open Scholarship Tools and Technologies, Reproducibility, Research Data Management Tools, Researchers, Web Scraping

Link to resource

Investigating Variation in Replicability: A “Many Labs” Replication Project

Although replication is a central tenet of science, direct replications are rare in psychology. This research tested variation in the …

Author(s): Klein et al.

Type of resources: Primary Source, Reading, Reading

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Investigation and its discontents: Some constraints on progress in psychological research.

Examines several prominent trends in the conduct of psychological research and considers how they may limit progress in the field. …

Author(s): Paul L. Wachtel

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Ioannidis, J. P. A. (2012). Why science is not necessarily self-correcting. Perspectives on Psychological Science, 7, 645-654.

The ability to self-correct is considered a hallmark of science. However, self-correction does not always happen to scientific evidence …

Author(s): John Ioannidis

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Is Preregistration Worthwhile?

Proponents of preregistration argue that, among other benefits, it improves the diagnosticity of statistical tests. In the strong …

Author(s): Aba Szollosi, David Kellen, Danielle J. Navarro, Richard Shiffrin, Iris van Rooij, Trisha Van Zandt

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Preregistration, Theory Development, Inference

Link to resource

Is Psychology Suffering From a Replication Crisis? What Does “Failure to Replicate” Really Mean?

Psychology has recently been viewed as facing a replication crisis because efforts to replicate past study findings frequently do not …

Author(s): Scott E. Maxwell, Michael Y. Lau and George S. Howard

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Is the call to abandon p-values the red herring of the replicability crisis?

A paper about Is the call to abandon p-values the red herring of the replicability crisis?

Author(s): Victoria Savalei and Elizabeth Dunn

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Is the replicability crisis overblown? Three arguments examined

We discuss three arguments voiced by scientists who view the current outpouring of concern about replicability as overblown. The first …

Author(s): Pashler, H., & Harris, C. R.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Is there a credibility crisis in strategic management research? Evidence on the reproducibility of study findings

Recent studies report an inability to replicate previously published research, leading some to suggest that scientific knowledge is …

Author(s): Donald D Bergh, Barton M Sharp, Herman Aguinis and Ming Li

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Is There a Free Lunch in Inference?

The field of psychology, including cognitive science, is vexed by a crisis of confidence. Although the causes and solutions are varied, …

Author(s): Jeffrey N Rouder, Richard D Morey, Josine Verhagen, Jordan M Province, Eric-Jan Wagenmakers

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Math & Statistics, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science, Transparency

Link to resource

Issues in the Registration of Clinical Trials

Public concerns about the perils associated with incomplete or delayed reporting of results from clinical trials has heightened …

Author(s): Deborah A. Zarin; Nicholas C. Ide; Tony Tse; William R. Harlan; Joyce C. West; Donald A. B. Lindberg

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Preregistration, Clinical Trials

Link to resource

It’s Time to Broaden the Replicability Conversation: Thoughts for and From Clinical Psychological Science

Psychology is in the early stages of examining a crisis of replicability stemming from several high-profile failures to replicate …

Author(s): Jennifer L. Tackett et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Joy and rigor in behavioral science

In the past decade, behavioral science has seen the introduction of beneficial reforms to reduce false positive results. Serving as the …

Author(s): Hanne K.Collins et al.

Type of resources: Reading

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Kindness

Link to resource

Jupyter Notebooks with R & Git

Today we are going to learn the basics of literate programming using Jupyter Notebooks, a popular tool in data science, with the R …

Author(s): Vicky Steeves

Type of resources: Activity/Lab

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Git, GitHub, Jupyter Notebooks, Open Scholarship Tools and Technologies, R, Reproducibility, Research Data Management Tools, Researchers

Link to resource

Just Post It: The Lesson From Two Cases of Fabricated Data Detected by Statistics Alone

I argue that requiring authors to post the raw data supporting their published results has the benefit, among many others, of making …

Author(s): Uri Simonsohn

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science

Tag(s):

Link to resource

Justify your alpha

In response to recommendations to redefine statistical significance to P ≤ 0.005, we propose that researchers should transparently …

Author(s): Daniel Lakens et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Key Practices for the Language Scientist

Materials for the Key Practices for the Language Scientist course taught from January to March 2020 at the Max Planck Institute for …

Author(s): Drs Julia Egger and Eirini Zormpa

Type of resources: Module, Syllabus

Primary user(s): Student, Teacher

Subject area(s): Social Science

Tag(s): Teaching

Link to resource

La Terminal de Unix

Software Carpentry lección para la terminal de Unix La terminal de Unix ha existido por más tiempo que la mayoría de sus usuarios. Ha …

Author(s): Adam Huffman, Alejandra Gonzalez-Beltran, AnaBVA, Andrew Sanchez, Anja Le Blanc, Ashwin Srinath, Brian Ballsun-Stanton, Colin Morris, csqrs, Dani Ledezma, Dave Bridges, Erin Becker, Francisco Palm, François Michonneau, Gabriel A. Devenyi, Gerard Capes, Giuseppe Profiti, Gordon Rhea, Jake Cowper Szamosi, Jared Flater, Jeff Oliver, Jonah Duckles, Juan M. Barrios, Katrin Leinweber, Kelly L. Rowland, Kevin Alquicira, Kunal Marwaha, LauCIFASIS, Marisa Lim, Martha Robinson, Matias Andina, Michael Zingale, Nicolas Barral, Nohemi Huanca Nunez, Olemis Lang, Otoniel Maya, Paula Andrea Martinez, Raniere Silva, Rayna M Harris, Shirley Alquicira, Silvana Pereyra, sjnair, Stéphane Guillou, Steve Leak, Thomas Mellan, Veronica Jimenez-Jacinto, William L. Close, Yee Mey

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science, Measurement and Data

Tag(s): Analysis, Data, Education, Open Scholarship Tools and Technologies, Reproducibility, Research Data Management Tools, Researchers, Shell

Link to resource

lab.js Online research made easy

lab.js is a free, open, online study builder for the behavioral and cognitive sciences. (it works great in the lab, too)

Author(s): Felix Henninger

Type of resources: Activity/Lab

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Educators, Open Source, Open Source Software, Researchers

Link to resource

Landmark College Institute for Research & Training (LCIRT)

LCIRT was established in 2001 to pioneer LD research, discover innovative strategies and practices, and improve teaching and learning …

Author(s): Landmark College Institute

Type of resources: Website

Primary user(s): Student, Teacher, Parent

Subject area(s): Education, Social Science

Tag(s): Diversity, Equity, Inclusion, Neurodiversity

Link to resource

Last Contact: A Science Adventure

Last Contact is a game is a game that is intended to inform people about the open science movement. In the game players encounter …

Author(s): Anita Eerland, Karin Fikkers, Francine Boon, Victor van Doorn

Type of resources: Activity/Lab, Game, Student Guide

Primary user(s): Student, Teacher, Librarian

Subject area(s): Arts and Humanities, Education, Social Science

Tag(s): Puzzle, Escape Room

Link to resource

Last Week Tonight with John Oliver: Scientific Studies (HBO)

A video about scientific studies

Author(s): Last Week Tonight/John Oliver

Type of resources: Student Guide, Teaching/Learning Strategy, Video

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Video

Link to resource

Latent Variable Modeling with R

This book demonstrates how to conduct latent variable modeling (LVM) in R by highlighting the features of each model, their specialized …

Author(s): W. Holmes Finch, Brian F. French

Type of resources: Textbook

Primary user(s): Student, Teacher, Researcher

Subject area(s): Applied Science, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Book

Link to resource

Learn Stats with Jamovi

A statistical tutorial about using Jamovi

Author(s): Danielle Navarro and David R Foxcraft

Type of resources: Data Set, Textbook, Tutorial

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s): Statistical Book, Tutorial

Link to resource

Learning Statistics with R

The book is associated with the lsr package on CRAN and GitHub. The package is probably okay for many introductory teaching purposes, …

Author(s): Danielle Navarro

Type of resources: Textbook

Primary user(s): student, teacher

Subject area(s): Statistics and Probability

Tag(s): Statistics

Link to resource

Leonardo da Vinci, preregistration and the Architecture of Science: Towards a More Open and Transparent Research Culture

There has been much talk of psychological science undergoing a renaissance with recent years being marked by dramatic changes in …

Author(s): DB Connor

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Social Science

Tag(s): Preregistration, Health Psychology

Link to resource

Let’s Publish Fewer Papers

A paper about publishing few papers

Author(s): Leif D. Nelson, Joseph P. Simmons & Uri Simonsohn

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Let’s Put Our Money Where Our Mouth Is: If Authors Are to Change Their Ways, Reviewers (and Editors) Must Change With Them

A number of scholars recently have argued for fundamental changes in the way psychological scientists conduct and report research. The …

Author(s): Jon K. Maner

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Level up the reproducibility of your data and code! A 2-hour, hands-on workshop

Purpose: To introduce methods and tools in organization, documentation, automation, and dissemination of research that nudge it further …

Author(s): April Clyburne-Sherin

Type of resources: Activity/Lab

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Librarians, Reproducibility, Research Data Management Tools, Researchers

Link to resource

Leveraging Open Ecosystems to Enhance Reproducible Workflows

Open source infrastructure has paved the way for mission-aligned research stakeholders to create a united vision of interoperable tools …

Author(s):

Type of resources: Lesson

Primary user(s):

Subject area(s): Education

Tag(s): Center for Open Science, Jupyter, Open Code, Open Infrastructure, Open Research, Open Science, Open Science Framework, Open Source, OSF, Osfr, Protocols.io, Research, Research Integration, Research Tools

Link to resource

Library Carpentry: Introduction to Git

Library Carpentry lesson: An introduction to Git. What We Will Try to Do Begin to understand and use Git/GitHub. You will not be an …

Author(s): 222064h, abracarambar, ajtag, Alexander Gary Zimmerman, Alexander Mendes, Alex Mendes, Amiya Maji, Amy Olex, Andrew Lonsdale, Annika Rockenberger, Begüm D. Topçuoğlu, Belinda Weaver, Benjamin Bolker, Bill McMillin, Brian Moore, butterflyskip, Casey Youngflesh, Christopher Erdmann, Christoph Junghans, cmjt, Dan Michael O. Heggø, David Jennings, DSTraining, Erin Alison Becker, Evan Williamson, Garrett Bachant, Grant Sayer, hdinkel, Ian Lee, Jake Lever, Jamene Brooks-Kieffer, James Baker, James E McClure, James O'Donnell, James Tocknell, Janoš Vidali, Jeffrey Oliver, Jeremy Teitelbaum, Jeyashree Krishnan, João Rodrigues, Joe Atzberger, Jonah Duckles, Jonathan Cooper, jonestoddcm, Katherine Koziar, Katrin Leinweber, Kunal Marwaha, Kurt Glaesemann, Lauren Ko, L.C. Karssen, Lex Nederbragt, Madicken Munk, Maneesha Sane, Marie-Helene Burle, Mark Woodbridge, Martino Sorbaro, Matt Critchlow, Matteo Ceschia, Matthew Bourque, Matthew Hartley, Maxim Belkin, Megan Potterbusch, Michael Torpey, Michael Zingale, Mingsheng Zhang, Nicola Soranzo, Nima Hejazi, Nora McGregor, Oscar Arbeláez, Peace Ossom Williamson, pllim, Raniere Silva, Rayna Harris, Rémi Emonet, Rene Gassmoeller, Richard Barnes, Rich McCue, Ruud Steltenpool, Ryan Wick, Samniqueka Halsey, Samuel Lelièvre, Sarah Stevens, Saskia Hiltemann, Schlauch, Tobias, Scott Bailey, Shari Laster, Simon Waldman, Stefan Siegert, Thea Atwood, Thomas Morrell, Tim Dennis, Tommy Keswick, Tracy Teal, Trevor Keller, TrevorLeeCline, Tyler Crawford Kelly, Tyler Reddy, Umihiko Hoshijima, Veronica Ikeshoji-Orlati, Wes Harrell, William Sacks, Will Usher, Wolmar Nyberg Åkerström, Yuri

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Applied Science, Computer Science, Information Science, Measurement and Data

Tag(s): Analysis, Data, Education, Git, Librarians, Open Scholarship Tools and Technologies, Reproducibility, Research Data Management Tools

Link to resource

Library Carpentry: Introduction to Working with Data (Regular Expressions)

This Library Carpentry lesson introduces librarians and others to working with data. This Library Carpentry lesson introduces people …

Author(s): Alexander Mendes, Alex Volkov, Angus Taggart, Belinda Weaver, BertrandCaron, Bianca Peterson, Christopher Edsall, Christopher Erdmann, Chuck McAndrew, Dan Michael Heggø, Dan Michael O. Heggø, Elizabeth Lisa McAulay, fdsayre, Felix Hemme, François Michonneau, James Baker, Janice Chan, Jeffrey Oliver, Jeremy Guillette, Jodi Schneider, Jonah Duckles, Katherine Koziar, Katrin Leinweber, Kunal Marwaha, lsult, Paul R. Pival, PH03N1X007, remerjohnson, Saskia Scheltjens, Shari Laster, Tim Dennis, yvonnemery

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Applied Science, Information Science, Measurement and Data

Tag(s): Analysis, Data, Education, Librarians, Open Scholarship Tools and Technologies, Reproducibility, Research Data Management Tools

Link to resource

Library Carpentry: OpenRefine

Library Carpentry lesson: an introduction to OpenRefine for Librarians This Library Carpentry lesson introduces people working in …

Author(s): Alexander Mendes, andreamcastillo, Anna Neatrour, Antonin Delpeuch, Betty Rozum, Christina Koch, Christopher Erdmann, Daniel Bangert, dnesdill, Elizabeth Lisa McAulay, Evan Williamson, hauschke, Jamene Brooks-Kieffer, James Baker, Jamie Jamison, Jeffrey Oliver, Katherine Koziar, mhidas, Naupaka Zimmerman, Paul R. Pival, Rémi Emonet, Tim Dennis, Tom Honeyman, Tracy Teal

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Applied Science, Information Science, Measurement and Data

Tag(s): Analysis, Data, Education, Librarians, OpenRefine, Open Scholarship Tools and Technologies, Reproducibility, Research Data Management Tools

Link to resource

Library Carpentry: SQL

Library Carpentry, an introduction to SQL for Librarians This Library Carpentry lesson introduces librarians to relational database …

Author(s): 222064h, Anna-Maria Sichani, Belinda Weaver, Christopher Erdmann, Dan Michael Heggø, David Kane, Elaine Wong, Emanuele Lanzani, Fernando Rios, Jamene Brooks-Kieffer, James Baker, Janice Chan, Jeffrey Oliver, Katrin Leinweber, Kunal Marwaha, mdschleu, orobecca, Reid Otsuji, Ruud Steltenpool, thegsi, Tim Dennis

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Applied Science, Information Science, Measurement and Data

Tag(s): Analysis, Data, Education, Librarians, Open Scholarship Tools and Technologies, Reproducibility, Research Data Management Tools, SQL

Link to resource

Library Carpentry: The UNIX Shell

Library Carpentry lesson to learn how to use the Shell. This Library Carpentry lesson introduces librarians to the Unix Shell. At the …

Author(s): Adam Huffman, Alexander Konovalov, Alexander Morley, Alex Kassil, Alex Mendes, Ana Costa Conrado, Andrew Reid, Andrew T. T. McRae, Ariel Rokem, Ashwin Srinath, Bagus Tris Atmaja, Belinda Weaver, Benjamin Bolker, Benjamin Gabriel, BertrandCaron, Brian Ballsun-Stanton, Christopher Erdmann, Christopher Mentzel, colinmorris, Colin Sauze, csqrs, Dan Michael Heggø, Dave Bridges, David McKain, Dmytro Lituiev, earkpr, ekaterinailin, Elena Denisenko, Eric Jankowski, Erin Alison Becker, Evan Williamson, Farah Shamma, Gabriel Devenyi, Gerard Capes, Giuseppe Profiti, Halle Burns, Hannah Burkhardt, hugolio, Ian Lessing, Ian van der Linde, Jake Cowper Szamosi, James Baker, James Guelfi, Jarno Rantaharju, Jarosław Bryk, Jason Macklin, Jeffrey Oliver, jenniferleeucalgary, John Pellman, Jonah Duckles, Jonny Williams, Katrin Leinweber, Kevin M. Buckley, Kunal Marwaha, Laurence, Marc Gouw, Marie-Helene Burle, Marisa Lim, Martha Robinson, Martin Feller, Megan Fritz, Michael Lascarides, Michael Zingale, Michele Hayslett, Mike Henry, Morgan Oneka, Murray Hoggett, Nicolas Barral, Nicola Soranzo, Noah D Brenowitz, Owen Kaluza, Patrick McCann, Peter Hoyt, Rafi Ullah, Raniere Silva, Rémi Emonet, reshama shaikh, Ruud Steltenpool, sjnair, Stéphane Guillou, Stephan Schmeing, Stephen Jones, Stephen Leak, Susan J Miller, Thomas Mellan, Tim Dennis, Tom Dowrick, Travis Lilleberg, Victor Koppejan, Vikram Chhatre, Yee Mey

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Applied Science, Information Science, Measurement and Data

Tag(s): Analysis, Data, Education, Librarians, Open Scholarship Tools and Technologies, Reproducibility, Research Data Management Tools, Shell

Link to resource

Licensing your research

Join us for a 30 minute guest webinar by Brandon Butler, Director of Information Policy at the University of Virginia. This webinar …

Author(s): Center for Open Science

Type of resources: Lecture

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science

Tag(s): Analysis, Data, Education, Licensing, Materials, Open Scholarship Tools and Technologies, OSF, Publishing, Reproducibility, Research Data Management Tools, Researchers

Link to resource

Likelihood of Null Effects of Large NHLBI Clinical Trials Has Increased over Time

Background: We explore whether the number of null results in large National Heart Lung, and Blood Institute (NHLBI) funded trials has …

Author(s): Kaplan RM, Irvin VL

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Life Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science, Transparency

Link to resource

Linking to Data - Effect on Citation Rates in Astronomy

Is there a difference in citation rates between articles that were published with links to data and articles that were not? Besides …

Author(s): Alberto Accomazzi, Edwin A. Henneken

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Physical Science

Tag(s): Astrophysics, Computer Science, Data, Data Sharing, Digital Libraries, Instrumentation, Methods, Metrics

Link to resource

Local Grassroots Networks Engaging Open Science in Their Communities

This recorded webinar features insights from international panelists currently nurturing culture change in research among their local …

Author(s): Brian Nosek

Type of resources: Lesson

Primary user(s):

Subject area(s): Education

Tag(s): Open Science, Reproducibility, Reproducible Research, Research

Link to resource

Logical and methodological issues affecting genetic studies of humans reported in top neuroscience journals.

Genetics and neuroscience are two areas of science that pose particular methodological problems because they involve detecting weak …

Author(s): Grabitz, C.R. et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Life Science, Math & Statistics, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Longitudinal Structural Equation Modeling (Methodology in the Social Sciences)

Featuring actual datasets as illustrative examples, this book reveals numerous ways to apply structural equation modeling (SEM) to any …

Author(s): Todd D. Little

Type of resources: Textbook

Primary user(s): Student, Teacher, Researcher

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Book

Link to resource

Low Research-Data Availability in Educational-Psychology Journals: No Indication of Effective Research-Data Policies

Research-data availability contributes to the transparency of the research process and the credibility of educational-psychology …

Author(s): Markus Huff, Elke C. Bongartz

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Education

Tag(s): Educational Psychology, Research, Data Sharing, Data Policy, FAIR Data Principles, Data Transparency Levels, Open Data

Link to resource

Making ERP research more transparent: Guidelines for preregistration

A combination of confirmation bias, hindsight bias, and pressure to publish may prompt the (unconscious) exploration of various …

Author(s): Mariella Paul, Gisela H. Govaart, Antonio Schettino

Type of resources: Student Guide

Primary user(s): Student, Teacher

Subject area(s): Life Science, Social Science

Tag(s): Preregistration, EEG, Bias, Analytic Flexibility, ERP

Link to resource

Making Prospective Registration of Observational Research a Reality

The vast majority of health-related observational studies are not prospectively registered and the advantages of registration have not …

Author(s): Rafael Dal-Ré, John P. Ioannidis, Michael B. Bracken, Patricia A. Buffler, An-Wen Chan, Eduardo L. Franco, Carlo La Vecchia, ElisabeteWeiderpass

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Making Qualitative Data Reusable - A Short Guidebook For Researchers And Data Stewards Working With Qualitative Data

This guidebook aims to give an overview of the challenges associated with making qualitative data reusable as well as providing …

Author(s): Verburg, Maaike; Braukmann, Ricarda; Mahabier, Widia

Type of resources: Primary Source, Guide

Primary user(s): Student, Librarian, Researcher

Subject area(s): Arts and Humanities, Education, English Language Arts, Social Science

Tag(s): Qualitative Data, Secondary Analysis

Link to resource

Making replication mainstream

Many philosophers of science and methodologists have argued that the ability to repeat studies and obtain similar results is an …

Author(s): Rolf A. Zwaan, Alexander Etz Richard E. Lucas and M. Brent Donnellan

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Making science public: a review of journalists’ use of Open Science research

Science journalists are uniquely positioned to increase the societal impact of open science by contextualizing and communicating …

Author(s): Alice Fleerackers, Natascha Chtena, Stephen Pinfield, Juan Pablo Alperin, Germana Barata, Monique Oliveira, Isabella Peters

Type of resources: Reading

Primary user(s): Student, Teacher, Administrator, Librarian

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Open Science, Journalism, COVID-19

Link to resource

Managing a Personal Research Archive

A class on setting up and managing research materials; caring for digital files to enable collaboration, sharing, and re-use; and …

Author(s): Nick Wolf, Vicky Steeves

Type of resources: Activity/Lab

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Analysis, Materials, Open Scholarship Tools and Technologies, Research Data Management, Researchers

Link to resource

Managing Qualitative Social Science Data

A website with modules and lessons on qualitative research and data management

Author(s): Anon

Type of resources: Interactive, Lesson, Module

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Qualitative research

Link to resource

Many Labs 3: Evaluating participant pool quality across the academic semester via replication.

The university participant pool is a key resource for behavioral research, and data quality is believed to vary over the course of the …

Author(s): Charles R. Ebersole et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Mapping the universe of registered reports

Registered reports present a substantial departure from traditional publishing models with the goal of enhancing the transparency and …

Author(s): John P. A. Ioannidis, Tom E. Hardwicke

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Psychology

Tag(s): Preregistration, Publishing, Registered Reports

Link to resource

Materials for the Webinar “Helping Science Succeed: The Librarian’s Role in Addressing the Reproducibility Crisis”

Headlines and scholarly publications portray a crisis in biomedical and health sciences. In this webinar, you will learn what the …

Author(s): Amy Riegelman, Frank Sayre

Type of resources: Lesson

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Librarians, Reproducibility

Link to resource

Maximizing the Reproducibility of Your Research

A chapter to discuss maximising the reproducibility of research

Author(s): Open Science Collaboration

Type of resources: Primary Source, Reading, Chapter

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Measurement error and the replication crisis.

The assumption that measurement error always reduces effect sizes is false

Author(s): Loken, E., & Gelman, A.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Measurement Matters

This resource list contains reading material on the topic of measurement in psychological sciences. We hope the list will be a useful …

Author(s): Eiko Fried and Jessica Flake

Type of resources: Reading

Primary user(s): Student, Researcher

Subject area(s): Social Science

Tag(s): Collection, Reproducibility Knowledge

Link to resource

Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling

Cases of clear scientific misconduct have received significant media attention recently, but less flagrantly questionable research …

Author(s): Leslie K. John, George Loewenstein, Drazen Prelec

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Medical Product Industry Ties to Patient Advocacy Organizations’ Executive Leadership

Patient advocacy organizations (PAOs) advance patient interests through promotion of disease awareness, engagement with policymakers, …

Author(s): Shamik Bhat, Joseph S. Ross, Reshma Ramachandran

Type of resources: Reading

Primary user(s): Teacher, Administrator, Librarian, Journal Editors

Subject area(s): Life Science, Physical Science, Social Science

Tag(s): Transparency, Conflicts of Interest, Patient Advocacy Organizations, Industry

Link to resource

Medicines and Healthcare products Regulatory Agency’s “Consultation on proposals for legislative changes for clinical trials”: a response from the Trials Methodology Research Partnership Adaptive Designs Working Group, with a focus on data sharing

In the UK, the Medicines and Healthcare products Regulatory Agency consulted on proposals “to improve and strengthen the UK clinical …

Author(s): Martin Law, Dominique-Laurent Couturier, Babak Choodari-Oskooei, Phillip Crout, Carrol Gamble, Peter Jacko, Philip Pallmann, Mark Pilling, David S. Robertson, Michael Robling, Matthew R. Sydes, Sofía S. Villar, James Wason, Graham Wheeler, S. Faye Williamson, Christina Yap, & Thomas Jaki

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Consultation, Data Sharing, Legislation

Link to resource

Meeting the Requirements of Funders Around Open Science: Open Resources and Processes for Education

Expectations by funders for transparent and reproducible methods are on the rise. This session covers expectations for preregistration, …

Author(s): Center for Open Science

Type of resources: Lecture

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science

Tag(s): Analysis, Data, Education, Materials, Open Scholarship Tools and Technologies, OSF, Publishing, Reproducibility, Research Data Management Tools, Researchers

Link to resource

Meta-analyses are no substitute for registered replications: a skeptical perspective on religious priming

According to a recent meta-analysis, religious priming has a positive effect on prosocial behavior (Shariff et al., 2015). We first …

Author(s): Michiel van Elk, Dora Matzke, Quentin F. Gronau, Maime Guan, Joachim Vandekerckhove and Eric-Jan Wagenmakers

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Meta-assessment of bias in science

Numerous biases are believed to affect the scientific literature, but their actual prevalence across disciplines is unknown. To gain a …

Author(s): Daniele Fanelli, John P. A. Ioannidis, Rodrigo Costas

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Health, Medicine and Nursing, Biology, Physical Science, Social Science

Tag(s): Bias, Integrity, Meta-analysis, Meta-research, Metascience, Meta-Science, Misconduct, Publishing, Reproducibility, Research Methods

Link to resource

Metascience Forum 2020

In his talk, Professor Nosek defines replication as gathering evidence that tests an empirical claim made in an original paper. This …

Author(s): Brian Nosek

Type of resources: Lecture

Primary user(s):

Subject area(s): Education

Tag(s): Reproducibiity, Research, Scientific Process

Link to resource

Meta‐regression approximations to reduce publication selection bias.

Publication selection bias is a serious challenge to the integrity of all empirical sciences. We derive meta-regression approximations …

Author(s): TD Stanley, H Doucouliagos

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Methodological Advances in Behavioral Research: Crowdsourcing Science

The results of many published studies across many scientific domains are not easily reproduced by independent laboratories. For …

Author(s): artin Schweinsberg, Neil Bearden, & Eric Luis Uhlmann, INSEAD

Type of resources: Syllabus

Primary user(s): Teacher

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution; Open Science; Conceptual and statistical knowledge

Link to resource

Methods for Reliable, Transparent, and Open Science

A syllabus about the methods for reliable, transparent and open science

Author(s): Jeffrey Rouder and Joachim Vandekerckhove

Type of resources: Syllabus

Primary user(s): Teacher

Subject area(s): Applied Science, Education, Life Science, Math & Statistics, Social Science

Tag(s): Syllabus

Link to resource

Mindless statistics

Statistical rituals largely eliminate statistical thinking in the social sciences. Rituals are indispensable for identification with …

Author(s): Gerd Gigerenzer

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s):

Link to resource

Mini course on reproducibility & open research

This mini course was designed to introduce Psychology undergraduate students at the University of Edinburgh (who are working on their …

Author(s): Dr Alex Mitchell (University of Edinburgh), Niamh MacSweeney (Edinburgh ReproducibiliTea) and Laura Klinkhamer (Edinburgh ReproducibiliTea)

Type of resources: Lecture

Primary user(s): Student

Subject area(s): Social Science

Tag(s): INTRODUCTION, INTRODUCTORY, REPRODUCIBLE, PRINCIPLES, UNDERGRADUATE, DISSERTATION, THESIS, LECTURE, SERIES, VIDEOS, THREE TS, THREE T'S, PSYCHOLOGY

Link to resource

Mini Meta-Analysis of Your Own Studies: Some Arguments on Why and a Primer on How

We outline the need to, and provide a guide on how to, conduct a meta-analysis on one’s own studies within a manuscript. Although …

Author(s): Jin X. Goh, Judith A. Hall and Robert Rosenthal

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Misaligned Incentives Hurt Science, but We Can Fix Them

In this talk, Professor Corker shows how researchers are typically evaluated and contrasts that with ideal ways to evaluate the process …

Author(s): Katie Corker

Type of resources: Lecture

Primary user(s):

Subject area(s): Education

Tag(s): Research

Link to resource

Mission: P-curve

A blog about the p-curve

Author(s): Will Gervais

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Social Science

Tag(s): Blog

Link to resource

Mitigating Illusory Results through Preregistration in Education

Like performance-enhancing drugs inflating apparent athletic achievements, several common social science practices contribute to the …

Author(s): Hunter Gehlbach, Carly D. Robinson

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Education

Tag(s): Educational Research, P-Hacking, Preregistration, Replication, Research Methods

Link to resource

Multilevel Analysis: An Introduction to Basic and Advanced Multilevel Modeling

The main methods, techniques and issues for carrying out multilevel modeling and analysis are covered in this book. The book is an …

Author(s): Tom A.B. Snijders, Roel Bosker

Type of resources: Textbook

Primary user(s): Student, Teacher, Researcher

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Book

Link to resource

Negative results are disappearing from most disciplines and countries

Concerns that the growing competition for funding and citations might distort science are frequently discussed, but have not been …

Author(s): Daniele Fanelli

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Negativity towards negative results: a discussion of the disconnect between scientific worth and scientific culture.

Science is often romanticised as a flawless system of knowledge building, where scientists work together to systematically find …

Author(s): Matosin, N., Frank, E., Engel, M., Lum, J. S., & Newell, K. A.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Neuroscientist Explains

Daniel Glaser apprehensively revisits an article of his that saw some fallout due to a study he cited. But that study was not the only …

Author(s): Daniel Glaser

Type of resources: Student Guide, Teaching/Learning Strategy, Podcast

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Podcast, Reproducibility Knowledge, Reproducibility Crisis and Credibility Revolution

Link to resource

New trends in science communication fostering evidence-informed policymaking

TRESCA – Trustworthy, Reliable and Engaging Scientific Communication Approaches – is a research project aimed at understanding how …

Author(s): Gábor Szüdi, Pamela Bartar, Gorazd Weiss, Giuseppe Pellegrini, Marina Tulin, Tessa Oomen

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Science Communication, Evidence-Informed Policy-Making, Trust, Dialogue Model, Open Science, Visualisation, Digitalisation

Link to resource

NIAID Centers of Excellence for Influenza Research and Response (CEIRR) Data Standards

The goal of the NIAID Centers of Excellence for Influenza Research and Response (CEIRR) awards is to provide the federal government, …

Author(s): Centers of Excellence for Influenza Research and Response

Type of resources: Primary Source

Primary user(s): Student, Teacher, Librarian, researcher

Subject area(s): Life Science, Social Science

Tag(s): Influenza, Flu, Respiratory Illness, Data Standards

Link to resource

NIGMS Clearinghouse for Training Modules to Enhance Data Reproducibility

In January 2014, NIH launched a series of initiatives to enhance rigor and reproducibility in research. As a part of this initiative, …

Author(s): National Institutes of Health

Type of resources: Lecture

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Reproducibility, Research Administration, Research Data Management, Researchers

Link to resource

NIH Scientific Data Sharing

This webpage describes the NIH’s data management and sharing policies, including policies for genomic data sharing, and sharing …

Author(s): National Institutes of Health

Type of resources: Student Guide

Primary user(s): Student, Teacher, Administrator, Librarian

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): data sharing, federal policy

Link to resource

No evidence of publication bias in climate change science

Non-significant results are less likely to be reported by authors and, when submitted for peer review, are less likely to be published …

Author(s): Christian Harlos, Johan Hollander, Tim C. Edgell

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Physical Science

Tag(s): Analysis, Publication Bias, Reporting, Statistics

Link to resource

Novel methods to deal with publication biases: secondary analysis of antidepressant trials in the FDA trial registry database and related journal publications.

Objective: To assess the performance of novel contour enhanced funnel plots and a regression based adjustment method to detect and …

Author(s): Santiago G Moreno, Alex J Sutton, Erick H Turner, Keith R Abrams, Nicola J Cooper, Tom M Palmer, A E Ades

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Null Hypothesis Significance Testing. On the Survival of a Flawed Method

Null hypothesis significance testing (NHST) is the researcher’s workhorse for making inductive inferences. This method has often …

Author(s): J Krueger

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Math & Statistics, Social Science

Tag(s):

Link to resource

Null hypothesis significance testing: a review of an old and continuing controversy

Null hypothesis significance testing (NHST) is arguably the most widely used approach to hypothesis evaluation among behavioral and …

Author(s): Nickerson, R. S. (2000).

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

On Not Confusing the Tree of Trustworthy Statistics with the Greater Forest of Good Science: A Comment on Simmons et al.’s Perspective on Pre-registration

In this commentary on Simmons, Nelson, and Simonsohn (this issue), we examine their rationale for pre-registration within the broader …

Author(s): Michel Tuan Pham, Travis Tae Oh

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Preregistration, Open Science, Reproducibility Crisis, Consumer Research, Philosophy of Science

Link to resource

On the challenges of drawing conclusions from p-values just below 0.05

In recent years, researchers have attempted to provide an indication of the prevalence of inflated Type 1 error rates by analyzing the …

Author(s): Daniel Lakens

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

On the origins of the .05 level of statistical significance

Examination of the literature in statistics and probability that predates Fisher’s Statistical Methods for Research Workers …

Author(s): MICHAEL COWLES and CAROLINE DAVIS

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

On the plurality of (methodological) worlds: estimating the analytic flexibility of fMRI experiments

How likely are published findings in the functional neuroimaging literature to be false? According to a recent mathematical model, the …

Author(s): Carp, J.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

On the reproducibility of meta-analyses: six practical recommendations

Background: Meta-analyses play an important role in cumulative science by combining information across multiple studies and attempting …

Author(s): Daniel Lakens et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

On the reproducibility of psychological science

Investigators from a large consortium of scientists recently performed a multi-year study in which they replicated 100 psychology …

Author(s): Valen E. Johnson, Richard D. Payne, Tianying Wang, Alex Asher &Soutrik Mandal

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

On the reproducibility of science: unique identification of research resources in the biomedical literature

Scientific reproducibility has been at the forefront of many news stories and there exist numerous initiatives to help address this …

Author(s): Gregory M. LaRocca, Holly Paddock, Laura Ponting, Matthew H. Brush, Melissa A. Haendel, Nicole A. Vasilevsky, Shreejoy J. Tripathy

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Biology, Social Science

Tag(s): Data, Reproducibility

Link to resource

On the Surprising Longevity of Flogged Horses: Why There Is a Case for the Significance Test

Criticisms of null-hypothesis significance tests (NHSTs) are reviewed. Used as formal, two-valued decision procedures, they often …

Author(s): Robert P. Abelson

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics, Social Science

Tag(s):

Link to resource

One cheer for null hypothesis significance testing

Null hypothesis testing as a tool in research is defended. Six examples are offered of situations in which, if all the researcher could …

Author(s): H. Wainer

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Online experimentation and sampling in cognitive aging research

Online data collection methods have become increasingly popular in many domains of psychology, but their use in cognitive aging studies …

Author(s): Nathaniel R. Greene, Moshe Naveh-Benjamin

Type of resources: Reading, Student Guide

Primary user(s): Student, Teacher

Subject area(s): Life Science, Social Science

Tag(s): Online Experiments, Sampling, Cognitive Aging

Link to resource

Online Experiments for Language Scientists

Many areas in the language sciences rely on collecting data from human participants, from grammaticality judgments to behavioural …

Author(s): Kenny Smith

Type of resources: Syllabus

Primary user(s): Student, Teacher, Researchers

Subject area(s): Social Science

Tag(s): Programming

Link to resource

Open + Reproducible Research Workshop

Topics covered:

Understanding reproducible research Setting up a reproducible project Understanding power Preregistering your study …

Author(s): April Clyburne-Sherin, Courtney Soderberg

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Information Science

Tag(s): Documentation, Organizing, Reproducibility, Version Control

Link to resource

Open Access Directory

The Open Access Directory is an online compendium of factual lists about open access to science and scholarship, maintained by the …

Author(s): OAD Simmons

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Governmental Policies, Institutional Policies, Open Access, Open Access Policies, Organizational Change, Preprints, Publishing Models, Research Administration, Researchers

Link to resource

Open access journal publication in health and medical research and open science: Benefits, challenges and limitations

Scientific progress, including in evidence-based medicine, requires all available evidence to be accessed, scrutinised, interpreted and …

Author(s): Patricia Logullo, Jennifer A de Beyer, Shona Kirtley, Michael Maia Schlüssel, Gary S Collins

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Life Science, Social Science

Tag(s): Open Access, Health Research, Medical Research

Link to resource

Open Access Target Validation Is a More Efficient Way to Accelerate Drug Discovery

There is a scarcity of novel treatments to address many unmet medical needs. Industry and academia are finally coming to terms with the …

Author(s): Wen Hwa Lee

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Biology

Tag(s): Acute Myeloid Leukemia, Drug Discovery, Drug Research and Development, Drug Therapy, Leukemias, Lymphomas, Open Access Publishing, Phase I Clinical Investigation

Link to resource

Open Access Tracking Project

OATP is a crowd-sourced social-tagging project running on free software to capture new developments on open access to research. Its …

Author(s): Peter Suber

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Open Access, Researchers

Link to resource

Open and Reproducible Methods Syllabus

This course will examine current controversies and new developments in research methods in psychology. The goal of the course is to …

Author(s): Simine Vazire

Type of resources: Syllabus

Primary user(s): Teacher

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Open and Reproducible Research in Psychology

Psychological research is in the midst of what has been called the “replication crisis”, in which questions have been raised about the …

Author(s): Krista Byers-Heinlein

Type of resources: Syllabus

Primary user(s): Teacher

Subject area(s): Social Science

Tag(s): Transparency, Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Open and reproducible science

This course is designed to be delivered to master and doctoral students at the Universidade Federaldo Rio Grande do Sul (UFRGS). The …

Author(s): Daniel Umpierre de Moraes

Type of resources: Syllabus

Primary user(s): Teacher

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Open Developmental Science: An Overview and Annotated Reading List

The increasing adoption of open science practices in the last decade has been changing the scientific landscape across fields. However, …

Author(s): Sara Hart, Tamara Kalandadze

Type of resources: Reading

Primary user(s):

Subject area(s): Education

Tag(s):

Link to resource

Open Education in Promotion, Tenure, and Faculty Development

This resource was developed by a working group from the Iowa Open Education Action Team (Iowa OER). Our team built upon DOERS3’s …

Author(s): Abbey Elder, Anne Marie Gruber, Iowa Open Education Action Team (Iowa OER), Mahrya Burnett, Teri Koch

Type of resources: Reading

Primary user(s):

Subject area(s): Education

Tag(s): Faculty Development, Promotion, Tenure

Link to resource

Open Educational Resources: Equitable and Affordable Nursing Education

This research report describes findings from a study conducted to learn students’ experiences using open educational resources (OER) in …

Author(s): Friday, Vivienne Evet; Hunt, Cynthia

Type of resources: Reading

Primary user(s): Teacher, Administrator, Librarian

Subject area(s): Life Science

Tag(s): Accelerated Nursing Education, Barriers to Higher Education, Higher Education Costs, Open Educational Resources, Textbooks

Link to resource

Open for Insight (Experimental Methods + Open Science)

A syllabus about open science and experimental methods

Author(s): Rahal

Type of resources: Syllabus

Primary user(s): Teacher

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution; Open Science; Conceptual and statistical knowledge

Link to resource

Open hardware in microscopy

The field of microscopy has been empowering humankind for many centuries by enabling the observation of objects that are otherwise too …

Author(s): Johannes Hohlbein, Sanli Faez

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian, researcher

Subject area(s): Life Science, Physical Science

Tag(s): Open Microscopy, Open Source, Open Science, Open Hardware, Microscopy

Link to resource

Open Intro Statistics

OpenIntro Statistics is a dynamic take on the traditional curriculum, being successfully used at Community Colleges to the Ivy League

Author(s): OpenIntro

Type of resources: Textbook

Primary user(s): Student, Teacher, Administrator, Librarian

Subject area(s): Applied Science, Arts and Humanities, Education, Life Science, Social Science

Tag(s): Textbook; Open Textbook

Link to resource

Open peer review, pros and cons from the perspective of an early career researcher

Peer review is considered by many to be a fundamental component of scientific publishing. In this context, open peer review (OPR) has …

Author(s): Tania Henriquez

Type of resources: Primary Source, Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Open Peer Review, Early Career Researcher, Academic Life

Link to resource

Open Research: Examples of good practice, and resources across disciplines

This is a document about open science across disciplines, together with resources to these disciplines.

Author(s): Emily K Farran, Priya Silverstein, Camilla Gilmore

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Open science

Link to resource

Open Scholarship: Where are the Self-Correcting Mechanisms of Science?

In her talk, Professor Vazire covers the role of open practices in allowing science to work the way it was intended to work—through …

Author(s):

Type of resources: Lecture

Primary user(s):

Subject area(s): Education

Tag(s): Research

Link to resource

Open Science & Inclusive Psychology

This course is an introduction to the Open Science approach to psychology. We will investigate if how the field has experienced a …

Author(s): Benjamin Le

Type of resources: Syllabus

Primary user(s): Teacher

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science, Inclusivity, Diversity, Equity

Link to resource

Open Science 2.0: Towards a truly collaborative research ecosystem

Conversations about open science have reached the mainstream, yet many open science practices such as data sharing remain uncommon. Our …

Author(s): Robert T. Thibault, Olavo B. Amaral, Felipe Argolo, Anita E. Bandrowski, Alexandra R. Davidson, Natascha I. Drude

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian, editors

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Open Science, Ecosystems, Reproducibility, Quality Control, Clinical Trials, Science Policy, Open Data, Research Quality Assessment

Link to resource

Open Science and Methodological Improvements

In recent years, meta-research methods issues (e.g., reproducibility, open science, etc.) have yielded provocative findings, vigorous …

Author(s): John M. Zelenski

Type of resources: Syllabus

Primary user(s): Student, Teacher

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Open science challenges, benefits and tips in early career and beyond

The movement towards open science is a consequence of seemingly pervasive failures to replicate previous research. This transition …

Author(s): Christopher Allen, David M. A. Mehler

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Biology

Tag(s): Careers, Data, Experimental Design, Neuroimaging, Open Data, Open Science, Peer Review, Reproducibility, Statistical Data

Link to resource

Open science in dementia care embedded pragmatic clinical trials

Few dementia care interventions have been translated to healthcare contexts for those who need them. Embedded pragmatic clinical trials …

Author(s): Zachary G. Baker, Allison M. Gustavson, Lauren L. Mitchell, Joseph E. Gaugler

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Life Science, Social Science

Tag(s): Embedded Pragmatic Clinical Trials, Open Science, Dementia Care

Link to resource

Open Science in Latin America

Note: This webinar was presented in Spanish. The slides presented during this webinar can be found here:https://osf.io/6qnse/ The …

Author(s): Center for Open Science

Type of resources: Lecture

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science

Tag(s): Analysis, Data, Education, Materials, Open Scholarship Tools and Technologies, OSF, Publishing, Reproducibility, Research Data Management Tools, Researchers

Link to resource

Open Science in Sport and Exercise Psychology: Review of Current Approaches and Considerations for Qualitative Inquiry

Open science practices including open access (OA) publication, open methods, study preregistration, and open data are gaining …

Author(s): Tamminen, K.A. and Poucher, Z.A.

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Open Science is for Aging Research, Too

In response to concerns about the replicability of published research, some disciplines have used open science practices to try to …

Author(s): Isaacowitz Derek M, Lind Majse

Type of resources: Reading

Primary user(s):

Subject area(s): Biology

Tag(s): Aging Science

Link to resource

Open Science Made Easy

7 steps towards transparent and reproducible research

Author(s): Felix Schönbrodt, Friederike Hendriks, Mitja Back, Network for Open Science Initiatives

Type of resources: Diagram/Illustration

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Analysis, Reproducibility, Researchers

Link to resource

Open Science Manual

About This Document: This manual was assembled and is being updated by Professor Benjamin Le (@benjaminle), who is on the faculty in …

Author(s): Benjamin Le

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Psychology

Tag(s): Data, Materials, Policy, Publishing, Reproducibility, Researchers, Students

Link to resource

Open Science Practices are on the Rise: The State of Social Science (3S) Survey

Has there been meaningful movement toward open science practices within the social sciences in recent years? Discussions about changes …

Author(s): David J. Birke, Edward Miguel, Elizabeth Levy Paluck, Garret Christensen, Nicholas Swanson, Rebecca Littman, Zenan Wang

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Economics, Psychology

Tag(s): Data

Link to resource

Open Science Practices Need Substantial Improvement in Prognostic Model Studies in Oncology Using Machine Learning

Objective: To describe the frequency of open science practices in a contemporary sample of studies developing prognostic models using …

Author(s): Gary S. Collins, Rebecca Whittle, Garrett S. Bullock, Patricia Logullo, Paula Dhiman, Jennifer A. de Beyer, Richard D. Riley, Michael M. Schlussel

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Life Science, Social Science

Tag(s): Open Science, Prognosis, Machine Learning, Reporting, Data Sharing, Code Sharing

Link to resource

Open Science Saves Lives: Lessons from the COVID-19 Pandemic

In the last decade Open Science principles, such as Open Access, study preregistration, use of preprints, making available data and …

Author(s): Lonni Besançon et al.

Type of resources: Reading

Primary user(s): Student, Teacher, Policy maker

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Open science

Link to resource

Open Science Seminar: How to do credible research with a high informational value (and how not to do it)

Students have an overview about the “historical” developments and debates of the replication movement in the last years, …

Author(s): Felix Schönbrodt

Type of resources: Syllabus

Primary user(s): Teacher

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution; Open Science; Conceptual and statistical knowledge

Link to resource

Open Science Session GSA 2019

Open Science Session GSA 2019

Author(s): Derek Isaacowitz

Type of resources: Lecture Notes

Primary user(s):

Subject area(s): Health, Medicine and Nursing, Social Science

Tag(s): Aging, Gerontology, Open Science

Link to resource

Open science takes on the coronavirus pandemic

Data sharing, open-source designs for medical equipment, and hobbyists are all being harnessed to combat COVID-19.

Author(s): Mark Zastrow

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Life Science

Tag(s): Open Science, COVID-19, Pandemic

Link to resource

Open Science Toolbox

There is a vast body of helpful tools that can be used in order to foster Open Science practices. For reasons of clarity, this toolbox …

Author(s): Lutz Heil

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Data, Materials, Open Scholarship Guidelines, Policy, Publishing, Reproducibility, Researchers

Link to resource

Open Science Training in TRIPLE

This case study focuses on the online training activities on Open Science delivered within the H2020 project Transforming Research …

Author(s): Lottie Provost, Francesca Di Donato, Erzsébet Tóth-Czifra, Suzanne Dumouchel, Emilie Blotière, Yin Chen

Type of resources: Reading

Primary user(s): Teacher, Librarian

Subject area(s): Applied Science, Arts and Humanities, Social Science

Tag(s): Open Science, Training and Education, FAIR data, EOSC, Scholarly Practice, Project Management

Link to resource

Open Science Workshop Materials of the LMU Open Science Center

Open Science workshop materials created at LMU Munich, available for everyone under a CC-BY license. Look in the README folder for more …

Author(s): Angelika Stefan Felix Schönbrodt and Lena Schiestel

Type of resources: Syllabus

Primary user(s): Teacher

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution; Open Science; Conceptual and statistical knowledge

Link to resource

Open science: many hands make light work

Open science has been highlighted as one of the priorities of the Dutch presidency of the European Union in 2016. Matthew Dovey …

Author(s): Jisc

Type of resources: Student Guide, Teaching/Learning Strategy, Podcast

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Podcast, Reproducibility Crisis and Credibility Revolution

Link to resource

Open Science: What, Why, and How

Open Science is a collection of actions designed to make scientific processes more transparent and results more accessible. Its goal is …

Author(s): Bobbie Spellman, Elizabeth Gilbert and Katherine Corker

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Open sharing of data on close relationships and other sensitive social psychological topics: Challenges, tools, and future directions

This article reports on an adversarial (but friendly) collaboration examining the issues that lie at the intersection of …

Author(s): Joel, S., Eastwick, P., & Finkel, E.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Open Source Guides

Open Source Guides (https://opensource.guide/) are a collection of resources for individuals, communities, and companies who want to …

Author(s): GitHub

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Analysis, Open Source, Researchers, Software Engineers

Link to resource

open stats lab

A website about open statistic labs with data and activities

Author(s): Kevin P. McIntyre

Type of resources: Activity/Lab, Data Set

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Website

Link to resource

Open, rigorous and reproducible research: A practitioner’s handbook

This book starts from the premise that there is a lot we can all do to increase the benefits of research.

Let’s consider the main …

Author(s): Dallas Card, Yan Min, Stylianos Serghiou

Type of resources: Textbook

Primary user(s): Student, Teacher, Administrator, Librarian

Subject area(s): Life Science, Social Science

Tag(s): Textbook; Open Textbook

Link to resource

OpenAccess.net

The open-access.net platform provides comprehensive information on the subject of Open Access (OA) and offers practical advice on its …

Author(s): OpenAccess Germany

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Funders, Librarians, Open Access, Open Access Policies, Policy, Policy Makers, Publishers, Publishing, Research Administration, Researchers

Link to resource

Opening Science

The Evolving Guide on How the Internet is Changing Research, Collaboration and Scholarly Publishing

Author(s): Sönke Bartling and Sascha Friesike

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

OpenRefine for Social Science Data

Lesson on OpenRefine for social scientists. A part of the data workflow is preparing the data for analysis. Some of this involves data …

Author(s): Erin Becker, François Michonneau, Geoff LaFlair, Karen Word, Lachlan Deer, Peter Smyth, Tracy Teal

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Information Science, Measurement and Data, Social Science

Tag(s): Analysis, Data, Education, OpenRefine, Open Scholarship Tools and Technologies, Reproducibility, Research Data Management Tools, Researchers

Link to resource

Optimizing Research Collaboration

In this webinar, we demonstrate the OSF tools available for contributors, labs, centers, and institutions that support stronger …

Author(s): Center for Open Science

Type of resources: Lecture

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science

Tag(s): Analysis, Data, Education, Materials, Open Scholarship Tools and Technologies, OSF, Publishing, Reproducibility, Research Data Management Tools, Researchers

Link to resource

Optimizing Research Collaboration for Remote Teams

Many in the global research community are adapting to conducting work remotely while exploring the best ways to maintain collaboration …

Author(s): Center for Open Science

Type of resources: Lecture

Primary user(s):

Subject area(s): Education

Tag(s): Research

Link to resource

Optimizing Research Payoff

In this article, we present a model for determining how total research payoff depends on researchers’ choices of sample sizes, α …

Author(s): Jeff Miller and Rolf Ulrich

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

ORCC UKRN Primer on Working in Open Research

This is an introductory guide for those working and considering working in the area of open research. It was drafted by members of the …

Author(s): UK Reproducibility NetworkAnna Rhiannon HughesValerie McCutcheonNick SheppardKirsty WallisAngus WhyteIlkay HoltElizabeth NewboldNeil JacobsKirsty MerrettKate Ehrig-PageHelen Clare

Type of resources: Module, Reading, Primer

Primary user(s): Teacher, Administrator, Librarian

Subject area(s): Education, Life Science, Social Science

Tag(s): collaboration, data management, introductory guide, open research, outputs reproducibility research, UKRN

Link to resource

OSF Collections: supporting research discoverability and reuse

The OSF Collections repository platform supports the discoverability and reuse of research by enabling the aggregation of related …

Author(s):

Type of resources: Lesson

Primary user(s):

Subject area(s): Education

Tag(s): Center for Open Science, Open Science, Open Science Framework, OSF, OSF Collections, Reproducability, Reproducibility, Research, Research Best Practices, Research Tools, Research Transparency, TOP Guidelines

Link to resource

OSF Deep Dive

This video will introduce users of the Open Science Framework into some of the more advanced features.

Author(s): Center for Open Science

Type of resources: Lecture

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science

Tag(s): Analysis, Data, Education, Materials, Open Scholarship Tools and Technologies, OSF, Publishing, Reproducibility, Research Data Management Tools, Researchers

Link to resource

OSF in the Classroom

This webinar will introduce how to use the Open Science Framework (OSF; https://osf.io) in a Classroom. The OSF is a free, open source …

Author(s): Center for Open Science

Type of resources: Lecture

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science

Tag(s): Analysis, Data, Education, Materials, Open Scholarship Tools and Technologies, OSF, Publishing, Reproducibility, Research Data Management Tools, Researchers

Link to resource

OSF In The Lab: Organizing related projects with Links, Forks, and Templates

Files for this webinar are available at: https://osf.io/ewhvq/ This webinar focuses on how to use the Open Science Framework (OSF) to …

Author(s): Center for Open Science

Type of resources: Lecture

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science

Tag(s): Analysis, Data, Education, Inside Your Classroom, Materials, Open Scholarship Tools and Technologies, OSF, Publishing, Reproducibility, Research Data Management Tools, Researchers

Link to resource

OSF101

This webinar walks you through the basics of creating an OSF project, structuring it to fit your research needs, adding collaborators, …

Author(s): Center for Open Science

Type of resources: Lecture

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science

Tag(s): Analysis, Data, Education, Materials, Open Scholarship Tools and Technologies, OSF, Publishing, Reproducibility, Research Data Management Tools, Researchers

Link to resource

Our data, ourselves: A framework for using emotion in qualitative analysis

Qualitative training rarely acknowledges the role of emotions in both data collection and analysis. While bracketing emotions is an …

Author(s): Hilary Lustick

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Social Science

Tag(s): Reflexivity, Positionality, Race and Education, Whiteness, Restorative Practices

Link to resource

Outcome reporting bias in randomized-controlled trials investigating antipsychotic drugs

Recent literature hints that outcomes of clinical trials in medicine are selectively reported. If applicable to psychotic disorders, …

Author(s): C. H. Vinkers, C. M. C. Lemmens, J. J. Luykx, M. Lancee, R. S. Kahn

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Health, Medicine and Nursing

Tag(s): Publishing, Reporting, Reporting Bias, Reproducibility

Link to resource

Overcoming the Knowledge Barrier in Open Science

Getting started with open science and knowing where to go. This webinar will introduce participants to major practices in open science …

Author(s): Mellor et al.

Type of resources: Syllabus

Primary user(s): Teacher

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution; Open Science; Conceptual and statistical knowledge

Link to resource

Oxford-Berlin Open Research summer school 2019

This projects contains materials from lectures and workshops associated with the Oxford-Berlin Open Research Summer School 2019.

Author(s): Heise et al.

Type of resources: Module

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Open Research

Link to resource

P Values and Statistical Practice

An article about P Values and Statistical Practice

Author(s): Andrew Gelman

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

P values in display items are ubiquitous and almost invariably significant: A survey of top science journals

P values represent a widely used, but pervasively misunderstood and fiercely contested method of scientific inference. Display items, …

Author(s): Ioana Alina Cristea, John P. A. Ioannidis

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Statistics and Probability

Tag(s): Analysis, Analysis of Variance, Bayesian Method, Bayesian Statistics, Computer Software, Data, Meta-analysis, Publishing, Scientific Publishing, Software Tools, Statistical Data, Statistics

Link to resource

P-curve

An abstract about the p-curve

Author(s): Will Gervais

Type of resources: Reading

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Blog

Link to resource

P-curve visualization updated with log x-axis

My p-curve tool now lets you show the x-axis on a log₁₀ scale, which makes it a lot easier to look at really small p-values

Author(s): Kristoffer Magnusson

Type of resources: Interactive, Reading, Student Guide, Teaching/Learning Strategy

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Blog, Interaction

Link to resource

P-curve: A key to the file-drawer.

Because scientists tend to report only studies (publication bias) or analyses (p-hacking) that “work,” readers must ask, “Are these …

Author(s): Simonsohn, Uri, Nelson, Leif D., Simmons, Joseph P.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

P-Hacking and Other Problems in Psychology Research

What’s wrong with the social sciences? In this episode, Massimo and Julia are joined by Professor Daniel Lakens from the …

Author(s): Julia Galef, Massimo Pigliucci and Benny Pollak

Type of resources: Student Guide, Teaching/Learning Strategy, Podcast

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Podcast, Reproducibility Crisis and Credibility Revolution

Link to resource

PANGEA

PANGEA is the first power analysis program for general ANOVA designs (e.g., Winer, Brown, & Michels, 1991). PANGEA can handle …

Author(s): Jake Westfall

Type of resources: Reading, Simulation

Primary user(s): Student, Teacher, Researcher

Subject area(s): Math & Statistics

Tag(s): Reproducibility Knowledge, Power Analysis Tool

Link to resource

Participatory Research Approaches in Alzheimer’s Disease and Related Dementias Literature: A Scoping Review

Background and Objectives Given the increase in methodological pluralism in research on brain health, cognitive aging, and …

Author(s): Laurent Reyes, Clara J Scher, Emily A Greenfield

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Aging, Dementia, Lived Experience, Participatory Research, Scoping Review

Link to resource

Paths in strange spaces: A comment on preregistration

This is an archived version of a blog post on preregistration. The first half of the post argues that there is not a strong …

Author(s): Danielle Navarro

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Preregistration

Link to resource

PBFJ Editorial … Engaging with responsible science. “OPEN FOR BUSINESS” – Launching the PBFJ pre-registration publication initiative

This editorial note is an “action-based” follow-up to the previous PBFJ Editorial published in September 2021 … “A Declaration about …

Author(s): Robert Faff

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Business and Communication, Social Science

Tag(s): Finance, Business, Preregistration, Responsible Science

Link to resource

Peer Review: Decisions, decisions

Journals are exploring new approaches to peer review in order to reduce bias, increase transparency and respond to author preferences. …

Author(s): Peter Rodgers

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Information Science

Tag(s): Bias, Data, Funding Agencies, Peer Review, Publishing

Link to resource

Peer-review practices of psychological journals: The fate of published articles, submitted again.

A growing interest in and concern about the adequacy and fairness of modern peer-review practices in publication and funding are …

Author(s): Peters, D. P., & Ceci, S. J.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Open Science, Open Review

Link to resource

Performing high‐powered studies efficiently with sequential analyses

Running studies with high statistical power, while effect size estimates in psychology are often inaccurate, leads to a practical …

Author(s): Daniel Lakens

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Perspectives on Improving Methods in Psychological Science

A syllabus about open science

Author(s): Charlotte Hartwright

Type of resources: Syllabus

Primary user(s): Teacher

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Perspectives: Replication is more than meets the eye

Drawing on recent research and debates in social sciences, this paper situates replication in an advertising research context. We …

Author(s): Lars Bergkvist, Freya De Keyzer, Cristian Buzeta

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Business and Communication, Social Science

Tag(s): Methodology, Marketing, Advertising, Replication, Exact Replication, Close Replication, Constructive Replication, Conceptual Replication, Truth Value

Link to resource

Philosophical Psychology 1989

P. E. Meehl did first 10 sessions (Winter Quarter, Jan–Mar 1989). In the Spring Quarter, several other department members lectured on …

Author(s): P. E. Meehl

Type of resources: Primary Source, Student Guide, Teaching/Learning Strategy, Video

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Course, Video

Link to resource

Platform-controlled social media APIs threaten open science

Social media data enable insights into human behaviour. Researchers can access these data via platform-provided application programming …

Author(s): Brittany I. Davidson, Darja Wischerath, Daniel Racek, Douglas A. Parry, Emily Godwin, Joanne Hinds, Dirk van der Linden, Jonathan F. Roscoe, Laura Ayravainen, Alicia G. Cork

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Human Behaviour, Science, Technology, Society, Social Sciences

Link to resource

Plotting and Programming in Python

This lesson is part of Software Carpentry workshops and teach an introduction to plotting and programming using python. This lesson is …

Author(s): Adam Steer, Allen Lee, Andreas Hilboll, Ashley Champagne, Benjamin, Benjamin Roberts, CanWood, Carlos Henrique Brandt, Carlos M Ortiz Marrero, Cephalopd, Cian Wilson, Daniel W Kerchner, Dan Mønster, Daria Orlowska, Dave Lampert, David Matten, Erin Alison Becker, Florian Goth, Francisco J. Martínez, Greg Wilson, ian, Jacob Deppen, Jarno Rantaharju, Jeremy Zucker, Jonah Duckles, Kees den Heijer, Keith Gilbertson, Kyle E Niemeyer, Lex Nederbragt, Logan Cox, Louis Vernon, Lucy Dorothy Whalley, Madeleine Bonsma-Fisher, Mark Phillips, Mark Slater, Maxim Belkin, Michael Beyeler, Mike Henry, mzc9, Narayanan Raghupathy, Nigel Bosch, Olav Vahtras, Pablo Hernandez-Cerdan, Paul Anzel, Phil Tooley, Raniere Silva, Robert Woodward, Ryan Avery, Ryan Gregory James, Sarah M Brown, SBolo, Shyam Dwaraknath, Sourav Singh, Stéphane Guillou, Steven Koenig, Taylor Smith, Thor Wikfeldt, Timothy Warren, Tyler Martin, Vasu Venkateshwaran, Vikas Pejaver

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science, Measurement and Data

Tag(s): Analysis, Data, Education, Open Scholarship Tools and Technologies, Python, Reproducibility, Research Data Management Tools, Researchers

Link to resource

Point of View: How open science helps researchers succeed

Open access, open data, open source and other open scholarship practices are growing in popularity and necessity. However, widespread …

Author(s): Erin C McKiernan et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Policy recommendations to ensure that research software is openly accessible and reusable

Research data is optimized when it can be freely accessed and reused. To maximize research equity, transparency, and reproducibility, …

Author(s): Erin C. McKiernan, Lorena Barba, Philip E. Bourne, Caitlin Carter, Zach Chandler, Sayeed Choudhury, Stephen Jacobs, Daniel S. Katz, Stefanie Lieggi, Beth Plale, Greg Tananbaum

Type of resources: Reading

Primary user(s): Teacher, Administrator, Librarian, Policymaker

Subject area(s): Applied Science, Career and Technical Education

Tag(s): research, software, open software

Link to resource

Poor replication validity of biomedical association studies reported by newspapers

Objective To investigate the replication validity of biomedical association studies covered by newspapers. Methods We used a database …

Author(s): Andy Smith, Estelle Dumas-Mallet, François Gonon, Thomas Boraud

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Health, Medicine and Nursing

Tag(s): ADHD, Analysis, Bibliometrics, Biomarkers, Breast Cancer, Data, Health Risk Analysis, Mental Health and Psychiatry, Meta-analysis, Publishing, Scientific Publishing

Link to resource

Poor statistical reporting, inadequate data presentation and spin persist despite editorial advice

The Journal of Physiology and British Journal of Pharmacology jointly published an editorial series in 2011 to improve standards in …

Author(s): Annie A. Butler, Joanna Diong, Martin E. Héroux, Simon C. Gandevia

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Health, Medicine and Nursing

Tag(s): Bioassays and Physiological Analysis, Clinical Trials, Data, Data Processing, Drug Research and Development, Medical Journals, Pharmacology, Research Integrity, Research Reporting Guidelines, Research Validity, Scientific Publishing, Statistical Data

Link to resource

Positionality Statements are just the tip of the Iceberg: Moving toward a Reflexive Process

In July 2020, Journal of Women and Minorities in Science and Engineering (JWM) started requiring a positionality statement in each …

Author(s): Julie Martin, Renee Desing, Maura Borrego

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian, Journal Editors

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Positionality, Author Guidelines, Reviewer Guidelines, Reflexivity, Transparency

Link to resource

Power Analysis and Effect Size in Mixed Effects Models: A Tutorial.

In psychology, attempts to replicate published findings are less successful than expected. For properly powered studies replication …

Author(s): Brysbaert, M. and Stevens, M.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Power failure: why small sample size undermines the reliability of neuroscience

A study with low statistical power has a reduced chance of detecting a true effect, but it is less well appreciated that low power also …

Author(s): Katherine Button et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Power Posing: Reassessing The Evidence Behind The Most Popular TED Talk

A recent paper in Psych Science (.pdf) reports a failure to replicate the study that inspired a TED Talk that has been seen 25 million …

Author(s): Joe Simmons and Uri Simonsohn

Type of resources: Reading

Primary user(s): Student

Subject area(s): Education, Social Science

Tag(s): Blog

Link to resource

Power-up: A reanalysis of ‘power failure’ in neuroscience using mixture modeling.

Recently, evidence for endemically low statistical power has cast neuroscience findings into doubt. If low statistical power plagues …

Author(s): Camilla L. Nord, Vincent Valton, John Wood and Jonathan P. Roiser

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Practical Considerations for Navigating Registered Reports

Practical Considerations for Navigating Registered Reports

Author(s): Albers C, Button K S, Chambers C D, Helen Wills Neuroscience Institute University Of California Berkeley Ca Usa, Jason M, Jscimeca Berkeley Edu, Kiyonaga Berkeley Edu, Lakens D, Nosek B A, Poldrack R A

Type of resources: Reading

Primary user(s):

Subject area(s): Education

Tag(s): Research

Link to resource

Practical guides on open science for researchers in the Netherlands

A recently published series of open science guides help researchers in the Netherlands navigate open science. They cover topics such as …

Author(s): Giulia Trentacosti

Type of resources: Activity/Lab, Simulation

Primary user(s): Student

Subject area(s): Applied Science

Tag(s): Early Career Researcher, Preprints, Creative Commons, Open Educational Resources

Link to resource

Practical Solutions for Sharing Data and Materials From Psychological Research

Widespread sharing of data and materials (including displays and text- and video-based descriptions of experimental procedures) will …

Author(s): Rick O. Gilmore, Joy Lorenzo Kennedy, and Karen E. Adolph

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Open Science

Link to resource

Practical steps for increasing opennes and reproducibility: an introduction to OSF, Workshop Manual

Reproducibility, Research Management Planning, Structuring a study, Preregistration + Analysis Plan, Files and Version Control, Sharing …

Author(s): Center For Open Science, Courtney K, Ian Sullivan, Jennifer Freeman Smith, Jolene Esposito, Natalie Meyers, Nicole Pfeiffer, Sara Bowman

Type of resources: Reading

Primary user(s):

Subject area(s): Education

Tag(s): Research

Link to resource

Pre-analysis Plans Have Limited Upside, Especially Where Replications Are Feasible

The social sciences—including economics—have long called for transparency in research to counter threats to producing robust and …

Author(s): Lucas C. Coffman, Muriel Niederle

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Preregistration, Critique

Link to resource

Pre-analysis Plans: A Stocktaking

The evidence-based community has championed the public registration of pre-analysis plans (PAPs) as a solution to the problem of …

Author(s): Daniel Posner, George Ofosu

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Economics

Tag(s): Analysis, Policy

Link to resource

Pre-Analysis Plans: An Early Stocktaking

Pre-analysis plans (PAPs) have been championed as a solution to the problem of research credibility, but without any evidence that PAPs …

Author(s): George K. Ofosu, Daniel N. Posner

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Preregistration, Pre-analysis Plans

Link to resource

Pre-registration and Results-Free Review in Observational and Qualitative Research

Amidst rising concern about publication bias, pre-registration and results-blind review have grown rapidly in use. Yet discussion of …

Author(s): Alan M. Jacobs

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Social Science

Tag(s): Preregistration, Publication Bias, Qualitative Research, Political Science, Observational Research, Results-Blind Review

Link to resource

Pre-registration in social psychology—A discussion and suggested template

Pre-registration of studies before they are conducted has recently become more feasible for researchers, and is encouraged by an …

Author(s): van ‘t Veer, A.E., & Giner-Sorolla, R.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Transparency

Link to resource

Pre-Registration in the Undergraduate Dissertation: A Critical Discussion

Over recent years, psychology has become increasingly concerned with reproducibility and replicability of research findings (Munafò et …

Author(s): Madeleine Pownall

Type of resources: Teaching/Learning Strategy

Primary user(s): Teacher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Psychology, Undergraduate Students, Student Research, Teaching Methods, Research Reports, Supervision, Barriers, Role, Research Design, Foreign Countries

Link to resource

Pre-registration of clinical trials is associated with fewer positive findings

Pre-registration of clinical trials is associated with fewer positive findings

Author(s): Bill Gardner

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Life Science, Math & Statistics, Social Science

Tag(s): Blog, Open Science, Reproducibility Crisis and Credibility Revolution

Link to resource

Pre-registration of mathematical models

Pre-registration is a research practice where a protocol is deposited in a repository before a scientific project is performed. The …

Author(s): John P.A. Ioannidis

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Mathematical Modeling, Preregistration, Bias, Reproducibility

Link to resource

Pre-registration: Weighing costs and benefits for researchers

In the past decade, the social and behavioral sciences underwent a methodological revolution, offering practical prescriptions for …

Author(s): Jennifer M. Logg, Charles A. Dorison

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Social Science

Tag(s): Preregistration

Link to resource

Pre-registration: Why and How

In this article, we (1) discuss the reasons why pre-registration is a good idea, both for the field and individual researchers, (2) …

Author(s): Joseph P. Simmons, Leif D. Nelson, Uri Simonsohn

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Social Science

Tag(s): Preregistration, Consumer Psychology, Advertising

Link to resource

Pre-results Review in Economics: Lessons Learned from Setting up Registered Reports

Hear from Andrew Foster, editor at the Journal of Development Economics, and Irenaeus Wolff, a guest editor for Experimental Economics, …

Author(s): Aleksandar Bogdanoski, Andrew Foster, Irenaeus Wolff

Type of resources: Lesson

Primary user(s):

Subject area(s): Education

Tag(s): Academic Publishing, Center for Open Science, Data, Publishing Formats, Registered Reports, Research, Research Methods, Research Transparency

Link to resource

Prediction Interval: What to Expect When You’re Expecting … A Replication

A challenge when interpreting replications is determining whether the results of a replication “successfully” replicate the original …

Author(s): Jeffrey R.Spence and David J. Stanley

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Preliminary analysis of COVID-19 academic information patterns: a call for open science in the times of closed borders

The Pandemic of COVID-19, an infectious disease caused by SARS-CoV-2 motivated the scientific community to work together in order to …

Author(s): J. Homolak, I. Kodvanj & D. Virag

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): COVID-19, Open Science, Data, Bibliometric, Pandemic

Link to resource

Premiering pre-registration at PLOS Biology

Pre-registration promises to address some of the problems with traditional peer-review. As we publish our first Registered Report, we …

Author(s): Nonia Pariente

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Life Science

Tag(s): Peer Review, Research Grants, Neuroscience, Research Assessment, Medical Facies, Pandemics, Publication Ethics, Research Design

Link to resource

Preparing code and data for computationally reproducible collaboration and publication: a hands-on workshop

Computational analyses are playing an increasingly central role in research. Journals, funders, and researchers are calling for …

Author(s): April Clyburne-Sherin

Type of resources: Activity/Lab

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Data, Librarians, Materials, Reproducibility, Research Data Management Tools, Researchers

Link to resource

Preregistering qualitative research

The threat to reproducibility and awareness of current rates of research misbehavior sparked initiatives to better academic science. …

Author(s): Tamarinde L. Haven, Leonie Van Grootel

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Social Science

Tag(s): Preregistration, Qualitative Research

Link to resource

Preregistering Qualitative Research: A Delphi Study

Preregistrations—records made a priori about study designs and analysis plans and placed in open repositories—are thought to strengthen …

Author(s): Tamarinde L. Haven, Timothy M. Errington, Kristian Skrede Gleditsch, Leonie van Grootel, Alan M. Jacobs, Florian G. Kern, Rafael Piñeiro, Fernando Rosenblatt, Lidwine B. Mokkink

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Social Science

Tag(s): Preregistration, Qualitative Research

Link to resource

Preregistration and Incentives

Preregistering study designs is broadly supported as improving scientific credibility but criticized for limiting the scope of what can …

Author(s): Cole Williams

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Preregistration, Information Acquisition, Verifiable Disclosure

Link to resource

Preregistration and Registered Reports in Sociology: Strengths, Weaknesses, and Other Considerations

Both within and outside of sociology, there are conversations about methods to reduce error and improve research quality—one such …

Author(s): Bianca Manago

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Social Science

Tag(s): Preregistration, Open Science, Registered Reports, Reproducibility, Transparency

Link to resource

Preregistration and reproducibility

Many view preregistration as a promising way to improve research credibility. However, scholars have argued that using pre-analysis …

Author(s): Eirik Strømland

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Business and Communication, Social Science

Tag(s): Experimental Economics, Preregistration, Estimation, Reproducibility

Link to resource

Preregistration as a way to limit questionable research practice in advertising research

This paper discusses two phenomena that threaten the credibility of scientific research and suggests an approach to limiting the extent …

Author(s): Lars Bergkvist

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Social Science

Tag(s): Preregistration, Advertising, Questionable Research Practices

Link to resource

Preregistration Becoming the Norm in Psychological Science

A blog about Preregistration Becoming the Norm in Psychological Science

Author(s): Brian Nosek and Stephen Lindsay

Type of resources: Reading, blog

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Blog

Link to resource

Preregistration for Quantitative Research in Psychology Template

A template to use for pre-registration

Author(s): Pre-registration standards for psychology

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Transparency

Link to resource

Preregistration in Complex Contexts: A Preregistration Template for the Application of Cognitive Models

In recent years, open science practices have become increasingly popular in psychology and related sciences. These practices aim to …

Author(s): Nathan Evans, Sophia Crüwell

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Analysis, Preregistration, Publishing, Reproducibility, Researchers

Link to resource

Preregistration in diverse contexts: A preregistration template for the application of cognitive models

In recent years, open science practices have become increasingly popular in psychology and related sciences. These practices aim to …

Author(s): Sophia Crüwel, Nathan J. Evans

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Life Science, Social Science

Tag(s): Reproducibility, Transparency, Open Science, Cognitive Modelling, Preregistration

Link to resource

Preregistration in experimental linguistics: Applications, challenges, and limitations

The current publication system neither incentivizes publishing null results nor direct replication attempts, which biases the …

Author(s): Timo B. Roettger

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Social Science

Tag(s): Confirmatory, Exploratory, Preregistration, Publication Bias, Registered Report, Researcher Degrees of Freedom

Link to resource

Preregistration in infant research—A primer

Preregistration, the act of specifying a research plan in advance, is becoming more common in scientific research. Infant researchers …

Author(s): Naomi Havron, Christina Bergmann, Sho Tsuji

Type of resources: Reading, Student Guide

Primary user(s): Student, Teacher

Subject area(s): Social Science

Tag(s): Infant Research, Preregistration

Link to resource

Preregistration in Single-Case Design Research

To draw informed conclusions from research studies, research consumers need full and accurate descriptions of study methods and …

Author(s): Austin H. Johnson, Bryan G. Cook

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Preregistration, Single Case Design

Link to resource

Preregistration Is Hard, And Worthwhile

Preregistration clarifies the distinction between planned and unplanned research by reducing unnoticed flexibility. This improves …

Author(s): Brian A. Nosek, Emorie D. Beck, Lorne Campbell, Jessica K. Flake, Tom E. Hardwicke, David T. Mellor, Anna E. van ’t Veer, Simine Vazire

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Preregistration

Link to resource

Preregistration Is Neither Sufficient nor Necessary for Good Science

To address widespread perceptions of a reproducibility crisis in the social sciences, a growing number of scholars recommend the …

Author(s): Michel Tuan Pham, Travis Tae Oh

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Preregistration, Open Science, Reproducibility Crisis, Consumer Research, Critiques

Link to resource

Preregistration of Analyses of Preexisting Data

The preregistration of a study’s hypotheses, methods, and data-analyses steps is becoming a popular psychological research practice. To …

Author(s): Gaëtan Mertens, Angelos-Miltiadis Krypotos

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Replicability, Transparency, Open Science, Archival Data, Preregistration

Link to resource

Preregistration of animal research protocols: development and 3-year overview of preclinicaltrials.eu

Open, prospective registration of a study protocol can improve research rigour in a number of ways. Through preregistration, key …

Author(s): Mira van der Naald, Steven A J Chamuleau, Julia M L Menon, Wim de Leeuw, Judith de Haan, Dirk J Duncker, Kimberley Elaine Wever

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Life Science

Tag(s): Biomedical Research, Research Design

Link to resource

Preregistration of Epidemiologic Studies: An Ill-founded Mix of Ideas

The idea for the registration of observational epidemiologic studies (with their protocols, hypotheses, and analysis plans) is modeled …

Author(s): Jan P. Vandenbroucke

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Life Science

Tag(s): Preregistration, Epidemiology

Link to resource

Preregistration of Epidemiology Protocols (Author Response)

Professors Kogevinas and Stayner do not address the central point of my commentary but they do pursue an issue on which I welcome the …

Author(s): Michael B. Bracken

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Life Science

Tag(s): Preregistration, Epidemiology

Link to resource

Preregistration of Epidemiology Protocols (Response)

There must be something wrong when a professor of epidemiology includes the following statement in the concluding remarks of a …

Author(s): Manolis Kogevinas, Leslie Stayner

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Life Science

Tag(s): Preregistration, Epidemiology

Link to resource

Preregistration of Epidemiology Protocols: A Commentary in Support

It has been proposed that observational epidemiology protocols, just like those for randomized controlled trials (RCTs), should be …

Author(s): Michael B. Bracken

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Life Science

Tag(s): Preregistration, Epidemiology

Link to resource

Preregistration of exploratory research: Learning from the golden age of discovery

Preregistration of study protocols and, in particular, Registered Reports are novel publishing formats that are currently gaining …

Author(s): Ulrich Dirnagl

Type of resources: Reading

Primary user(s): Teacher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Preregistration, Study Protocols, Registered Reports, Exploratory Research, Confirmatory Research

Link to resource

Preregistration of Information Systems Research

In this paper, we introduce the preregistration concept for experiments in the information systems (IS) discipline. Preregistration …

Author(s): Eric Bogert, Aaron Schecter, Richard T. Watson

Type of resources: Reading, Student Guide

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Business and Communication, Social Science

Tag(s): Preregistration, Information Systems

Link to resource

Preregistration of Machine Learning Research

It is interesting to note that human intelligence thrives on what Peirce called abductive inferences (Peirce and Turrisi 1997, 241-56), …

Author(s): Mireille Hildebrandt

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Preregistration, Machine Learning, P-Hacking

Link to resource

Preregistration of Modeling Exercises May Not Be Useful

This is a commentary on Lee et al.’s (2019) article encouraging preregistration of model development, fitting, and evaluation. While we …

Author(s): Steven N. MacEachern, Trisha Van Zandt

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Preregistration, Modeling, Replication Crisis, Open Science, Exploratory, Robust Practices

Link to resource

Preregistration of Qualitative Research

In this webinar, Tamarinde Haven provides an overview of the process of preregistration in qualitative research. We review the process …

Author(s): Tamarinde Haven

Type of resources: Lecture

Primary user(s):

Subject area(s): Education

Tag(s): Open Science, OSF, Preregistration, Qualitative Methods, Qualitative Research, Research, Research Best Practices

Link to resource

Preregistration of Randomized Controlled Trials

Randomized controlled trials (RCTs) are designed to answer causal questions with internal validity. However, threats to internal …

Author(s): Bryan G. Cook, Vivian C. Wong, Jesse I. Fleming, Emily J. Solari

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Preregistration, Randomized Experiment, Outcome Study

Link to resource

Preregistration of secondary data analysis: A template and tutorial

Preregistration has been lauded as one of the solutions to the so-called ‘crisis of confidence’ in the social sciences and has …

Author(s): Olmo van den Akker, Sara Weston, Lorne Campbell, Bill Chopik, Rodica Damian, Pamela Davis-Kean, Andrew Hall, Jessica Kosie, Elliott Kruse, Jerome Olsen, Stuart Ritchie, KD Valentine, Anna van 't Veer, Marjan Bakker

Type of resources: Student Guide

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Preregistration, Secondary Data Analysis

Link to resource

Preregistration of Studies with Existing Data

Preregistration of research plans is becoming an increasingly popular and common tool to enhance the transparency of a study’s …

Author(s): Gaëtan Mertens, Angelos-Miltiadis Krypotos

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Replicability, Transparency, Questionable Research Practices, Open Science, Existing Data, Secondary Data Analysis, Preregistration

Link to resource

Preregistration of Study Protocols Is Unlikely to Improve the Yield From Our Science, But Other Strategies Might

A recent workshop focused on the idea that protocols for nonrandomized studies should be registered in advance of their conduct. In …

Author(s): Timothy Lash

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Epidemiology, Preregistration

Link to resource

Preregistration Overview page

What is Preregistration? When you preregister your research, you’re simply specifying your research plan in advance of your study …

Author(s): Center for Open Science

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Funders, Preregistration, Publishers, Reproducibility, Researchers

Link to resource

Preregistration specificity & adherence: A review of preregistered gambling studies & cross-disciplinary comparison

Study preregistration is one of several “open science” practices (e.g., open data, preprints) that researchers use to improve the …

Author(s): Robert Heirene, Debi LaPlante, Eric R. Louderback, Brittany Keen, Marjan Bakker, Anastasia Serafimovska, Sally Melissa Gainsbury

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Preregistration, Gambling Studies

Link to resource

Preregistration, Replication, and Nonexperimental Studies

In last month’s column, I worried about whether encouraging us to preregister our hypotheses and analysis plan before running studies …

Author(s): Susan Goldin-Meadow

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Preregistration, Longitudinal Research

Link to resource

Preregistration: Definition, Advantages, Disadvantages, and How It Can Help Against Questionable Research Practices

Questionable research practices (QRPs), such as p-hacking (i.e., the inappropriate manipulation of data analysis to find statistical …

Author(s): Angelos-Miltiadis Krypotos, Gaetan Mertens, Irene Klugkist, Iris M. Engelhard

Type of resources: Reading, Student Guide

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Questionable Research Practices, Preregistration, Psychological Science, Clinical Science

Link to resource

Preregistration: Improve Research Rigor, Reduce Bias

In this webinar Professor Brian Nosek, Executive Director of the Center for Open Science (https://cos.io), outlines the practice of …

Author(s): Center for Open Science

Type of resources: Lecture

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science

Tag(s): Analysis, Data, Education, Materials, Open Scholarship Tools and Technologies, OSF, Preregistation, Publishing, Reproducibility, Research Data Management Tools, Researchers

Link to resource

Preregistration: Practical Considerations for Speech, Language, and Hearing Research

Purpose: In the last decade, psychology and other sciences have implemented numerous reforms to improve the robustness of our research, …

Author(s): Violet A. Brown, Julia F. Strand

Type of resources: Student Guide

Primary user(s): Student, Teacher

Subject area(s): Business and Communication, Social Science

Tag(s): Speech, Language, Hearing, Preregistration

Link to resource

Preregistration: the good, the bad, and the confusing

Preregistration is a tool to enhance the reliability of science that has been promoted as a normative requirement for the award of …

Author(s): Alain de Cheveigné

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Meta-science, Neuroscience, Social and Behavioral Sciences, Life Sciences

Link to resource

Presentations Given by Center for Open Science

A collection of slides for virtually all presentations given by Center for Open Science staff since its founding in 2013.

Author(s): Center for Open Science

Type of resources: Lesson

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Data, Funders, Librarians, Materials, Open Science Policy, Policy, Policy Makers, Publishers, Publishing, Reproducibility, Research Administration, Researchers, Workflow Tools

Link to resource

Preserving privacy in the era of openness: Commentary on open science requirements for identifiable data in psychological science journals

Psychological science journals are increasingly adopting open science (OS) policies (e.g., Transparency and Openness Promotion) …

Author(s): Mollie Ruben, Morgan Stosic

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Arts and Humanities, Social Science

Tag(s): open data, transparency and openness promotion, TOP factor, privacy

Link to resource

Priming, Replication, and the Hardest Science

Concerns have been raised recently about the replicability of behavioral priming effects, and calls have been issued to identify …

Author(s): Joseph Cesairo

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Privacy and Data-Based Research

What can we, as users of microdata, formally guarantee to the individuals (or firms) in our dataset, regarding their privacy? We retell …

Author(s): Heffetz, Ori, and Katrina Liggett.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Social Science

Tag(s):

Link to resource

Probing Birth-Order Effects on Narrow Traits Using Specification-Curve Analysis

The idea that birth-order position has a lasting impact on personality has been discussed for the past 100 years. Recent large-scale …

Author(s): Julia M Rohrer, Boris Egloff, Stefan C Schmukle

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Problems in using p-curve analysis and text-mining to detect rate of p-hacking.

Background. The p-curve is a plot of the distribution of p-values reported in a set of scientific studies. Comparisons between ranges …

Author(s): Bishop, D. V, & Thompson, P. A.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Procedimentos para investigação científica

O profissional formado pelo Curso de Graduação em Psicologia da Unifesspa caracterizar-se-á porpossuir uma formação pluralista e …

Author(s): Caio Maximino and Lúcia Cavalcante

Type of resources: Syllabus

Primary user(s): Teacher

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution; Open Science; Conceptual and statistical knowledge

Link to resource

Programming Psychological Experiments using python and OpenSesame

Using this material, you can learn how to code in python, build experiments using OpenSesame, and do basic data preprocessing in …

Author(s): Jeshua Tromp, Henk van Steenbergen

Type of resources: Full Course

Primary user(s): Student, Teacher

Subject area(s): Social Science

Tag(s): Python, Experiments, OpenSesame

Link to resource

Programming with MATLAB

The best way to learn how to program is to do something useful, so this introduction to MATLAB is built around a common scientific …

Author(s): Gerard Capes

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science, Measurement and Data

Tag(s): Analysis, Data, Education, MATLAB, Open Scholarship Tools and Technologies, Reproducibility, Research Data Management Tools, Researchers

Link to resource

Programming with Python

The best way to learn how to program is to do something useful, so this introduction to Python is built around a common scientific …

Author(s): Anne Fouilloux, Lauren Ko, Maxim Belkin, Trevor Bekolay, Valentina Staneva

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science, Measurement and Data

Tag(s): Analysis, Data, Education, Open Scholarship Tools and Technologies, Python, Reproducibility, Research Data Management Tools, Researchers

Link to resource

Programming with R

The best way to learn how to program is to do something useful, so this introduction to R is built around a common scientific task: …

Author(s): Diya Das, Katrin Leinweber, Rohit Goswami

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science, Measurement and Data

Tag(s): Analysis, Data, Education, Open Scholarship Tools and Technologies, R, Reproducibility, Research Data Management Tools, Researchers

Link to resource

Project Organization and Management for Genomics

Data Carpentry Genomics workshop lesson to learn how to structure your metadata, organize and document your genomics data and …

Author(s): Amanda Charbonneau, Bérénice Batut, Daniel O. S. Ouso, Deborah Paul, Erin Alison Becker, François Michonneau, Jason Williams, Juan A. Ugalde, Kevin Weitemier, Laura Williams, Paula Andrea Martinez, Peter R. Hoyt, Rayna Michelle Harris, Taylor Reiter, Toby Hodges, Tracy Teal

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Management, Genetics

Tag(s): Analysis, Data, Education, Genomics, Metadata, Open Scholarship Tools and Technologies, Reproducibility, Research Data Management Tools, Researchers

Link to resource

Project Teaching Integrity in Empirical Research (TIER)

The Project Teaching Integrity in Empirical Research (TIER) develops methods and tools for enhancing research transparency through …

Author(s): TIER Team

Type of resources: Teaching/Learning Strategy

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Education, Educators, Open Scholarship Guidelines, Researchers, Workflow Tools

Link to resource

Promises and Perils of Pre-analysis Plans

The purpose of this paper is to help think through the advantages and costs of rigorous pre-specification of statistical analysis plans …

Author(s): Benjamin A. Olken

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Business and Communication

Tag(s): Preregistration, Pre-Analysis Plans, Economics

Link to resource

Promoting an open research culture

Author guidelines for journals could help to promote transparency, openness, and reproducibility

Author(s): Nosek et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Promoting Transparency in Social Science Research

Social scientists should adopt higher transparency standards to improve the quality and credibility of research.

Author(s): Miguel, E., C. Camerer, K. Casey, J. Cohen, K. M. Esterling, A. Gerber, R. Glennerster, D. P. Green, M. Humphreys, G. Imbens, D. Laitin, T. Madon, L. Nelson, B. A. Nosek, M. Petersen, R. Sedlmayr, J. P. Simmons, U. Simonsohn, M. Van der Laan.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Promotion & Tenure: Aligning incentives with institutional values and open science

This repository provides resources for faculty and academic units to re-imagine faculty rewards and incentives, and in particular how …

Author(s): Michael Dougherty, Ulrich Mayr, Sanjay Srivastava, Morton Ann Gernsbacher, Junaid Salim Merchant, Megan Fitter, Russell Poldrack, Edward Bernat, L. Robert Slevc, & Conor T. McLennan

Type of resources: Case Study, Lecture, Reading

Primary user(s): Teacher, Administrator, Librarian

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Incentives, Tenure, Institutional Values, Inclusion

Link to resource

Psychological testing and psychological assessment: A review of evidence and issues.

This article summarizes evidence and issues associated with psychological assessment. Data from more than 125 meta-analyses on test …

Author(s): Meyer et al.

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science

Tag(s):

Link to resource

Psychologists Are Open to Change, yet Wary of Rules

Psychologists must change the way they conduct and report their research—this notion has been the topic of much debate in recent years. …

Author(s): Heather M. Fuchs, Mirjam Jenny, Susann Fiedler

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Psychology as a Robust Science

Is psychology a robust science? To answer such a question, this course will encourage you to think critically about how psychological …

Author(s): Amy Orben

Type of resources: Syllabus

Primary user(s): Teacher

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution; Open Science; Conceptual and statistical knowledge

Link to resource

Psychology’s renaissance

In 2010–2012, a few largely coincidental events led experimental psychologists to realize that their approach to collecting, analyzing, …

Author(s): Leif D. Nelson, Joseph Simmons, and Uri Simonsohn

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Psychology’s Replication Crisis and the Grant Culture: Righting the Ship

The past several years have been a time for soul searching in psychology, as we have gradually come to grips with the reality that some …

Author(s): Scott O Lilienfeld

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Psychology, Science, and Knowledge Construction: Broadening Perspectives from the Replication Crisis

Psychology advances knowledge by testing statistical hypotheses using empirical observations and data. The expectation is that most …

Author(s): Patrick E. Shrout and Joseph L. Rodgers

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Psychology’s Credibility Revolution

In 2011, Daryl Bem published a paper that seemed to demonstrate evidence for extra sensory perception (ESP). Four years later, the Open …

Author(s): Julia Strand

Type of resources: Syllabus

Primary user(s): Teacher

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

PsyTeachR

Materials for the University of Glasgow Institute of Neuroscience and Psychology’s undergraduate and MSc methods courses + Experiences, …

Author(s):

Type of resources: Textbook

Primary user(s): student, teacher

Subject area(s): Psychology

Tag(s): Curriculum Change, Education, Educators, Open Education, Open Scholarship Tools and Technologies, Reproducibility, Students

Link to resource

Public Availability of Published Research Data in High-Impact Journals

Background There is increasing interest to make primary data from published research publicly available. We aimed to assess the current …

Author(s): Alawi A. Alsheikh-Ali, John P. A. Ioannidis, Mouaz H. Al-Mallah, Waqas Qureshi

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Health, Medicine and Nursing

Tag(s): Bibliometrics, Clinical Trials, Data, Data Processing, Microarrays, Policy, Public Policy, Publishing, Reproducibility, Science Policy, Scientific Publishing

Link to resource

Public Data Archiving in Ecology and Evolution: How Well Are We Doing?

Policies that mandate public data archiving (PDA) successfully increase accessibility to data underlying scientific publications. …

Author(s): Dominique G. Roche, Loeske E. B. Kruuk, Robert Lanfear, Sandra A. Binning

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Biology

Tag(s): Archives, Data, Data Processing, Evolutionary Biology, Evolutionary Genetics, Public Policy, Reproducibility, Science Policy, Scientific Publishing

Link to resource

Publication bias and the canonization of false facts.

Science is facing a “replication crisis” in which many experimental findings cannot be replicated and are likely to be false. Does this …

Author(s): Nissen, S. B., Magidson, T., Gross, K., & Bergstrom, C. T.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Publication bias and the limited strength model of self-control: has the evidence for ego depletion been overestimated?

Few models of self-control have generated as much scientific interest as has the limited strength model. One of the entailments of this …

Author(s): Evan C. Carter and Michael E. McCullough

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Publication bias in clinical research

In a retrospective survey, 487 research projects approved by the Central Oxford Research Ethics Committee between 1984 and 1987, were …

Author(s): P.J Easterbrook, R Gopalan, J.A Berlin, D.R Matthews

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Life Science, Social Science

Tag(s): Publication Bias, Clinical Research

Link to resource

Publication Bias in Psychology: A Diagnosis Based on the Correlation between Effect Size and Sample Size

Background: The p value obtained from a significance test provides no information about the magnitude or importance of the underlying …

Author(s): Anton Kühberger, Astrid Fritz, Thomas Scherndl

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Publication bias in the social sciences: Unlocking the file drawer

We studied publication bias in the social sciences by analyzing a known population of conducted studies—221 in total—in which there is …

Author(s): Annie Franco, Neil Malhotra, Gabor Simonovits

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Publication Decisions and their Possible Effects on Inferences Drawn from Tests of Significance—or Vice Versa

There is some evidence that in fields where statistical tests of significance are commonly used, research which yields nonsignificant …

Author(s): Theodore D. Sterling

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics, Social Science

Tag(s):

Link to resource

Publication of NIH funded trials registered in ClinicalTrials.gov: Cross sectional analysis

Objective To review patterns of publication of clinical trials funded by US National Institutes of Health (NIH) in peer reviewed …

Author(s): Joseph S Ross, Tony Tse, Deborah A Zarin, Hui Xu, Lei Zhou, Harlan M Krumholz

Type of resources: Reading

Primary user(s): Student, Teacher, Administrator, Librarian

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): NIH, Clinical Research, Preregistration

Link to resource

Publication Prejudices: An Experimental Study of Confirmatory Bias in the Peer Review System

Confirmatory bias is the tendency to emphasize and believe experiences which support one’s views and to ignore or discredit those …

Author(s): Michael J. Mahoney

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Publication pressure and scientific misconduct in medical scientists

There is increasing evidence that scientific misconduct is more common than previously thought. Strong emphasis on scientific …

Author(s): Joeri K. Tijdink, Reinout Verbeke, Yvo M. Smulders

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Publication rate in preclinical research: A plea for preregistration

Objectives The ultimate goal of biomedical research is the development of new treatment options for patients. Animal models are used if …

Author(s): Mira van der Naald, Steven Wenker, Pieter A Doevendans, Kimberley E Wever, Steven A J Chamuleau

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Life Science, Social Science

Tag(s): Preregistration, Translational Research, Publication Rate, Publication Bias

Link to resource

Python for Harvesting Data on the Web

This session is an intermediate-to-advanced level class that offers some ideas for how to approach the following common data wrangling …

Author(s): Nick Wolf, Vicky Steeves

Type of resources: Activity/Lab

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Analysis, Open Scholarship Tools and Technologies, Research Data Management Tools, Researchers

Link to resource

Python for Humanities

Python is a general purpose programming language that is useful for writing scripts to work effectively and reproducibly with data. …

Author(s): Iain Emsley

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science, Measurement and Data

Tag(s): Analysis, Data, Education, Open Scholarship Tools and Technologies, Python, Reproducibility, Research Data Management Tools, Researchers

Link to resource

QDR Data for Teaching

As part of its mission, QDR promotes the pedagogical use of published qualitative and multi-method data projects. Shared qualitative …

Author(s): Qualitative Data Repository

Type of resources: Data Set, Primary Source

Primary user(s): Student, Teacher, Librarian

Subject area(s): Arts and Humanities, Education, English Language Arts, History, Law, Social Science

Tag(s):

Link to resource

Qualitative Research Using Open Tools

Qualitative research has long suffered from a lack of free tools for analysis, leaving no options for researchers without significant …

Author(s): Beth M. Duckles, Vicky Steeves

Type of resources: Lesson

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Analysis, Librarians, Open Scholarship Tools and Technologies, Qualitative, R, Research Data Management Tools, Researchers, Taguette

Link to resource

Quality Indicators of Secondary Data Analyses in Special Education Research: A Preregistration Guide

Secondary data analyses occur when new analyses are proposed for existing data. Although they are prevalent in special education …

Author(s): Allison R. Lombardi, Graham G. Rifenbark, Ashley Taconet

Type of resources: Student Guide

Primary user(s): Student, Teacher

Subject area(s): Education

Tag(s): Secondary Data Analysis, Special Education, Preregistration

Link to resource

Quality Uncertainty Erodes Trust in Science

When consumers of science (readers and reviewers) lack relevant details about the study design, data, and analyses, they cannot …

Author(s): S. Vazire

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Quantitative Emerging Practices and Methods in Psychology

Recently, a reporter in the Chronicle of Higher Education wrote that “psychology is having an uneasy moment” (Zamudio-Suarez, 2016). …

Author(s): Jennifer Howell

Type of resources: Syllabus

Primary user(s): Student, Teacher

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Questionable and Open Research Practices in Education Research

Discussions of how to improve research quality are predominant in a number of fields, including education. But how prevalent are the …

Author(s): Bryan G. Cook, Jaret Hodges, Jonathan Plucker, Matthew C. Makel

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Education

Tag(s): Data, Open Data, Preregistration, Research Methods

Link to resource

Questionable research practices among Italian research psychologists

A survey in the United States revealed that an alarmingly large percentage of university psychologists admitted having used …

Author(s): Coosje L. S. Veldkamp, Franca Agnoli, Jelte M. Wicherts, Paolo Albiero, Roberto Cubelli

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Psychology

Tag(s): Analysis, Behavior, Experimental Psychology, Italian People, Metrics, Psychologists, Psychology, Psychometrics, Questionnaires, Reproducibility, Research Methods, United States

Link to resource

Questionable research practices following pre-registration

The credibility of psychological findings can be undermined by a history of questionable research practices (QRPs) by researchers. One …

Author(s): Ayumi Ikeda, Haoqin Xu, Naoto Fuji, Siqi Zhu, Yuki Yamada

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Arts and Humanities, Social Science

Tag(s): Open Science, Preregistration, Transparency, Questionable Research Practices, Embodied Cognition

Link to resource

Questionable research practices in ecology and evolution

We surveyed 807 researchers (494 ecologists and 313 evolutionary biologists) about their use of Questionable Research Practices (QRPs), …

Author(s): Ashley Barnett, Fiona Fidler, Hannah Fraser, Shinichi Nakagawa, Tim Parker

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Life Science, Biology, Ecology

Tag(s): Analysis, Behavioral Ecology, Community Ecology, Data, Ecology and Environmental Sciences, Evolutionary Biology, Evolutionary Ecology, HARKing, Psychology, Publication Ethics, Statistical Data, Statistics

Link to resource

Questionable Research Practices Revisited

The current discussion of questionable research practices (QRPs) is meant to improve the quality of science. It is, however, important …

Author(s): Klaus Fiedler and Norbert Schwarz

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

R for Reproducible Scientific Analysis

This lesson in part of Software Carpentry workshop and teach novice programmers to write modular code and best practices for using R …

Author(s): Adam H. Sparks, Ahsan Ali Khoja, Amy Lee, Ana Costa Conrado, Andrew Boughton, Andrew Lonsdale, Andrew MacDonald, Andris Jankevics, Andy Teucher, Antonio Berlanga-Taylor, Ashwin Srinath, Ben Bolker, Bill Mills, bippuspm, Bret Beheim, butterflyskip, Clare Sloggett, Daniel, Dave Bridges, David J. Harris, David Mawdsley, Dean Attali, Diego Rabatone Oliveira, Drew Tyre, Elise Morrison, Erin Alison Becker, Fernando Mayer, François Michonneau, Giulio Valentino Dalla Riva, Gordon McDonald, Greg Wilson, Harriet Dashnow, Ido Bar, Jaime Ashander, James Balamuta, James Mickley, Jamie McDevitt-Irwin, Jeffrey Arnold, Jeffrey Oliver, John Blischak, Jonah Duckles, Josh Quan, Julia Piaskowski, Kara Woo, Kate Hertweck, Katherine Koziar, Katrin Leinweber, Kellie Ottoboni, Kevin Weitemier, Kiana Ashley West, Kieran Samuk, Kunal Marwaha, Kyriakos Chatzidimitriou, Lachlan Deer, Lex Nederbragt, Liz Ing-Simmons, Lucy Chang, Luke W Johnston, Luke Zappia, Marc Sze, Marie-Helene Burle, Marieke Frassl, Mark Dunning, Martin John Hadley, Mary Donovan, Matt Clark, Melissa Kardish, Mike Jackson, Murray Cadzow, Narayanan Raghupathy, Naupaka Zimmerman, Nelly Sélem, Nicholas Lesniak, Nicholas Potter, Nima Hejazi, Nora Mitchell, Olivia Rata Burge, Paula Andrea Martinez, Pete Bachant, Phil Bouchet, Philipp Boersch-Supan, Piotr Banaszkiewicz, Raniere Silva, Rayna Michelle Harris, Remi Daigle, Rémi Emonet, Research Bazaar, Richard Barnes, Robert Bagchi, Sam Penrose, Sandra Brosda, Sarah Munro, Sasha Lavrentovich, Scott Allen Funkhouser, Scott Ritchie, Sebastien Renaut, Thea Van Rossum, Timothy Eoin Moore, Timothy Rice, Tobin Magle, Trevor Bekolay, Tyler Crawford Kelly, Vicken Hillis, waiteb5, Yuka Takemon

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science, Measurement and Data

Tag(s): Analysis, Data, Education, Open Scholarship Tools and Technologies, R, Reproducibility, Research Data Management Tools, Researchers

Link to resource

R for Social Scientists

Data Carpentry lesson part of the Social Sciences curriculum. This lesson teaches how to analyse and visualise data used by social …

Author(s): Angela Li, Ben Marwick, Christina Maimone, Danielle Quinn, Erin Alison Becker, Francois Michonneau, Geoffrey LaFlair, Hao Ye, Jake Kaupp, Juan Fung, Katrin Leinweber, Martin Olmos, Murray Cadzow

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Information Science, Measurement and Data, Social Science

Tag(s): Analysis, Data, Education, Open Scholarship Tools and Technologies, R, Reproducibility, Research Data Management Tools, Researchers, RStudio

Link to resource

R Graphics Cookbook: Practical Recipes for Visualizing Data

This practical guide provides more than 150 recipes to help you generate high-quality graphs quickly, without having to comb through …

Author(s): Winston Chang

Type of resources: Textbook

Primary user(s): Student, Teacher, Researcher

Subject area(s): Applied Science, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Book, Software, R

Link to resource

R para Análisis Científicos Reproducibles

Una introducción a R utilizando los datos de Gapminder. El objetivo de esta lección es enseñar a las programadoras principiantes a …

Author(s): 0xgc, Alejandra Gonzalez-Beltran, Ana Beatriz Villaseñor Altamirano, Antonio, AntonioJBT, A. s, Belinda Weaver, Claudia Engel, Cynthia Monastirsky, Daniel Beiter, David Mawdsley, David Pérez-Suárez, Erin Becker, EuniceML, François Michonneau, Gordon McDonald, Guillermina Actis, Guillermo Movia, Hely Salgado, Ido Bar, Ivan Ogasawara, Ivonne Lujano, James J Balamuta, Jamie McDevitt-Irwin, Jeff Oliver, Jonah Duckles, Juan M. Barrios, juli arancio, Katrin Leinweber, Kevin Alquicira, Kevin Martínez-Folgar, Laura Angelone, Laura-Gomez, Leticia Vega, Marcela Alfaro Córdoba, Marceline Abadeer, Maria Florencia D'Andrea, Marie-Helene Burle, Marieke Frassl, Matias Andina, Murray Cadzow, Narayanan Raghupathy, Naupaka Zimmerman, Paola Prieto, Paula Andrea Martinez, Raniere Silva, raynamharris, Rayna M Harris, Richard Barnes, Richard McCosh, Romualdo Zayas-Lagunas, Sandra Brosda, Sasha Lavrentovich, saynomoregrl, Shirley Alquicira Hernandez, Silvana Pereyra, Tobin Magle, Veronica Jimenez

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science, Measurement and Data

Tag(s): Analysis, Data, Education, Gapminder, Open Scholarship Tools and Technologies, R, Reproducibility, Research Data Management Tools, Researchers

Link to resource

Race to the Bottom: Competition and Quality in Science

This paper investigates how competition to publish first and thereby establish priority impacts the quality of scientific research. We …

Author(s): Ryan Hill and Caroline Stein

Type of resources: Reading

Primary user(s): Student

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Publish or perish culture, Quality of publication

Link to resource

Raiders of the lost HARK: a reproducible inference framework for big data science

Hypothesizing after the results are known (HARK) has been disparaged as data dredging, and safeguards including hypothesis …

Author(s): Iain E. Buchan, James S. Koopman, Jiang Bian, Matthew Sperrin, Mattia Prosperi, Mo Wang

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Health, Medicine and Nursing

Tag(s): Analysis, Data, HARKing, Reproducibility

Link to resource

Raise standards for preclinical cancer research

C. Glenn Begley and Lee M. Ellis propose how methods, publications and incentives must change if patients are to benefit.

Author(s): C. Glenn Begley and Lee M. Ellis

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Life Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Randomization Does Not Help Much, Comparability Does

According to R.A. Fisher, randomization “relieves the experimenter from the anxiety of considering innumerable causes by which the data …

Author(s): Uwe Saint-Mont

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s):

Link to resource

Rate and success of study replication in ecology and evolution

The recent replication crisis has caused several scientific disciplines to self-reflect on the frequency with which they replicate …

Author(s): Clint D. Kelly

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Biology, Ecology

Tag(s): Reproducibility

Link to resource

Rationalizing pre-analysis plans: Statistical decisions subject to implementability

Pre-analysis plans (PAPs) are a potential remedy to the publication of spurious findings in empirical research, but they have been …

Author(s): Maximilian Kasy, Jann Spiess

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Social Science

Tag(s): Implementability, Pre-Analysis Plans, Statistical Decisions, Economics

Link to resource

Re-run, Repeat, Reproduce, Reuse, Replicate: Transforming Code into Scientific Contributions

Scientific code is different from production software. Scientific code, by producing results that are then analyzed and interpreted, …

Author(s): Fabien C. Y. Benureau, Nicolas P. Rougier

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science, Social Science

Tag(s): Best Practices, Computational Science, Replicability, Reproducibility of Results, Reproducible Research, Reproducible Science, Software Development

Link to resource

Recommendations for Increasing Replicability in Psychology

Replicability of findings is at the heart of any empirical science. The aim of this article is to move the current replicability debate …

Author(s): Jens B. Asendorpf, Mark Conner, Filip De Fruyt, Jan De Houwer, Jaap J. A. Denissen, Klaus Fiedler, Susann Fiedler, David C. Funder, Reinhold Kliegl, Brian A. Nosek, Marco Perugini, Brent W. Roberts, Manfred Schmitt, Marcel A. G. van Aken, Hannelore Weber, Jelte M. Wicherts

Type of resources: Primary Source, Reading, Chapter

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s):

Link to resource

Reconceptualizing replication as a sequence of different studies: A replication typology

In contrast to the truncated view that replications have only a little to offer beyond what is already known, we suggest a broader …

Author(s): Joachim Hüffmeier, Jens Mazei and Thomas Schultze

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Redefine statistical signifcance

We propose to change the default P-value threshold for statistical signifcance from 0.05 to 0.005 for claims of new discoveries.

Author(s): Daniel J. Benjamin

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Reducing bias, increasing transparency and calibrating confidence with preregistration

Flexibility in the design, analysis and interpretation of scientific studies creates a multiplicity of possible research outcomes. …

Author(s): Tom E. Hardwicke & Eric-Jan Wagenmakers

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Science, Technology, Society, Preregistration, Scientific Community

Link to resource

Registered replication report: Hart & Albarracín (2011).

Language can be viewed as a complex set of cues that shape people’s mental representations of situations. For example, people think of …

Author(s): Eerland, A.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Transparency, Open Science, Registered Reports

Link to resource

Registered Replication Report: Rand, Greene, and Nowak (2012).

In an anonymous 4-person economic game, participants contributed more money to a common project (i.e., cooperated) when required to …

Author(s): Bouwmeester, S., Verkoeijen, P. P., Aczel, B., Barbosa, F., Bègue, L., Brañas-Garza, P., ... & Evans, A. M.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Registered replication report: Schooler and engstler-schooler (1990).

Trying to remember something now typically improves your ability to remember it later. However, after watching a video of a simulated …

Author(s): Alogna, V. K et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Transparency, Open Science, Registered Reports

Link to resource

Registered Replication Report: Strack, Martin, & Stepper (1988)

According to the facial feedback hypothesis, people’s affective responses can be influenced by their own facial expression (e.g., …

Author(s): Wagenmakers et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Registered Replication Report: Study 1 From Finkel, Rusbult, Kumashiro, & Hannon (2002).

Finkel, Rusbult, Kumashiro, and Hannon (2002, Study 1) demonstrated a causal link between subjective commitment to a relationship and …

Author(s): Cheung, I., Campbell, L., LeBel, E. P., Ackerman, R. A., Aykutoğlu, B., Bahník, Š., ... & Carcedo, R. J.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Registered Report Protocol: Survey on attitudes and experiences regarding preregistration in psychological research

Background Preregistration, the open science practice of specifying and registering details of a planned study prior to knowing the …

Author(s): Lisa Spitzer, Stefanie Mueller

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Psychological Attitudes, Psychology, Behavior, Surveys, Open Science, Motivation, Pilot Studies, Psychometrics

Link to resource

Registered Reports Factsheet for Editors

Registered Reports Factsheet for Editors

Author(s): Center For Open Science

Type of resources: Reading

Primary user(s):

Subject area(s): Education

Tag(s): Research

Link to resource

Registered Reports FAQ

Registered Reports FAQ

Author(s): Center For Open Science

Type of resources: Reading

Primary user(s):

Subject area(s): Education

Tag(s): Research

Link to resource

Registered Reports Q&A

This webinar addresses questions related to writing, reviewing, editing, or funding a study using the Registered Report format, …

Author(s): Chris Chambers, david mellor

Type of resources: Lesson

Primary user(s):

Subject area(s): Education

Tag(s): Open Science, Peer Review, Publishing, Registered Reports, Research, Research Best Practices

Link to resource

Registered reports: A method to increase the credibility of published results

Registered reports a Method to Increase the Credibility of Published Results

Author(s): Brian Nosek and Daniel Lakens

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Registered Reports: A New Publishing Initiative at Cortex

An article about registered Reports: A new publishing initiative at Cortex

Author(s): Christopher Chambers

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Registered reports: an early example and analysis

The recent ‘replication crisis’ in psychology has focused attention on ways of increasing methodological rigor within the behavioral …

Author(s): Caroline Watt, Diana Kornbrot, Richard Wiseman

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Information Science, Psychology

Tag(s): Data, Preregistation, Publishing, Registered Reports

Link to resource

Reimagining peer review: the emergence of peer community in registered reports system

The traditional peer review process in medicine faces a crisis marked by publication delays, potential bias, and a lack of …

Author(s): Carmine Zoccali & Francesca Mallamaci

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian, researcher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Medical Publishing, Peer Community In Research Reports, Peer Review, Referees; Publication

Link to resource

Releasing a preprint is associated with more attention and citations for the peer-reviewed article

Preprints in biology are becoming more popular, but only a small fraction of the articles published in peer-reviewed journals have …

Author(s): Darwin Y Fu, Jacob J Hughey

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Biology

Tag(s): Preprints, Publishing

Link to resource

Replacing p-values with Bayes-Factors: A Miracle Cure for the Replicability Crisis in Psychological Science

A blog post about choosing Bayes Factor over p-value

Author(s): Ulrich Schimmack

Type of resources: Reading, Student Guide

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Blog

Link to resource

Replicability and reproducibility

A video about replicability and reproducibility debate

Author(s): BPSOfficial/Nick Brown

Type of resources: Video

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Video, Replication Research

Link to resource

Replicability and Reproducibility in Comparative Psychology

Replicability and Reproducibility in Comparative Psychology Psychology faces a replication crisis. The Reproducibility Project: …

Author(s): Jeffrey R. Stevens

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Economics, Psychology

Tag(s): Animal Research, Comparative Psychology, Pre-registration, Replication, Reproducibility, Reproducible Research

Link to resource

Replicability syllabus (2018)

A syllabus about replicability seminar

Author(s): Simine Vazire

Type of resources: Syllabus

Primary user(s): Student, Teacher

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Replicability Syllabus (2020)

This course will examine current controversies and new developments in research methods in psychology. The goal of the course is to …

Author(s): Simine Vazire

Type of resources: Syllabus

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Replicating Studies in Which Samples of Participants Respond to Samples of Stimuli

In a direct replication, the typical goal is to reproduce a prior experimental result with a new but comparable sample of participants …

Author(s): Jacob Westfall, Charles M. Judd, David A. Kenny

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution

Link to resource

Replication and Bias in (Special) Education Research Base

This is a clearinghouse for resources related to open science in Special Education. If you find a good resource that has not been …

Author(s): OSKB Admin

Type of resources: Reading

Primary user(s):

Subject area(s): education

Tag(s): Open Practices, Replication, Research, Special Education

Link to resource

Replication and Open Science for Undergraduates

Blog post going over the replication crisis and how it has led to the open science movement.

Author(s): David Funder

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Life Science, Math & Statistics, Social Science

Tag(s): Blog, Open Science, Reproducibility Crisis and Credibility Revolution

Link to resource

Replication and Robustness in Developmental Research

Replications and robustness checks are key elements of the scientific method and a staple in many disciplines. However, leading …

Author(s): Greg J Duncan, Mimi Engel, Amy Claessens, Chantelle J Dowsett

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Replication and the Manufacture of Scientific Inferences: A Formal Approach

The field of replication studies remains a controversial, misunderstood, and unappreciated piñata of 18 replication typologies spanning …

Author(s): Fernando Martel García

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Replication in Psychology: A historical perspective

The reproducibility of psychological findings has generated much discussion of late. However, the question of replication is not a new …

Author(s): Michael Pettit

Type of resources: Interactive

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Blog, Reproducibility Knowledge

Link to resource

Replication is Relevant in Qualitative Research

Replication has received increasing attention over the last decade. This comes on the heels of prominent instances of data fabrication …

Author(s): Matthew C. Makel, Melanie S. Meyer, Mary A. Simonsen, Anne M. Roberts, Jonathan A. Plucker

Type of resources: Reading

Primary user(s): Student, Parent, Librarian

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Social Science

Tag(s): Replication, Qualitative Research

Link to resource

Replication, Communication, and the Population Dynamics of Scientific Discovery

Many published research results are false (Ioannidis, 2005), and controversy continues over the roles of replication and publication …

Author(s): Richard McElreath, Paul E. Smaldino

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Replications in Psychology Research: How Often Do They Really Occur?

Recent controversies in psychology have spurred conversations about the nature and quality of psychological research. One topic …

Author(s): Matthew C. Makel, Jonathan A. Plucker, and Boyd Hegarty

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Repligate: Reliability and Reproducibility in Psychology Syllabus

This seminar will address issues relevant to the current controversy over the reliability of psychological research, a controversy …

Author(s): Richard Ball, Sara Bowman, Garret Christensen, Michael C. Frank, David Funder, Konrad Hinsen, Nicole Janz, Norm Medeiros, Edward Miguel, Brian A. Nosek, Russell Poldrack, Simine Vazire, Felix Schönbrodt, Eric-Jan Wagenmakers, Lorne Campbell, Eric Luis Uhlmann, Martin Schweinsberg, Neil Bearden, John M. Zelenski, Daniel J. Simons, Gustav Nilsonne, Michael J. Kane, Joseph Hilgard, Jennifer Howell, Michèle B. Nuijten, , Rodica I. Damian, Sean Mackinnon, Henrik Danielsson, Brent Roberts, Heather L. Urry, Katherine S. Corker, Edouard Machery, Caio Maximino, David Thomas Mellor, Jon Grahe, Timothy H. Parker, Don A. Moore, Flavio Azevedo, Charlotte Hartwright, Benjamin Le, Julia Strand, Maia Salholz-Hillel, Amy Orben, Moin Syed, Ana P. Herrmann, Argiro Vatakis, Rima-Maria Rahal, Simon Schwab, Lisa DeBruine, Daniel Umpierre, Krista Byers-Heinlein, Melanie Soderstrom, and Nate Breznau

Type of resources: Syllabus

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Reply to Ledgerwood: Predictions without analysis plans are inert

Ledgerwood (1) argues that there are two independent uses of preregistration that are conflated in Nosek et al. (2) and elsewhere: …

Author(s): Brian A. Nosek, Charles R. Ebersole, Alexander C. DeHaven, David T. Mellor

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Preregistration, Reply, Prediction, Analyses

Link to resource

Reply to Uri Simonsohn’s Critique of Default Bayesian Tests

A blog that summarises a reply to Uri Simonsohn’s Critique of Default Bayesian Tests

Author(s): Jeff Rouder

Type of resources: Reading

Primary user(s): Student, Teacher, Researcher

Subject area(s): Life Science, Math & Statistics, Social Science

Tag(s): Blog

Link to resource

Reporting Effect Sizes in Original Psychological Research: A Discussion and Tutorial

Statistical practice in psychological science is undergoing reform which is reflected in part by strong recommendations for reporting …

Author(s): Jolynn Pek and David B Flora

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Reporting in Experimental Philosophy: Current Standards and Recommendations for Future Practice

Recent replication crises in psychology and other fields have led to intense reflection about the validity of common research …

Author(s): Andrea Polonioli, Brittany Blankinship, David Carmel, Mariana Vega-Mendoza

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Psychology

Tag(s): Reproducibility

Link to resource

Repository of Psychological Instruments in Serbian (REPOPSI)

An Open-Access repository of psychological measures, scales, tests, and other research instruments in Serbian. Documented are …

Author(s): Laboratory for Research of Individual Differences (LIRA) at the University of Belgrade

Type of resources: Research Material

Primary user(s): Student, Teacher, Researcher

Subject area(s): Applied Science, Social Science

Tag(s): Digital Repository, Research Data, Psychological Measures, Open Repository, Scales, Translation, Adaptation

Link to resource

ReproducibiliTea

Serving mugs of Reproducibili☕️: Blends include transparency, openness and robustness + a spoonful of science.

Author(s): Sophia Cruwell, Amy Orben and Sam Parsons

Type of resources: Student Guide, Unit of Study

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Podcast, Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Reproducibility for Everyone

Reproducibility for Everyone produces resources to help grow researchers' awareness of and ability to do reproducible research.

Author(s): April Clyburne-Sherin

Type of resources: Lesson

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Reproducibility, Research Data Management Tools, Researchers

Link to resource

Reproducibility Immersive Course

Various fields in the natural and social sciences face a ‘crisis of confidence’. Broadly, this crisis amounts to a pervasiveness of …

Author(s): Vicky Steeves

Type of resources: Activity/Lab

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Analysis, Git, GitHub, GitLab, Jupyter Notebooks, Open Scholarship Tools and Technologies, Open Science Framework, R, Reproducibility, ReproZip, Research Data Management Tools, Researchers, RStudio, Version Control

Link to resource

Reproducibility in Cancer Biology: The challenges of replication

Interpreting the first results from the Reproducibility Project: Cancer Biology requires a highly nuanced approach. Reproducibility is …

Author(s): eLife Editors

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Health, Medicine and Nursing

Tag(s): Metascience, Methodology, Open Science, Replication, Reproducibility, Reproducibility Project: Cancer Biology

Link to resource

Reproducibility in Psychology Syllabus

Psychological science has been going through a crisis of confidence concerning theveracity of the findings that serve as the foundation …

Author(s): Brent Roberts and Dan Simons

Type of resources: Syllabus

Primary user(s): Teacher

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution; Open Science; Conceptual and statistical knowledge

Link to resource

Reproducibility in the Psychological Sciences

Science is undergoing a paradigm shift in the way research is conducted and what is considered evidence. Psychology is leading other …

Author(s): Jon Grahe

Type of resources: Syllabus

Primary user(s): Teacher

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Reproducibility Librarianship in Practice

As research across domains of study has become increasingly reliant on digital tools (librarianship included), the challenges in …

Author(s): Birgit Schmidt, Vicky Steeves

Type of resources: Lesson

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Institutional Policies, Librarians, Librarianship, Reproducibility

Link to resource

Reproducibility, Preservation, and Access to Research with ReproZip and ReproServer

The adoption of reproducibility remains low, despite incentives becoming increasingly common in different domains, conferences, and …

Author(s): Fernando Chirigati, Rémi Rampin, Vicky Steeves

Type of resources: Activity/Lab

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Open Scholarship Tools and Technologies, Reproducibility, Research Data Management Tools, Researchers

Link to resource

Reproducibility/Open Science and Clinical Psych Reading List

A reading list for reproducibility in clinical psychology

Author(s): Cody Christopherson

Type of resources: Reading

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Clinical psychology

Link to resource

Reproducible and reusable research: are journal data sharing policies meeting the mark?

Background There is wide agreement in the biomedical research community that research data sharing is a primary ingredient for ensuring …

Author(s): Jessica Minnier, Melissa A. Haendel, Nicole A. Vasilevsky, Robin E. Champieux

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Health, Medicine and Nursing, Biology

Tag(s): Data, Inside Your Classroom, Reproducibility

Link to resource

Reproducible and transparent research practices in published neurology research

The objective of this study was to evaluate the nature and extent of reproducible and transparent research practices in neurology …

Author(s): Austin L. Johnson, Daniel Tritz, Jonathan Pollard, Matt Vassar, Shelby Rauh, Trevor Torgerson

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Health, Medicine and Nursing, Biology, Social Science

Tag(s): Data, Inside Your Classroom, Reproducibility

Link to resource

Reproducible Research

Modern scientific research takes advantage of programs such as Python and R that are open source. As such, they can be modified and …

Author(s): Christopher Ahern

Type of resources: Full Course

Primary user(s): student, teacher

Subject area(s): Information Science

Tag(s): Analysis, Literate Programming, Reproducibility, Version Control

Link to resource

Reproducible research in sport and exercise psychology: The role of sample sizes.

Objectives: We aim to introduce the discussion on the crisis of confidence to sport and exercise psychology. We focus on an important …

Author(s): Schweizer, G., & Furley, P.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Social Science

Tag(s):

Link to resource

Reproducible Research Methods

This is the website for the Autumn 2014 course “Reproducible Research Methods” taught by Eric C. Anderson at NOAA’s Southwest Fisheries …

Author(s): Eric C. Anderson

Type of resources: Full Course

Primary user(s): student, teacher

Subject area(s): Information Science

Tag(s): Analysis, Literate Programming, Reproducibility, Version Control

Link to resource

Reproducible research practices, transparency, and open access data in the biomedical literature, 2015–2017

Currently, there is a growing interest in ensuring the transparency and reproducibility of the published scientific literature. …

Author(s): John P. A. Ioannidis, Joshua D. Wallach, Kevin W. Boyack

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Biology

Tag(s): Conflicts of Interest, Data, Medical Journals, Open Access Publishing, Open Science, PublishingGovernment Funding of Science, Replication Studies, Reproducibility, Scientific Publishing, Sequence Databases, Systematic Reviews

Link to resource

Reproducible Research: True or False?

A video detailing whether reproducible research is true or false

Author(s): John Ioannidis

Type of resources: Student Guide, Teaching/Learning Strategy, Video

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Video

Link to resource

Reproducible Research: Walking the Walk

Description

This hands-on tutorial will train reproducible research warriors on the practices and tools that make experimental …

Author(s): Matt McCormick

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Information Science

Tag(s): Analysis, Organizing, Report Writing, Reproducibility, Version Control

Link to resource

Reproducible Science Curriculum Lesson for Automation

Workshop goals

  • Why are we teaching this
  • Why is this important
  • For future and current you
  • For research as a whole
  • Lack of …

Author(s): François Michonneau, Kim Gilbert, Matt Pennell

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Information Science

Tag(s): Analysis, Report Writing, Reproducibility

Link to resource

Reproducible Science Curriculum Lesson for Literate Programming

Workshop goals

  • Why are we teaching this
  • Why is this important
  • For future and current you
  • For research as a whole
  • Lack of …

Author(s): Ciera Martinez, Courtney Soderberg, Hilmar Lapp, Jennifer Bryan, Kristina Riemer, Naupaka Zimmerman

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Information Science

Tag(s): Literate Programming, Reproducibility

Link to resource

Reproducible Science Curriculum Lesson for Organization

Workshop goals

  • Why are we teaching this
  • Why is this important
  • For future and current you
  • For research as a whole
  • Lack of …

Author(s): Ciera Martinez, Courtney Soderberg, Hilmar Lapp, Jennifer Bryan, Kristina Riemer, Naupaka Zimmerman

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Information Science

Tag(s): Organizing, Reproducibility

Link to resource

Reproducible Science Curriculum Lesson for Publication

Workshop goals

  • Why are we teaching this
  • Why is this important
  • For future and current you
  • For research as a whole
  • Lack of …

Author(s): Dave Clements, Hilmar Lapp, Karen Cranston

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Information Science

Tag(s): Report Writing, Reproducibility

Link to resource

Reproducible Science Curriculum Lesson for Version Control

Workshop goals

  • Why are we teaching this
  • Why is this important
  • For future and current you
  • For research as a whole
  • Lack of …

Author(s): Ciera Martinez, Hilmar Lapp, Karen Cranston

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Information Science

Tag(s): Reproducibility, Version Control

Link to resource

Reproducible Science Workshop

Workshop goals

  • Why are we teaching this
  • Why is this important
  • For future and current you
  • For research as a whole
  • Lack of …

Author(s): Dan Leehr

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Information Science

Tag(s): Documentation, Organizing, Reproducibility

Link to resource

Reproducible statistics for psychologists with R Lab Tutorials

This is a series of labs/tutorials currently under development (2020-2021) for a two-semester graduate-level statistics sequence in …

Author(s): Matthew C.Crump

Type of resources: Reading, Simulation, Textbook

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Statistics

Link to resource

Research Methods 2020

Broad-based philosophical and methodological perspectives on conducting and interpreting psychological research; considers basic, …

Author(s): Michael Kane

Type of resources: Syllabus

Primary user(s): Teacher

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Research Methods in Psychology

Broad-based philosophical and methodological perspectives on conducting and interpreting psychological research; considers basic, …

Author(s): Michael Kane

Type of resources: Syllabus

Primary user(s): Student, Teacher

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Research Methods in Social Psychology

This course will acquaint students with the major research designs and procedures in social psychology, as well as explore recent …

Author(s): Lorne Campbell

Type of resources: Syllabus

Primary user(s): Teacher

Subject area(s): Math & Statistics, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Research Methods Syllabi

A syllabus about evaluating research methods

Author(s): Morton-Ann Gernsbacher

Type of resources: Syllabus

Primary user(s): Teacher

Subject area(s): Applied Science, Education, Social Science

Tag(s): Syllabus

Link to resource

Research practices and statistical reporting quality in 250 economic psychology master’s theses: a meta-research investigation

The replicability of research findings has recently been disputed across multiple scientific disciplines. In constructive reaction, the …

Author(s): Erich Kirchler, Jerome Olsen, Johanna Mosen, Martin Voracek

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Psychology

Tag(s): Reproducibility

Link to resource

Research practices for a robust psychological science of adult development and aging

This first issue of 2022 marks the transition of Psychology and Aging in adopting a transparency and openness promotion (TOP) …

Author(s): Elizabeth A. L. Stine-Morrow

Type of resources: Reading

Primary user(s): Teacher, Librarian

Subject area(s): Life Science, Social Science

Tag(s): TOP Factor

Link to resource

Research Practices That Can Prevent an Inflation of False-Positive Rates

Recent studies have indicated that research practices in psychology may be susceptible to factors that increase false-positive rates, …

Author(s): Kou Murayama, Reinhard Pekrun and Klaus Fiedler

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Research preregistration 101

A blog about pre-registration

Author(s): Lindsay, D.S., Simons, D.J., Lilienfeld, S.O.

Type of resources: Blog

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Blog

Link to resource

Research Preregistration as a Teaching and Learning Tool in Undergraduate Psychology Courses

The preregistration of research plans and hypotheses may prevent publication bias and questionable research practices. We incorporated …

Author(s): Sarai Blincoe, Stephanie Buchert

Type of resources: Lesson Plan, Teaching/Learning Strategy

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Preregistration, Questionable Research Practices, Undergraduate Preregistration Assignment

Link to resource

Research Preregistration in Political Science: The Case, Counterarguments, and a Response to Critiques

This article describes the current debate on the practice of preregistration in political science—that is, publicly releasing a …

Author(s): James E. Monogan III

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Social Science

Tag(s): Political Science, Preregistration

Link to resource

Research project initialization and organization following reproducible research guidelines

Workshop goals

  • Why are we teaching this
  • Why is this important
  • For future and current you
  • For research as a whole
  • Lack of …

Author(s): Hilmar Lapp

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Information Science

Tag(s): Organizing, Reproducibility

Link to resource

Research Project Management Using the Open Science Framework

An introduction to managing, annotating, organizing, archiving, and publishing research data using the Open Science Framework.

Author(s): Nick Wolf, Vicky Steeves

Type of resources: Activity/Lab

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Analysis, Materials, Open Scholarship Tools and Technologies, Research Data Management Tools, Researchers

Link to resource

Researchers’ Intuitions About Power in Psychological Research

Many psychology studies are statistically underpowered. In part, this may be because many researchers rely on intuition, rules of …

Author(s): Bakker et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s):

Link to resource

Resolving the Tension Between Exploration and Confirmation in Preclinical Biomedical Research

Confirmation through competent replication is a founding principle of modern science. However, biomedical researchers are rewarded for …

Author(s): Ulrich Dirnagl

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Health, Medicine and Nursing

Tag(s): False Negative, False Positive, Preclinical Randomized Controlled Trial, Replication, Reproducibility, Statistics

Link to resource

Resources (tutorials, papers, analysis scripts, utilities) for testing moderation and mediation

A website about moderation and mediation

Author(s): Kristopher J. Preacher

Type of resources: Lesson

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Website

Link to resource

Resources for Practicing Open Science with Qualitative Research in Education

This list of resources consists of resources for researchers, editors, and reviewers interested in practicing open science principles, …

Author(s): Rachel Renbarger, Crystal N. Steltenpohl

Type of resources: Reading, Student Guide

Primary user(s):

Subject area(s): education, social-science, political-science, psychology

Tag(s): Data Sharing, Education Research, Methodology, Open Science, Open Science Training, Posititionality, Qualitative, Research, Transparency

Link to resource

Response to Comment on “Estimating the reproducibility of psychological science”

Gilbert et al. conclude that evidence from the Open Science Collaboration’s Reproducibility Project: Psychology indicates high …

Author(s): Christopher J. Anderson et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Rethinking Research Assessment:Addressing Institutional Biases in Review, Promotion, and Tenure Decision-Making (part IV)

In our final installment we get into how incumbent processes and perceptions have the advantage.

Author(s): Ruth Schmidt

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Diversity, Equity and Inclusion

Link to resource

Rethinking transparency and rigor from a qualitative open science perspective

Discussions around transparency in open science focus primarily on sharing data, materials, and coding schemes, especially as these …

Author(s): Crystal N. Steltenpohl, Hilary Lustick, Melanie S. Meyer, Lindsay Ellis Lee, Sondra M. Stegenga, Laurel Standiford Reyes, Rachel L. Renbarger

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Open Science, Transparency, Rigor, Qualitative, Quantitative

Link to resource

Retraction Watch: Michael LaCour archives

How easy is it to change people’s minds? In 2014, a Science study suggested that a short conversation could have a lasting impact on …

Author(s): Alison McCook

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Education, Math & Statistics, Physical Science, Social Science

Tag(s): Blog

Link to resource

Review of Four Preregistration Registries for Special Education Researchers

Preregistration involves researchers publicly registering key study elements before conducting a study to increase the transparency of …

Author(s): Jesse I. Fleming, Alan S. McLucas, Bryan G. Cook

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Change, Innovation, Legal Issues, Policy Issues, Research Methodology

Link to resource

Reviewer Bias Against Replication Research

Social science journal reviewers (N=8) responded to questionnaires regarding their reviewing history, and attitudes towards and …

Author(s): Neuliep, J W; Crandall, R.

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Rewarding Replications: A Sure and Simple Way to Improve Psychological Science

Although replications are vital to scientific progress, psychologists rarely engage in systematic replication efforts. In this article, …

Author(s): Sander L Koole, Daniël Lakens

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Richard McElreath

A series of youtube videos about his book on statistical rethinking and Bayesian statistics

Author(s): Richard McElreath

Type of resources: Videos

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Bayesian statistics

Link to resource

Rigor and Reproducibility

The information provided on this website is designed to assist the extramural community in addressing rigor and transparency in NIH …

Author(s): NIH

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Health, Medicine and Nursing

Tag(s): Policy, Reproducibility

Link to resource

Rigor Champions and Resources

Efforts to Instill the Fundamental Principles of Rigorous ResearchRigorous experimental procedures and transparent reporting of …

Author(s): National Institutes of Health

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Reproducibility, Research Administration, Research Integrity, Rigor

Link to resource

Rigorous & Reproducible Research Practices

How do you know whether the quantitative research you’re consuming and producing is rigorous and reproducible? This course will draw on …

Author(s): Heather Urry

Type of resources: Syllabus

Primary user(s): Teacher

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science, Ethics

Link to resource

RIOT science club

A videos that discusses world-leading open science

Author(s): RIOT

Type of resources: Video

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Social Science

Tag(s): Open science

Link to resource

RIPOSTE: A Framework for Improving the Design and Analysis of Laboratory-Based Research

Lack of reproducibility is an ongoing problem in some areas of the biomedical sciences. Poor experimental design and a failure to …

Author(s): Masca et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Risk of Bias in Reports of In Vivo Research: A Focus for Improvement

The reliability of experimental findings depends on the rigour of experimental design. Here we show limited reporting of measures to …

Author(s): Aaron Lawson McLean, Aikaterini Kyriakopoulou, Andrew Thomson, Aparna Potluru, Arno de Wilde, Cristina Nunes-Fonseca, David W. Howells, Emily S. Sena, Gillian L. Currie, Hanna Vesterinen, Julija Baginskitae, Kieren Egan, Leonid Churilov, Malcolm R. Macleod, Nicki Sherratt, Rachel Hemblade, Stylianos Serghiou, Theo Hirst, Zsanett Bahor

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Biology

Tag(s): Bibliometrics, Experimental Design, Publishing, Research Assessment, Research Quality Assessment, Research Validity, Scientific Publishing, Scientists, Systematic Reviews

Link to resource

Robust Modeling in Cognitive Science

In an attempt to increase the reliability of empirical findings, psychological scientists have recently proposed a number of changes in …

Author(s): Michael D. Lee, Amy H. Criss, Berna Devezer, Christopher Donkin, Alexander Etz, Fábio P. Leite, Dora Matzke, Jeffrey N. Rouder, Jennifer S. Trueblood, Corey N. White & Joachim Vandekerckhove

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Robustness, Cognitive Modeling, Open Science, Reproducibility

Link to resource

RPM: An open-source Rotation Platform for open- and closed-loop vestibular stimulation in head-fixed Mice

Head fixation allows the recording and presentation of controlled stimuli and is used to study neural processes underlying spatial …

Author(s): Xavier Cano-Ferrer, Alexandra Tran-Van-Minh, Ede Rancz

Type of resources: Interactive

Primary user(s): Student, Teacher

Subject area(s): Life Science, Social Science

Tag(s): Vestibular, Virtual Reality, Navigation, Sensory Processing, Sensorimotor Processing, Eye Movements, Open Science, Open Hardware

Link to resource

RStudio Cheatsheets

RStudio Cheatsheets

The cheatsheets below make it easy to use some of our favorite packages. Cheatsheets include the following topics: …

Author(s): RStudio

Type of resources: Student Guide

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Inside Your Classroom, Open Scholarship Tools and Technologies, R, Reproducibility, Researchers, RStudio, Workflow Tools

Link to resource

Safeguard Power as a Protection Against Imprecise Power Estimates

An essential first step in planning a confirmatory or a replication study is to determine the sample size necessary to draw …

Author(s): Marco Perugini, Marcello Gallucci, and Giulio Costantini

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Sailing From the Seas of Chaos Into the Corridor of Stability: Practical Recommendations to Increase the Informational Value of Studies

Recent events have led psychologists to acknowledge that the inherent uncertainty encapsulated in an inductive science is amplified by …

Author(s): Daniel Lakens and Ellen R K Evers

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Sample Size in Psychological Research Over the Past 30 Years

The American Psychological Association (APA) Task Force on Statistical Inference was formed in 1996 in response to a growing body of …

Author(s): Jacob M Marszalek, Carolyn Barber, Julie Kohlhart, Cooper B Holmes

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Sample Size Justification Shiny App

This Shiny app accompanies the paper ‘Sample Size Justification’ by Daniël Lakens. You can download the pre-print of this …

Author(s): Daniël Lakens

Type of resources: Interactive

Primary user(s): Student, Teacher, researcher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Social Science

Tag(s): Sample Size Justification, Study Planning

Link to resource

Sample Size Planning for Statistical Power and Accuracy in Parameter Estimation

This review examines recent advances in sample size planning, not only from the perspective of an individual researcher, but also with …

Author(s): Scott E. Maxwell, Ken Kelley and Joseph R. Rausch

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Sample Size Planning for the Standardized Mean Difference: Accuracy in Parameter Estimation via Narrow Confidence Intervals

Methods for planning sample size (SS) for the standardized mean difference so that a narrow confidence interval (CI) can be obtained …

Author(s): Ken Kelley 1, Joseph R Rausch

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics, Social Science

Tag(s):

Link to resource

Sample-size planning for more accurate statistical power: A method adjusting sample effect sizes for publication bias and uncertainty.

The sample size necessary to obtain a desired level of statistical power depends in part on the population value of the effect size, …

Author(s): Anderson, S. F., Kelley, K., & Maxwell, S. E.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Scanning the horizon: towards transparent and reproducible neuroimaging research

Functional neuroimaging techniques have transformed our ability to probe the neurobiological basis of behaviour and are increasingly …

Author(s): Russell A. Poldrack, Chris I. Baker, Joke Durnez, Krzysztof J. Gorgolewski, Paul M. Matthews, Marcus R. Munafò, Thomas E. Nichols, Jean-Baptiste Poline, Edward Vul & Tal Yarkoni

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Life Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Science and Pseudoscience BBC Radio Talk

A podcast about science and pseudoscience

Author(s): London School of Economics and Political Science

Type of resources: Student Guide, Teaching/Learning Strategy, Podcast

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Podcast, Reproducibility Crisis and Credibility Revolution

Link to resource

Science isn’t Broken

If you follow the headlines, your confidence in science may have taken a hit lately. Peer review? More like self-review. An …

Author(s): Christie Aschwanden

Type of resources: Interactive, Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Metascience, Scientific Method, P-values, Science, Health

Link to resource

Science Literacy

Fake news or good science? In a world where we have access to unlimited information, it is hard to sift through the echo chamber of …

Author(s): University of Alberta

Type of resources: Full Course

Primary user(s): Student, Parent, Laypeople

Subject area(s): Applied Science, Social Science

Tag(s): Scientific communication

Link to resource

Science or art? How aesthetic standards grease the way through the publication bottleneck but undermine science.

The current crisis in psychological research involves issues of fraud, replication, publication bias, and false positive results. I …

Author(s): Giner-Sorolla, R.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Scientific apophenia in strategic management research: Significance tests & mistaken inference

This article uses distributional matching and posterior predictive checks to estimate the extent of false and inflated findings in …

Author(s): BRENT GOLDFARB and ANDREW A. KING

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Scientific Utopia II. Restructuring incentives and practices to promote truth over publishability.

An academic scientist’s professional success depends on publishing. Publishing norms emphasize novel, positive results. As such, …

Author(s): Nosek, B. A., Spies, J. R., & Motyl, M.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Scientific Utopia: I. Opening Scientific Communication

Existing norms for scientific communication are rooted in anachronistic practices of bygone eras making them needlessly inefficient. We …

Author(s): Brian A. Nosek & Yoav Bar-Anan

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Scientific Utopia: II. Restructuring Incentives and Practices to Promote Truth Over Publishability

An academic scientist’s professional success depends on publishing. Publishing norms emphasize novel, positive results. As such, …

Author(s): Brian A. Nosek, Jeffrey R. Spies, Matt Motyl

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Psychology

Tag(s): Policy, Reproducibility

Link to resource

Scientists’ Reputations Are Based on Getting It Right, Not Being Right

Replication is vital for increasing precision and accuracy of scientific claims. However, when replications “succeed” or “fail,” they …

Author(s): Charles R. Ebersole, Jordan R. Axt, Brian A. Nosek

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Secondary Data Preregistration

Preregistration is the process of specifying project details, such as hypotheses, data collection procedures, and analytical decisions, …

Author(s): Alexander C. DeHaven, Andrew Hall, Brian Brown, Charles R. Ebersole, Courtney K. Soderberg, David Thomas Mellor, Elliott Kruse, Jerome Olsen, Jessica Kosie, K. D. Valentine, Lorne Campbell, Marjan Bakker, Olmo van den Akker, Pamela Davis-Kean, Rodica I. Damian, Sara J. Weston, Stuart J. Ritchie, Thuy-vy Ngugen, William J. Chopik

Type of resources: Reading

Primary user(s):

Subject area(s): Applied Science

Tag(s): Aging Science, Open Science, Secondary Data Preregistration

Link to resource

Shall we really do it again? The powerful concept of replication is neglected in the social sciences

Replication is one of the most important tools for the verification of facts within the empirical sciences. A detailed examination of …

Author(s): Stefan Schmidt

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Sharing Detailed Research Data Is Associated with Increased Citation Rate

Background Sharing research data provides benefit to the general scientific community, but the benefit is less obvious for the …

Author(s): Douglas B. Fridsma, Heather A. Piwowar, Roger S. Day

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Health, Medicine and Nursing

Tag(s): Bibliometrics, Cancers and Neoplasms, Citation Analysis, Clinical Trials (cancer Treatment), Data, Internet, Linear Regression Analysis, Microarrays, Open Data, Scientific Publishing

Link to resource

Short R script to plot effect sizes (Cohen’s d) and share overlapping are

This blog describes how to plot effect sizes (Cohen’s d) and shade overlapping area with R scripts

Author(s): Kristoffer Magnusson

Type of resources: Reading, R code

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Blog, Tutorial

Link to resource

Short, Sweet, and Problematic? The Rise of the Short Report in Psychological Science

Our field has witnessed a rapid increase in the appeal and prevalence of the short report format over the last two decades. In this …

Author(s): Alison Ledgerwood, Jeffrey W. Sherman

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Should Preregistration of Epidemiologic Study Protocols Become Compulsory? Reflections and a Counterproposal

There is an ongoing debate regarding preregistration of epidemiologic study protocols. We examine the basic idea that preregistration …

Author(s): Timothy L. Lash, Jan P. Vandenbroucke

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Life Science

Tag(s): Epidemiology, Preregistration

Link to resource

Signaling the trustworthiness of science

Trust in science increases when scientists and the outlets certifying their work honor science’s norms. Scientists often fail to signal …

Author(s): Kathleen Hall Jamieson, Marcia McNutt, Richard Sever, Veronique Kiermer

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Life Science, Social Science

Tag(s): OSKB

Link to resource

Significance

Visual comics about significance testing

Author(s): Randall Munroe

Type of resources: Diagram/Illustration

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Life Science, Math & Statistics, Social Science

Tag(s): Comic

Link to resource

Significance tests have their place

Null-hypothesis significance tests (NHST), properly used, tell us whether we have sufficient evidence to be confident of the sign of …

Author(s): Richard J. Harris

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Small telescopes Detectability and the evaluation of replication results.

This article introduces a new approach for evaluating replication results. It combines effect-size estimation with hypothesis testing, …

Author(s): Uri Simonsohn

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

SnapShot: Reporting practices for publishing results with human PSCs and tissue stem cells

This checklist is intended to help scientists, reviewers, and editors prepare and assess manuscripts for inclusion of critical details …

Author(s): ISSCR Task Force for Basic Research Standards

Type of resources: Primary Source, Reading

Primary user(s): Student, Teacher, Librarian, Researcher

Subject area(s): Life Science

Tag(s): Pluripotent Stem Cells, Tissue Stem Cells, Metadata, Culture, Genomic Characterization, Molecular Characterization, Experiment, Data

Link to resource

Social Identity and morality lab teaching

I believe that higher education should be focused on developing passionate, critical, independent and creative thought, and the …

Author(s): Jay Van Bavel

Type of resources: Module, Reading

Primary user(s): Teacher

Subject area(s): Social Science

Tag(s): Materials; Mentoring

Link to resource

Social Science Workshop Overview

Workshop overview for the Data Carpentry Social Sciences curriculum. Data Carpentry’s aim is to teach researchers basic concepts, …

Author(s): Angela Li, Erin Alison Becker, Francois Michonneau, Maneesha Sane, Sarah Brown, Tracy Teal

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Information Science, Measurement and Data, Social Science

Tag(s): Analysis, Data, Education, Open Scholarship Tools and Technologies, Python, R, Reproducibility, Research Data Management Tools, Researchers, SQL

Link to resource

Social Sciences Research Methods Centre Replication Workshop

This workshop will introduce students to the process of reproducing published work. Replicating other scholars’ work is an essential …

Author(s): Nicole Janz

Type of resources: Syllabus

Primary user(s): Teacher

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Social, Behavioral, and Economic Sciences Perspectives on Robust and Reliable Science

A paper about social, behavioural economic perspective on reproducible science

Author(s): Cacioppo, J. T., Kaplan, R. M., Krosnick, J. A., Olds, J. L., & Dean, H.

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Software Carpentry

Since 1998, Software Carpentry has been teaching researchers the computing skills they need to get more done in less time and with less …

Author(s): Software Carpentry Community

Type of resources: Full Course

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Analysis, Open Source Software, Reproducibility, Researchers, Version Control, Workflow Tools

Link to resource

Sources of nonreplicability in aging ethnoracial health disparities research

The older adult population in the U.S. is becoming increasingly diverse across a constellation of factors including ethnoracial group, …

Author(s): Ian M.McDonough, Erin R. Harrell, Sheila R. Black, Rebecca S. Allen, Patricia A. Parmelee

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Life Science, Social Science

Tag(s): Health Disparities, Nonreplicability, Replicability, Replication, Subgroup Analyses, Aging, Biases

Link to resource

SPARC Popular Resources

SPARC is a global coalition committed to making Open the default for research and education. SPARC empowers people to solve big …

Author(s): Nick Shockey

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Funders, Governmental Policies, Librarians, Open Access, Publishers, Publishing, Researchers

Link to resource

Special Issue: Innovation in Aging

Special Issue: Innovation in Aging

Author(s): OSKB Admin

Type of resources: Reading

Primary user(s):

Subject area(s): life-science

Tag(s): Aging Science

Link to resource

Special Issue: Preregistered Studies of Personality Development and Aging Using Existing Data

SPECIAL ISSUE: PREREGISTERED STUDIES OF PERSONALITY DEVELOPMENT AND AGING USING EXISTING DATA (Volume 76, Issue 1, January 2021)

Author(s): OSKB Admin

Type of resources: Reading

Primary user(s):

Subject area(s): life-science

Tag(s): Aging Science, Research

Link to resource

SPSP experts - open science

A video about open science, pre-registration etc.

Author(s): Society for Social and Personality Psychology

Type of resources: Teaching/Learning Strategy, Video

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Video, Reproducibility Knowledge

Link to resource

Spurious Correlations

A website detailing spurious correlations

Author(s): Tyler Vigen

Type of resources: Teaching/Learning Strategy

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Education, Math & Statistics, Social Science

Tag(s): Website

Link to resource

Standard Operating Procedures: A Safety Net for Pre-Analysis Plans

Across the social sciences, growing concerns about research transparency have led to calls for pre-analysis plans (PAPs) that specify …

Author(s): Winston Lin and Donald P.Green

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Star wars: The empirics strike back

Using 50,000 tests published in the AER, JPE, and QJE, we identify a residual in the distribution of tests that cannot be explained …

Author(s): Sangnier, M., & Zylberberg, Y.

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Stat 545

This site is about everything that comes up during data analysis except for statistical modelling and inference. This might strike you …

Author(s): Jenny Bryan

Type of resources: Reading, Student Guide, Tutorial

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Tutorial

Link to resource

Statcheck

statcheck is a program that checks for errors in statistical reporting in APA-formatted documents. It was originally written in the R …

Author(s): Michele B. Nuijten, Sacha Epskamp

Type of resources: Activity/Lab

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Reproducibility, Researchers, Software, Workflow Tools

Link to resource

Statistical errors: P values, the ‘gold standard’ of statistical validity, are not as reliable as many scientists assume

P values, the ‘gold standard’ of statistical validity, are not as reliable as many scientists assume.

Author(s): Nuzzo, R.

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Statistical Evidence: A Likelihood Paradigm

Interpreting statistical data as evidence, Statistical Evidence: A Likelihood Paradigm focuses on the law of likelihood, fundamental to …

Author(s): Richard Royall

Type of resources: Textbook

Primary user(s): Student, Teacher, Researcher

Subject area(s): Math & Statistics

Tag(s): Book

Link to resource

Statistical Methods in Psychological Research Syllabus

A statistics book and tutorial about statistics

Author(s): Dr Matt Crump

Type of resources: Data Set, Full Course, Lecture, Lecture Notes, Lesson, Lesson Plan, Module, Reading, Student Guide, Syllabus, Teaching/Learning Strategy, Tutorial, Textbook

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Statistical Book, Tutorial

Link to resource

Statistical methods in psychology journals: Guidelines and explanations.

In the light of continuing debate over the applications of significance testing in psychology journals and following the publication of …

Author(s): Wilkinson, L., & Task Force on Statistical Inference, American Psychological Association, Science Directorate.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Statistical Modeling, Causal Inference, and Social Science

A blog about statistics and open science

Author(s): Andrew Gelman

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Blog, Reproducibility Knowledge

Link to resource

Statistical power analysis

A paper about statistical power

Author(s): Jacob Cohen

Type of resources: Primary Source, Reading

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Statistical power and optimal design in experiments in which samples of participants respond to samples of stimuli

Researchers designing experiments in which a sample of participants responds to a sample of stimuli are faced with difficult questions …

Author(s): Jacob Westfall, David A Kenny, Charles M Judd

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Statistical Power and the Testing of Null Hypotheses: A Review of Contemporary Management Research and Recommendations for Future Studies

The purpose of this study is to determine how well contemporary management research fares on the issue of statistical power with regard …

Author(s): Luke H. Cashen, Scott W. Geiger

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Statistical power in operations management research

This paper discusses the need and importance of statistical power analysis in field-based empirical research in Production and …

Author(s): Rohit Verma and John C. Goodale

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Statistical Power of Psychological Research: What Have We Gained in 20 Years?

Power was calculated for 6,155 statistical tests in 221 journal articles published in the 1982 volumes of the Journal of Abnormal …

Author(s): J S Rossi

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Statistical Power Problems with Moderated Multiple Regression in Management Research

Due to the increasing importance of moderating (i.e., interaction) effects, the use of moderated multiple regression (MMR) has become …

Author(s): Herman Aguinis

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Statistical Power, Sample Size, and Their Reporting in Randomized Controlled Trials

Objective. —To describe the pattern over time in the level of statistical power and the reporting of sample size calculations in …

Author(s): David Moher, MSc; Corinne S. Dulberg, PhD, MPH; George A. Wells, PhD

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics, Social Science

Tag(s):

Link to resource

Statistical Procedures and the Justification of Knowledge in Psychological Science

Justification, in the vernacular language of philosophy of science, refers to the evaluation, defense, and confirmation of claims of …

Author(s): Rosnow, R.L., & Rosenthal, R.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Statistical Reporting Errors and Collaboration on Statistical Analyses in Psychological Science

Statistical analysis is error prone. A best practice for researchers using statistics would therefore be to share data among …

Author(s): Coosje L. S. Veldkamp, 1, * Michèle B. Nuijten, Linda Dominguez-Alvarez, Marcel A. L. M. van Assen, and Jelte M. Wicherts

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s):

Link to resource

Statistical Rethinking Winter 2015

A video about Statistical Rethinking: A Bayesian Course with R

Author(s): Richard McElreath

Type of resources: Lecture, Teaching/Learning Strategy, Video

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Video, Bayesian

Link to resource

Statistical Significance and the Dichotomization of Evidence

In light of recent concerns about reproducibility and replicability, the ASA issued a Statement on Statistical Significance and …

Author(s): Blakeley B. McShane & David Gal

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Statistical significance in psychological research.

MOST THEORIES IN THE AREAS OF PERSONALITY, CLINICAL, AND SOCIAL PSYCHOLOGY PREDICT ONLY THE DIRECTION OF A CORRELATION, GROUP …

Author(s): Lykken

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s):

Link to resource

Statistical significance testing and cumulative knowledge in psychology: implications for training researchers.

Data analysis methods in psychology still emphasize statistical significance testing, despite numerous articles demonstrating its …

Author(s): F.L. Schmidt

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Statistical tests, p values, confidence intervals, and power: a guide to misinterpretations.

Misinterpretation and abuse of statistical tests, confidence intervals, and statistical power have been decried for decades, yet remain …

Author(s): Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Statistics and Quantitative Methods Example Videos for Teaching

The goal of this repository is to index and host short videos that can be used to supplement the teaching of introductory statistics …

Author(s): Jessica Kay Flake

Type of resources: Lesson

Primary user(s):

Subject area(s): Education

Tag(s): Quantitative Methods, Statistics

Link to resource

Statistics of DOOM

About Stats of DOOM

Support Statistics of DOOM! This page and the YouTube channel to help people learn statistics by including …

Author(s): Erin M. Buchanan

Type of resources: Lecture

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Educators, Open Education, Researchers, Statistics

Link to resource

Statistics with JASP and the Open Science Framework

This webinar will introduce the integration of JASP Statistical Software (https://jasp-stats.org/) with the Open Science Framework …

Author(s): Center for Open Science

Type of resources: Lecture

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science

Tag(s): Analysis, Data, Education, JASP, Materials, Open Scholarship Tools and Technologies, OSF, Publishing, Reproducibility, Research Data Management Tools, Researchers, Statistics

Link to resource

Stereotype threat

A podcast about stereotype threat and replication

Author(s): Simon Adler, Amanda Aronczyk and Dan Engber

Type of resources: Teaching/Learning Strategy, Podcast

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Podcast, Reproducibility Knowledge, Reproducibility Crisis and Credibility Revolution

Link to resource

Strong inference

certain systematic methods of scientific thinking may produce much more rapid progress than others.

Author(s): John R Platt

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Study Preregistration: An Evaluation of a Method for Transparent Reporting

Study preregistration promotes transparency in scientific research by making a clear distinction between a priori and post hoc …

Author(s): Allison A. Toth, George C. Banks, David Mellor, Ernest H. O’Boyle, Ashleigh Dickson, Daniel J. Davis, Alex DeHaven, Jaime Bochantin & Jared Borns

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Preregistration, Open Science, Reproducibility, Questionable Research Practices, Methodology

Link to resource

Supporting Open Science Data Curation, Preservation, and Access by Libraries

Openness in research can lead to greater reproducibility, an accelerated pace of discovery, and decreased redundancy of effort. In …

Author(s):

Type of resources: Lecture

Primary user(s):

Subject area(s): Education

Tag(s): Reproducability, Research

Link to resource

Supporting robust research on adult emotional development by considering context

A criterion for high quality science is to produce findings that are robust and replicable across studies. A potential hinderance to …

Author(s): Tabea Springstein, Claire M. Growney, Tammy English

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Life Science, Social Science

Tag(s): Robustness, Adult Emotional Development, Context, Sociocultural, Socioeconomic

Link to resource

Surrogate Science: The Idol of a Universal Method for Scientific Inference

The application of statistics to science is not a neutral act. Statistical tools have shaped and were also shaped by its objects. In …

Author(s): Gerd Gigerenzer and Julian N. Marewski

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s):

Link to resource

SWIRL

Swirl teaches you R programming and data science interactively, at your own pace, and right in the R console!

Author(s): Anon

Type of resources: Activity/Lab, Homework/Assignment, Student Guide, Teaching/Learning Strategy, Tutorial

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics

Tag(s): Tutorial

Link to resource

Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias — An Updated Review

Background The increased use of meta-analysis in systematic reviews of healthcare interventions has highlighted several types of bias …

Author(s): Carrol Gamble, Jamie J. Kirkham, Kerry Dwan, Paula R. Williamson

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Health, Medicine and Nursing

Tag(s): Clinical Trials, Drug Discovery, Meta-analysis, Peer Review, Publication Ethics, Publishing, Questionnaires, Research Funding, Scientific Publishing

Link to resource

Systematic Reviews and Meta-Analysis

When used together, systematic review methods and meta-analysis can produce comprehensive, accurate, and useful summaries of empirical …

Author(s): Valentine, J. C., Littell, J. H., & Young, S.

Type of resources: Full Course

Primary user(s): Student

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Systematic Review; Meta-Analysis

Link to resource

Taking stock of the credibility revolution: Scientific reform 2011-no

A book about the credibility revolution

Author(s): Gilad Feldman

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Credibility revolution

Link to resource

Teaching and Mentoring Open Science

A syllabi about mentoring and teaching open science

Author(s): Matthew Kim, Adrienne D. Woods, Alexa Ellis and Pamela Davis-Kean

Type of resources: Syllabus

Primary user(s): Teacher

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution; Open Science; Conceptual and statistical knowledge

Link to resource

teaching psych science

A collection of activities to teach APA writing and statistics

Author(s): Lewandowski, Jr., Ciarocco and Strohmetz

Type of resources: Activity/Lab, Asssessment

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics, Social Science

Tag(s):

Link to resource

Teaching replicable and reproducible science

Participants will develop materials for teaching replicability and reproducible science. Possible materials to be generated include …

Author(s): Kirsten Lane et al.

Type of resources: Module, Teaching/Learning Strategy

Primary user(s): Teacher

Subject area(s): Social Science

Tag(s): Teaching

Link to resource

Teaching replication

Replication is held as the gold standard for ensuring the reliability of published scientific literature. But conducting direct …

Author(s): Michael C. Frank and Rebecca Saxe

Type of resources: Primary Source, Reading, Paper

Primary user(s): Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution

Link to resource

Teaching Replication in Psychology: A Guide for Teachers and Students

This symposium explores the “replication crisis” from the perspective of teachers and students. Presenters will describe the major …

Author(s): Bradford Wiggins

Type of resources: Lecture, Lecture Notes, Syllabus

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): OSF Project

Link to resource

Teaching Replication to Graduate Students

Replicating published studies promotes active learning of quantitative research skills. Drawing on experiences from a replication …

Author(s): Dragana Stojmenovska, Thijs Bol, and Thomas Leopold

Type of resources: Case Study, Reading, Teaching/Learning Strategy

Primary user(s): Teacher

Subject area(s): Social Science

Tag(s): Teaching, Mentoring, Quantitative

Link to resource

Teaching resources spreadsheet

An excel spreadsheet about collection of open science items

Author(s): Courtney Soderberg

Type of resources: Reading

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Reproducibility Knowledge

Link to resource

Template preregistration Registered Report

If you can answer these TEN questions you will have built the engine of a Stage 1 Registered Report.

Author(s): Chris Chambers

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Funders, Librarians, Open Scholarship Guidelines, Publishers, Publishing, Researchers

Link to resource

Ten quick tips for building FAIR workflows

Research data is accumulating rapidly and with it the challenge of fully reproducible science. As a consequence, implementation of …

Author(s): Casper de Visser, Lennart F. Johansson, Purva Kulkarni, Hailiang Mei, Pieter Neerincx, K. Joeri van der Velde, Péter Horvatovich, Alain J. van Gool, Morris A. Swertz, Peter A. C. ‘t Hoen, Anna Niehues

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian, researcher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Computer Software, Metadata, Source Code, Programming Languages, Reproducibility, Data Management, Research Design, Software Tools

Link to resource

Ten Simple Rules for Effective Statistical Practice

A paper about Ten Simple Rules for Effective Statistical Practice

Author(s): Robert E.Kass et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Ten Simple Rules for Reproducible Computational Research

Replication is the cornerstone of a cumulative science. However, new tools and technologies, massive amounts of data, interdisciplinary …

Author(s): Anton Nekrutenko, Eivind Hovig, Geir Kjetil Sandve, James Taylor

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science

Tag(s): Archives, Computer and Information Sciences, Computer Applications, Data, Habits, Replication Studies, Reproducibility, Sequence Analysis, Source Code

Link to resource

Ten Simple Rules for the Care and Feeding of Scientific Data

A paper about ten Simple Rules for the Care and Feeding of Scientific Data

Author(s): Goodman, Alyssa, et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The (mis)reporting of statistical results in psychology journals

In order to study the prevalence, nature (direction), and causes of reporting errors in psychology, we checked the consistency of …

Author(s): Marjan Bakker & Jelte M. Wicherts

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s):

Link to resource

The 52 symptoms of major depression: Lack of content overlap among seven common depression scales

Depression severity is assessed in numerous research disciplines, ranging from the social sciences to genetics, and used as a dependent …

Author(s): Eiko I.Fried

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The Amazing Significo: why researchers need to understand poker

A post that describes significance values

Author(s): Dorothy Bishop

Type of resources: Reading

Primary user(s): Student, Teacher, Researcher

Subject area(s): Education, Life Science, Math & Statistics, Social Science

Tag(s): Blog, Reproducibility Crisis and Credibility Revolution

Link to resource

The appropriate use of null hypothesis testing.

The many criticisms of null hypothesis testing suggest when it is not useful and what is should not be used for. This article explores …

Author(s): R.W. Frick

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics

Tag(s):

Link to resource

The ASA Statement on p-Values: Context, Process, and Purpose

An editorial about p value

Author(s): Ronald L. Wasserstein & Nicole A. Lazar

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

The Baby Factory: Difficult Research Objects, Disciplinary Standards, and the Production of Statistical Significance

Science studies scholars have shown that the management of natural complexity in lab settings is accomplished through a mixture of …

Author(s): David Peterson

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The Bayes Factor

In this episode JP and Alex interview Zoltan Dienes. They discuss Zoltan’s passion for the martial arts, why Bayesian inference …

Author(s): Prof. JP De Ruiter

Type of resources: Student Guide, Teaching/Learning Strategy, Podcast

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Podcast

Link to resource

The Bayesfactor blog

Blog about Bayesfactor and statistics

Author(s): Richard Morey

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Blog

Link to resource

The Bayesian Reproducibility Project

An abstract about bayesian reproducibility project

Author(s): Alexander Etz

Type of resources: Data Set, Reading, Simulation, R code

Primary user(s): Student, Teacher, Researcher

Subject area(s): Applied Science, Life Science, Math & Statistics, Social Science

Tag(s): Blog, R code

Link to resource

The benefits of preregistration for hypothesis-driven bilingualism research

Preregistration is an open science practice that requires the specification of research hypotheses and analysis plans before the data …

Author(s): Daniela Mertzen, Sol Lago, Shravan Vasishth

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Business and Communication, Social Science

Tag(s): Preregistration, Open Science, Bilingualism, Psycholinguistics, Confirmatory Analysis, Exploratory Analysis

Link to resource

The Black Goat

Three psychologists talk about doing science. Hosted by Sanjay Srivastava, Alexa Tullett, and Simine Vazire.

Author(s): Sanjay Srivastava, Alexa Tullett, and Simine Vazire

Type of resources: Student Guide, Unit of Study, Podcast

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Podcast, Reproducibility Crisis and Credibility Revolution

Link to resource

The case against statistical significance testing

In recent years the use of traditional statistical methods in educational research has increasingly come under attack. In this article, …

Author(s): Ronald P Carver

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

The case for formal methodology in scientific reform

Current attempts at methodological reform in sciences come in response to an overall lack of rigor in methodological and scientific …

Author(s): Berna Devezer, Danielle J. Navarro, Joachim Vandekerckhove, Erkan Ozge Buzbas

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Double-dipping, Exploratory Research, Replication, Scientific Reform, Reproducibility

Link to resource

The Chrysalis Effect: How Ugly Initial Results Metamorphosize Into Beautiful Articles

The issue of a published literature not representative of the population of research is most often discussed in terms of entire studies …

Author(s): Ernest Hugh O’Boyle, Jr., George Christopher Banks, Erik Gonzalez-Mulé

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The citation advantage of linking publications to research data

Efforts to make research results open and reproducible are increasingly reflected by journal policies encouraging or mandating authors …

Author(s): Barbara McGillivray, Giovanni Colavizza, Iain Hrynaszkiewicz, Isla Staden, Kirstie Whitaker

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Life Science, Social Science

Tag(s): Computer Science, Data, Data Sharing, Digital Libraries, Policy, Publishing, Reproducibility

Link to resource

The ClinicalTrials.gov Results Database — Update and Key Issues

BACKGROUND The ClinicalTrials.gov trial registry was expanded in 2008 to include a database for reporting summary results. We summarize …

Author(s): Deborah A. Zarin, Tony Tse, Rebecca J. Williams, Robert M. Califf, Nicholas C. Ide

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Clinical Trials, Registry, Clinical Research

Link to resource

The Costs of HARKing

Kerr ([1998]) coined the term ‘HARKing’ to refer to the practice of ‘hypothesizing after the results are known’. This questionable …

Author(s): Mark Rubin

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Preregistration

Link to resource

The crisis of confidence in social psychology.

Notes that social psychologists' early enthusiasm has been replaced by serious doubts about the future of their field. Difficulties in …

Author(s): A.C. Elms

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The cumulative effect of reporting and citation biases on the apparent efficacy of treatments: the case of depression

Evidence-based medicine is the cornerstone of clinical practice, but it is dependent on the quality of evidence upon which it is based. …

Author(s): A. M. Roest, J. A. Bastiaansen, M. R. Munafò, P. Cuijpers, P. de Jonge, Y. A. de Vries

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Psychology

Tag(s): Antidepressants, Bias, Citation Bias, Depression, Psychotherapy, Reporting Bias

Link to resource

The default bayesian test is prejudiced against small effects

When considering any statistical tool I think it is useful to answer the following two practical questions: 1. “Does it give …

Author(s): Uri Simonsohn

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Blog

Link to resource

The earth is flat (p > 0.05): significance thresholds and the crisis of unreplicable research

The widespread use of ‘statistical significance’ as a license for making a claim of a scientific finding leads to considerable …

Author(s): Fränzi Korner-Nievergelt, Tobias Roth, Valentin Amrhein

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Statistics and Probability

Tag(s): Analysis, Data, Reproducibility, Statistics

Link to resource

The earth is round (p < .05).

After 4 decades of severe criticism, the ritual of null hypothesis significance testing (mechanical dichotomous decisions around a …

Author(s): Jacob Cohen

Type of resources: Primary Source, Reading

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s):

Link to resource

The Economics of Reproducibility in Preclinical Research

Low reproducibility rates within life science research undermine cumulative knowledge production and contribute to both delays and …

Author(s): Iain M. Cockburn, Leonard P. Freedman, Timothy S. Simcoe

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Biology

Tag(s): Drug Discovery, Drug Research and Development, Drug Therapy, Economics, Finance, Internet, Peer Review, Reproducibility

Link to resource

The effect of horizontal eye movements on free recall: A preregistered adversarial collaboration.

A growing body of research has suggested that horizontal saccadic eye movements facilitate the retrieval of episodic memories in free …

Author(s): Matzke et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science, Transparency

Link to resource

The effect of preregistration on trust in empirical research findings: results of a registered report

The crisis of confidence has undermined the trust that researchers place in the findings of their peers. In order to increase trust in …

Author(s): Sarahanne M. Field, E.-J. Wagenmakers, Henk A. L. Kiers, Rink Hoekstra, Anja F. Ernst, Don van Ravenzwaaij

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Preregistration, Registered Reporting, Trustworthiness, Questionable Research Practice

Link to resource

The effect of publishing peer review reports on referee behavior in five scholarly journals

To increase transparency in science, some scholarly journals are publishing peer review reports. But it is unclear how this practice …

Author(s): Bahar Mehmani, Emilia López-Iñesta, Flaminio Squazzoni, Francisco Grimaldo, Giangiacomo Bravo

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Information Science

Tag(s): Peer Review, Publishing

Link to resource

The empirical benefits of conceptual rigor: Systematic articulation of conceptual hypotheses can reduce the risk of non-replicable results (and facilitate novel discoveries too)

Most discussions of rigor and replication focus on empirical practices (methods used to collect and analyze data). Typically overlooked …

Author(s): Mark Schaller

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The Empirical March: Making Science Better at Self-Correction

Psychology has been criticized recently for a range of research quality issues. The current article organizes these problems around the …

Author(s): Matthew Makel

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The existence of publication bias and risk factors for its occurrence.

Publication bias is the tendency on the parts of investigators, reviewers, and editors to submit or accept manuscripts for publication …

Author(s): Kay Dickersin

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The Experiment Experiment

A few years back, a famous psychologist published a series of studies that found people could predict the future — not all the time, …

Author(s): Planet Money

Type of resources: Student Guide, Teaching/Learning Strategy, Podcast

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Podcast, Reproducibility Knowledge, Reproducibility Crisis and Credibility Revolution

Link to resource

The Extent and Consequences of P-Hacking in Science

A focus on novel, confirmatory, and statistically significant results leads to substantial bias in the scientific literature. One type …

Author(s): Head, M. L., Holman, L., Lanfear, R., Kahn, A. T., & Jennions, M. D.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

The fickle P value generates irreproducible results

The reliability and reproducibility of science are under scrutiny. However, a major cause of this lack of repeatability is not being …

Author(s): Lewis G Halsey, Douglas Curran-Everett, Sarah L Vowler & Gordon B Drummond

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

The file drawer problem and tolerance for null results.

For any given research area, one cannot tell how many studies have been conducted but never reported. The extreme view of the …

Author(s): R.Rosenthal

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The frequency of excess success for articles in Psychological Science.

Recent controversies have questioned the quality of scientific practice in the field of psychology, but these concerns are often based …

Author(s): Gregory Francis

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The garden of forking paths: Why multiple comparisons can be a problem, even when there is no “fishing expedition” or “p-hacking” and the research hypothesis was posited ahead of time

Data-dependent analysis—a “garden of forking paths”— explains why many statistically significant comparisons don’t hold up.

Author(s): Andrew Gelman and Eric Loken

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

The General Linear Model: Semester 2

A syllabi used for general linear model

Author(s): Patrick S. Forscher

Type of resources: Syllabus

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s): Statistics

Link to resource

The Generalizability of Survey Experiments

Survey experiments have become a central methodology across the social sciences. Researchers can combine experiments’ causal power with …

Author(s): Kevin J. Mullinix et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The GRIM Test: A Simple Technique Detects Numerous Anomalies in the Reporting of Results in Psychology

We present a simple mathematical technique that we call granularity-related inconsistency of means (GRIM) for verifying the summary …

Author(s): Nicholas J. L. Brown, James A. J. Heathers

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The Hardest Science

Blogposts about psychology, reproducibility, replication etc.

Author(s): Sanjay Srivastava

Type of resources: Reading

Primary user(s): Student, Researcher

Subject area(s): Math & Statistics, Social Science

Tag(s): Blog, Reproducibility Knowledge

Link to resource

The harm done by tests of significance

Three historical episodes in which the application of null hypothesis significance testing (NHST) led to the mis-interpretation of data …

Author(s): Ezra Hauer

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s):

Link to resource

The influence of journal submission guidelines on authors' reporting of statistics and use of open research practices

From January 2014, Psychological Science introduced new submission guidelines that encouraged the use of effect sizes, estimation, and …

Author(s): David Giofrè, Geoff Cumming, Ingrid Boedker, Luca Fresc, Patrizio Tressoldi

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Psychology

Tag(s): Data, Data Management, Experimental Psychology, Meta-analysis, Open Data, Open Science, Peer Review, Publishing, Research Reporting Guidelines, Scientific Publishing

Link to resource

The ironic effect of significant results on the credibility of multiple-study articles.

Cohen (1962) pointed out the importance of statistical power for psychology as a science, but statistical power of studies has not …

Author(s): Schimmack U

Type of resources: Primary Source, Reading, Reading

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

The meaning of “significance” for different types of research

Adrianus Dingeman de Groot (1914-2006) was one of the most influential Dutch psychologists. He became famous for his work …

Author(s): De Groot, A. D. translated and annotated by Eric-Jan Wagenmakers, Denny Borsboom, Josine Verhagen, Rogier Kievit, Marjan Bakker, Angelique Cramer, Dora Matzke, Don Mellenbergh, and Han LJ van der Maas

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science, Transparency

Link to resource

The Meaningfulness of Effect Sizes in Psychological Research: Differences Between Sub-Disciplines and the Impact of Potential Biases

Effect sizes are the currency of psychological research. They quantify the results of a study to answer the research question and are …

Author(s): Marcus A. Schwarz, Thomas Schäfer

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Psychology

Tag(s): Cohen, Effect Size, Power, Publication Bias, Replicability, Replication, Sample Size

Link to resource

The Missing Semester of Your CS Education

Course on computer sciences skills needed for all scientific research

Author(s): MIT

Type of resources: Full Course

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Career and Technical Education, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Computer Sciences

Link to resource

The N-Pact Factor: Evaluating the Quality of Empirical Journals with Respect to Sample Size and Statistical Power

The authors evaluate the quality of research reported in major journals in social-personality psychology by ranking those journals with …

Author(s): R. Chris Fraley, Simine Vazire

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

The natural selection of bad science.

Poor research design and data analysis encourage false-positive findings. Such poor methods persist despite perennial calls for …

Author(s): Smaldino, P. E., & McElreath, R.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The need for public opinion and survey methodology research to embrace preregistration and replication, exemplified by a team’s failure to replicate their own findings on visual cues in grid-type questions

Survey researchers take great care to measure respondents’ answers in an unbiased way; but, how successful are we as a field at …

Author(s): Sebastian Lundmark, John Protzko, Marcus Weissenbilder

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Social Science

Tag(s): Preregistration, Replication, Public Opinion

Link to resource

The New Statistics: Confidence Intervals, NHST, and p Values (Workshop Part 1)

A video about Confidence Intervals, NHST, and p Values

Author(s): Psychological Science/Geoff Cumming

Type of resources: Student Guide, Teaching/Learning Strategy, Video

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Video

Link to resource

The New Statistics: Effect Sizes and Confidence Intervals (Workshop Part 3)

A video about effect sizes and confidence intervals

Author(s): Psychological Science/Geoff Cumming

Type of resources: Student Guide, Teaching/Learning Strategy, Video

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics, Social Science

Tag(s): Video

Link to resource

The New Statistics: Meta-Analysis and Meta-Analytic Thinking (workshop Part 6)

A video about meta analysis and meta-analytical thinking

Author(s): Psychological Science/Geoff Cumming

Type of resources: Student Guide, Teaching/Learning Strategy, Video

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s): Video

Link to resource

The New Statistics: Planning, Power, and Precision (Workshop Part 5)

A video about power analysis and precision

Author(s): Psychological Science/Geoff Cumming

Type of resources: Student Guide, Teaching/Learning Strategy, Video

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Math & Statistics

Tag(s): Video

Link to resource

The New Statistics: Research Integrity & the New Statistics (Workshop Part 2)

A video about Research Integrity & the New Statistics

Author(s): Psychological Science/Geoff Cumming

Type of resources: Student Guide, Teaching/Learning Strategy, Video

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Life Science, Math & Statistics, Social Science

Tag(s): Video, Reproducibility Knowledge

Link to resource

The New Statistics: The New Statistics in Action (Workshop Part 4)

A video about New statistics in action

Author(s): Psychological Science/Geoff Cumming

Type of resources: Student Guide, Teaching/Learning Strategy, Video

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Video

Link to resource

The new statistics: Why and how

We need to make substantial changes to how we conduct research. First, in response to heightened concern that our published research …

Author(s): Geoff Cumming

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

The Null Hypothesis Significance-Testing Debate and Its Implications for Personality Research

A chapter about null hypothesis significance testing in personality research

Author(s): R. Chris Fraley and Michael J. Marks

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The Null Ritual What You Always Wanted to Know About Significance Testing but Were Afraid to Ask

A chapter about significance testing

Author(s): Gerd Gigerenzer, Stefan Krauss, and Oliver Vitouch

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Social Science

Tag(s):

Link to resource

The Open Research Lifecycle

Open science reduces waste and accelerates the discovery of knowledge, solutions, and cures for the world’s most pressing needs. …

Author(s):

Type of resources: Lecture

Primary user(s):

Subject area(s): Education

Tag(s): Center for Open Science, Open Science, Open Science Framework, Open Science Research Lifecycle, OSF, Reproducibility, Research, Research Best Practices, Research Integrity, Research Lifecycle, Research Rigor, Research Transparency

Link to resource

The Open Science Training Handbook

A collection about open science

Author(s): Sonja Bezjak, Philipp Conzett, Pedro L. Fernandes, Edit Görögh, Kerstin Helbig, Bianca Kramer, Ignasi Labastida, Kyle Niemeyer, Fotis Psomopoulos, Tony Ross-Hellauer, René Schneider, Jon Tennant, Ellen Verbakel

Type of resources: Activity/Lab, Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Life Science, Social Science

Tag(s):

Link to resource

The p-value misconception eradication challenge

If you have educational material that you think will do a better job at preventing p-value misconceptions than the material in my MOOC, …

Author(s): Daniel Lakens

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Teaching

Link to resource

The past, present and future of Registered Reports

Registered Reports are a form of empirical publication in which study proposals are peer reviewed and pre-accepted before research is …

Author(s): Christopher D. Chambers, Loukia Tzavella

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Culture, Publishing

Link to resource

The Peer Reviewers’ Openness Initiative: incentivizing open research practices through peer review.

Openness is one of the central values of science. Open scientific practices such as sharing data, materials and analysis scripts …

Author(s): Richard Morey et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Transparency, Peer-Review

Link to resource

The Persistence of Underpowered Studies in Psychological Research: Causes, Consequences, and Remedies

Underpowered studies persist in the psychological literature. This article examines reasons for their persistence and the effects on …

Author(s): S.E. Maxwell

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The pipeline project: Pre-publication independent replications of a single laboratory’s research pipeline

This crowdsourced project introduces a collaborative approach to improving the reproducibility of scientific research, in which …

Author(s): Martin Schweinsberg et al.

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The poor availability of psychological research data for reanalysis.

The origin of the present comment lies in a failed attempt to obtain, through e-mailed requests, data reported in 141 empirical …

Author(s): Wicherts, Jelte M., Borsboom, Denny, Kats, Judith, Molenaar, Dylan

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The poor availability of syntaxes of structural equation modeling

The syntax or codes used to fit Structural Equation Models (SEMs) convey valuable information on model specifications and the manner in …

Author(s): Jelte M. Wicherts and Elise A. V. Crompvoets

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

The Post-Embargo Open Access Citation Advantage: It Exists (Probably), It’s Modest (Usually), and the Rich Get Richer (of Course)

Many studies show that open access (OA) articles—articles from scholarly journals made freely available to readers without requiring …

Author(s): Jim Ottaviani

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Health, Medicine and Nursing

Tag(s): Careers, Citation Analysis, Institutional Repositories, Medicine and Health Sciences, Open Access Publishing, Peer Review, Physical Sciences, Scientific Publishing

Link to resource

The Preregistration Challenge: A How To Guide

This video shows interested researchers how to get started on their own preregistration as part of the Preregistration Challenge. Learn …

Author(s):

Type of resources: Lesson

Primary user(s):

Subject area(s): Education

Tag(s): Analysis Plan, Arnold Foundation, Bias, Center for Open Science, Cos, Data, OSF, Preregistration, Preregistration Challenge, Reproducibility, Research, Research Planning

Link to resource

The Preregistration Prescriptiveness Trade-Off and Unknown Unknowns in Science: Comments on Van Drimmelen (2023)

I discuss Van Drimmelen’s (2023) Metascience2023 presentation on researchers’ decision making during the research process. In …

Author(s): Mark Rubin

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Metaresearch, Metascience, Open Science, Preregistration, Questionable Metascience Practices, Researcher Degrees of Freedom, Researcher Discretion, Unknown Unknowns

Link to resource

The preregistration revolution

Progress in science relies in part on generating hypotheses with existing observations and testing hypotheses with new observations. …

Author(s): Brian A. Nosek, Charles R. Ebersole, Alexander C. DeHaven, and David T. Mellor

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s):

Link to resource

The preregistration revolution needs to distinguish between predictions and analyses

Nosek et al. (1) recently joined others in advocating for “widespread adoption of preregistration” as a tool for advancing science. The …

Author(s): Alison Ledgerwood

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Preregistration, Predictions, Analyses

Link to resource

The prevalence of statistical reporting errors in psychology (1985–2013)

This study documents reporting errors in a sample of over 250,000 p-values reported in eight major psychology journals from 1985 until …

Author(s): Michèle B. Nuijten, Chris H. J. Hartgerink, Marcel A. L. M. van Assen, Sacha Epskamp & Jelte M. Wicherts

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

The Problem of New Evidence: P-Hacking and Pre-Analysis Plans

We provide a novel articulation of the epistemic peril of p-hacking using three resources from philosophy: predictivism, Bayesian …

Author(s): Hitzig Zoë, Stegenga Jacob

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Arts and Humanities, Social Science

Tag(s): Bayesian Confirmation Theory, Pre-Analysis Plans, Replication Crisis, Predictivism, P-hacking, Philosophy

Link to resource

The psychology of experimental psychologists: Overcoming cognitive constraints to improve research: The 47th Sir Frederic Bartlett Lecture:

Like many other areas of science, experimental psychology is affected by a “replication crisis” that is causing concern in many fields …

Author(s): Dorothy VM Bishop

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Psychology

Tag(s): Reproducibility

Link to resource

The relation between statistical power and inference in fMRI.

Statistically underpowered studies can result in experimental failure even when all other experimental considerations have been …

Author(s): Cremers, H. R., Wager, T. D., & Yarkoni, T.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Life Science, Math & Statistics

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The Replication Crisis in Psychology

In science, replication is the process of repeating research to determine the extent to which findings generalize across time and …

Author(s): Edward Diener and Robert Biswas-Diener

Type of resources: Primary Source, Reading, Blog

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The replication crisis is less of a “crisis” in the Lakatosian approach than it is in the Popperian and naïve methodological falsificationism approaches

Popper’s (1983, 2002) philosophy of science has enjoyed something of a renaissance in the wake of the replication crisis, offering a …

Author(s): Mark Rubin

Type of resources: Reading

Primary user(s): Student, Teacher, researcher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Lakatos, Popper, Metaresearch, Metascience, Philosophy of Science, Replication Crisis, Theory Testing, Theory

Link to resource

The Replication Recipe: What makes for a convincing replication?

Psychological scientists have recently started to reconsider the importance of close replications in building a cumulative knowledge …

Author(s): Brandt et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The Reputational Consequences of Failed Replications and Wrongness Admission among Scientists

Scientists are dedicating more attention to replication efforts. While the scientific utility of replications is unquestionable, the …

Author(s): Adam K. Fetterman and Kai Sassenberg

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The Role of Libraries in the Age of Computational Reproducibility

A lighting talk at csv,conf,4 about how libraries and librarians are helping researchers with reproducibility.

Author(s): Gabriele Hayden, Vicky Steeves

Type of resources: Lesson

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Librarians, Organizational Change, Reproducibility, Researchers

Link to resource

The Role Of Replication Research In Advancing Gerontological Science: Trajectories, Transitions, And Typologies

The analysis of longitudinal observational data can take many forms and requires many decisions, with research findings and conclusions …

Author(s): Hofer S M

Type of resources: Reading

Primary user(s):

Subject area(s): Life Science, Social Science

Tag(s): Aging Research, Longitudinal Studies, Replication

Link to resource

The rules of the game called psychological science

If science were a game, a dominant rule would probably be to collect results that are statistically significant. Several reviews of the …

Author(s): Marjan Bakker, Annette van Dijk, Jelte M. Wicherts

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The secret to writing a great NASA proposal

Use a real NASA proposal as a roadmap and follow these tips for clearly presenting your research ideas. I’m a 100% soft money-funded …

Author(s): Chelle Gentemann

Type of resources: Reading

Primary user(s): Student, Teacher, Administrator, Librarian, researcher

Subject area(s): Applied Science, Math & Statistics, Physical Science

Tag(s): NASA, Grant Writing

Link to resource

The State of Social and Personality Science: Rotten to the Core, Not So Bad, Getting Better, or Getting Worse?

The scientific quality of social and personality psychology has been debated at great length in recent years. Despite research on the …

Author(s): Motyl et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The statistical power of abnormal-social psychological research: A review.

An article about statistical power of abnormal and social psychology

Author(s): Jacob Cohen

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Life Science, Social Science

Tag(s):

Link to resource

The STEM Education Hub Helps Your Research Workflow with Open Tools

OSF-specific screenshots and walk-throughs are very helpful, preregistration also discusses qualitative preregistration

Author(s): Alexander C, Bryan G, Center For Open Science, David Thomas Mellor, Marcy Reedy, Zachary Loomas

Type of resources: Lecture Notes

Primary user(s):

Subject area(s): Education

Tag(s): Research

Link to resource

The Superego, the Ego, and the Id in Statistical Reasoning

Statistical reasoning is an art and so demands both mathematical knowledge and informed judgment. When it is mechanized, as with the …

Author(s): Gerd Gigerenzer

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Math & Statistics, Social Science

Tag(s):

Link to resource

The t-distritbution and its normal approximation

I just published a new interactive visualization in my series of basic statistical concepts and techniques. This time I am trying to …

Author(s): Kristoffer Magnusson

Type of resources: Interactive, Simulation, Student Guide, Teaching/Learning Strategy

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Blog, Interaction, Simulation, Tutorial

Link to resource

The Test of Insufficient Variance (TIVA): A New Tool for the Detection of Questionable Research Practices

It has been known for decades that published results tend to be biased (Sterling, 1959). For most of the past decades this inconvenient …

Author(s): Ulrich Schimmack

Type of resources: Reading, Student Guide

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Blog

Link to resource

The Tone Debate: Knowledge, Self, and Social Order

In the replication crisis in psychology, a “tone debate” has developed. It concerns the question of how to conduct scientific debate …

Author(s): Maarten Derksen, Sarahanne Field

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Social Science

Tag(s): Tone Debate, Replication Crisis, Psychology, Social Media, Code of Conduct, Debate

Link to resource

The transparency of quantitative empirical legal research published in highly ranked law journals (2018–2020): an observational study

Background: Scientists are increasingly concerned with making their work easy to verify and build upon. Associated practices include …

Author(s): Jason Chin, Kathryn Zeiler, Natali Dilevski, Alex Holcombe, Rosemary Gatfield-Jeffries, Ruby Bishop, Simine Vazire, Sarah Schiavone

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Law, Social Science

Tag(s): Metaresearch, Open Science, Transparency, Credibility, Empirical Legal Research

Link to resource

The trustworthiness of the cumulative knowledge in industrial/organizational psychology: The current state of affairs and a path forward

The goal of industrial/organizational (IO) psychology, is to build and organize trustworthy knowledge about people-related phenomena in …

Author(s): Sheila K. Keener, Sven Kepes, Ann-Kathrin Torka

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Social Science

Tag(s): Reproducibility, Replicability, Scientific Misconduct, Questionable Research Practices, Trustworthiness of our Scientific Knowledge, Open Science Practices

Link to resource

The Unix Shell

Software Carpentry lesson on how to use the shell to navigate the filesystem and write simple loops and scripts. The Unix shell has …

Author(s): Adam Huffman, Adam James Orr, Adam Richie-Halford, AidaMirsalehi, Alexander Konovalov, Alexander Morley, Alex Kassil, Alex Mac, Alix Keener, Amy Brown, Andrea Bedini, Andrew Boughton, Andrew Reid, Andrew T. T. McRae, Andrew Walker, Ariel Rokem, Armin Sobhani, Ashwin Srinath, Bagus Tris Atmaja, Bartosz Telenczuk, Ben Bolker, Benjamin Gabriel, Bertie Seyffert, Bill Mills, Brian Ballsun-Stanton, BrianBill, Camille Marini, Chris Mentzel, Christina Koch, Colin Morris, Colin Sauze, csqrs, Damien Irving, Dana Brunson, Daniel Baird, Danielle M. Nielsen, Daniel McCloy, Daniel Standage, Dan Jones, Dave Bridges, David Eyers, David McKain, David Vollmer, Dean Attali, Devinsuit, Dmytro Lituiev, Donny Winston, Doug Latornell, Dustin Lang, earkpr, ekaterinailin, Elena Denisenko, Emily Dolson, Emily Jane McTavish, Eric Jankowski, Erin Alison Becker, Ethan P White, Evgenij Belikov, Farah Shamma, Fatma Deniz, Filipe Fernandes, Francis Gacenga, François Michonneau, Gabriel A. Devenyi, Gerard Capes, Giuseppe Profiti, Greg Wilson, Halle Burns, Hannah Burkhardt, Harriet Alexander, Hugues Fontenelle, Ian van der Linde, Inigo Aldazabal Mensa, Jackie Milhans, Jake Cowper Szamosi, James Guelfi, Jan T. Kim, Jarek Bryk, Jarno Rantaharju, Jason Macklin, Jay van Schyndel, Jens vdL, John Blischak, John Pellman, John Simpson, Jonah Duckles, Jonny Williams, Joshua Madin, Kai Blin, Kathy Chung, Katrin Leinweber, Kevin M. Buckley, Kirill Palamartchouk, Klemens Noga, Kristopher Keipert, Kunal Marwaha, Laurence, Lee Zamparo, Lex Nederbragt, Mahdi Sadjadi, Marcel Stimberg, Marc Rajeev Gouw, Maria Doyle, Marie-Helene Burle, Marisa Lim, Mark Mandel, Martha Robinson, Martin Feller, Matthew Gidden, Matthew Peterson, M Carlise, Megan Fritz, Michael Zingale, Mike Henry, Mike Jackson, Morgan Oneka, Murray Hoggett, Nicolas Barral, Nicola Soranzo, Noah D Brenowitz, Noam Ross, Norman Gray, nther, Orion Buske, Owen Kaluza, Patrick McCann, Paul Gardner, Pauline Barmby, Peter R. Hoyt, Peter Steinbach, Philip Lijnzaad, Phillip Doehle, Piotr Banaszkiewicz, Rafi Ullah, Raniere Silva, Rémi Emonet, reshama shaikh, Robert A Beagrie, Ruud Steltenpool, Ry4an Brase, Sarah Mount, Sarah Simpkin, s-boardman, Scott Ritchie, sjnair, Stéphane Guillou, Stephan Schmeing, Stephen Jones, Stephen Turner, Steve Leak, Susan Miller, Thomas Mellan, Tim Keighley, Tobin Magle, Tom Dowrick, Trevor Bekolay, Varda F. Hagh, Victor Koppejan, Vikram Chhatre, Yee Mey

Type of resources: Module

Primary user(s): student, teacher

Subject area(s): Computer Science, Measurement and Data

Tag(s): Analysis, Data, Education, Open Scholarship Tools and Technologies, Reproducibility, Research Data Management Tools, Researchers, Shell

Link to resource

The value of direct replication.

Reproducibility is the cornerstone of science. If an effect is reliable, any competent researcher should be able to obtain it when …

Author(s): Daniel Simons

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

The value of preregistration for psychological science: A conceptual analysis

For over two centuries researchers have been criticized for using research practices that makes it easier to present data in line with …

Author(s): Daniël Lakens

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Social Science

Tag(s): Preregistration, Registered Reports, Severity, Hypothesis Testing, Metascience

Link to resource

The Weak Spots in Contemporary Science (and How to Fix Them)

In this review, the author discusses several of the weak spots in contemporary science, including scientific misconduct, the problems …

Author(s): Jelte M. Wicherts

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Biology

Tag(s): Meta-research, Questionable Research Practices, Replicability, Reproducibility, Validity

Link to resource

The what, why, and how of born-open data

Although many researchers agree that scientific data should be open to scrutiny to ferret out poor analyses and outright fraud, most …

Author(s): Jeffrey N. Rouder

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s):

Link to resource

The What, Why, and How of Preregistration

More researchers are preregistering their studies as a way to combat publication bias and improve the credibility of research findings. …

Author(s): Center for Open Science

Type of resources: Lecture

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science

Tag(s): Analysis, Data, Education, Materials, Open Scholarship Tools and Technologies, OSF, Preregistation, Publishing, Reproducibility, Research Data Management Tools, Researchers

Link to resource

Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology.

Theories in “soft” areas of psychology (e.g., clinical, counseling, social, personality, school, and community) lack the …

Author(s): Meehl, P. E

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Theory Construction and Model-Building Skills: A Practical Guide for Social Scientists

Meeting a crucial need for graduate students and newly minted researchers, this innovative text provides hands-on tools for generating …

Author(s): James Jaccard, Jacob Jacoby

Type of resources: Textbook

Primary user(s): Student, Teacher, Researcher

Subject area(s): Applied Science, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Book

Link to resource

Theory-Testing in Psychology and Physics: A Methodological Paradox

Because physical theories typically predict numerical values, an improvement in experimental precision reduces the tolerance range and …

Author(s): Paul E.Meehl

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Things I have learned (so far)

This is an account of what I have learned (so far) about the application of statistics to psychology and the other sociobiomedical …

Author(s): Jacob Cohen

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics, Social Science

Tag(s):

Link to resource

Tie my hands loosely: Pre-analysis plans in political science

In this short article, I want to provide some of my thoughts on these developments from the perspective of someone who writes PAPs and …

Author(s): Daniel Rubenson

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Social Science

Tag(s): Preregistration, Political Science

Link to resource

Too true to be bad: When sets of studies with significant and nonsignificant findings are probably true

Psychology journals rarely publish nonsignificant results. At the same time, it is often very unlikely (or “too good to be true”) that …

Author(s): Lakens, D., & Etz, A. J.

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Tools for De-Identification of Personal Health Information

This report identifies useful and available tools and techniques for the deidentification of personal information from interoperable …

Author(s): Ross Fraser and Don Willison

Type of resources: Reading

Primary user(s): Administrator, Researcher

Subject area(s): Life Science, Social Science

Tag(s):

Link to resource

Tools for Reproducible Research

Course summary A minimal standard for data analysis and other scientific computations is that they be reproducible: that the code and …

Author(s): Karl Broman

Type of resources: Full Course

Primary user(s): student, teacher

Subject area(s): Information Science

Tag(s): Documentation, Literate Programming, Reproducibility, Version Control

Link to resource

TOP Guidelines

The Transparency and Openness Promotion guidelines include eight modular standards, each with three levels of increasing stringency. …

Author(s): Open Science Collaboration

Type of resources: Lesson

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Funders, Open Scholarship Policy, Open Standards, Policy, Policy Makers, Publishers, Publishing

Link to resource

Topics in Social Psychology and Personality

This seminar class will focus on the theme of Reproducibility in Social Psychology. We will discuss issues surrounding open science as …

Author(s): Sean Mackinnon

Type of resources: Syllabus

Primary user(s): Teacher

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Toward Reproducible Computational Research: An Empirical Analysis of Data and Code Policy Adoption by Journals

Journal policy on research data and code availability is an important part of the ongoing shift toward publishing reproducible …

Author(s): Peixuan Guo, Victoria Stodden, Zhaokun Ma

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science

Tag(s): Bibliometrics, Computational Biology, Data, Data Management, Open Access Publishing, Open Data, Policy, Publishing, Reproducibility, Science Policy, Scientific Publishing

Link to resource

Towards inclusive funding practices for early career researchers

Securing research funding is a challenge faced by most scientists in academic institutions worldwide. Funding success rates for all …

Author(s): Charlotte de Winde

Type of resources: Reading

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Funding

Link to resource

Towards reproducible radiomics research: introduction of a database for radiomics studies

Objectives To investigate the model-, code-, and data-sharing practices in the current radiomics research landscape and to introduce a …

Author(s): Tugba Akinci D’Antonoli, Renato Cuocolo, Bettina Baessler & Daniel Pinto dos Santos

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Life Science

Tag(s): Multiomics, Radiomics, Artificial intelligence, Reproducibility of results

Link to resource

Tracking replicability as a method of post-publication open evaluation

Recent reports have suggested that many published results are unreliable. To increase the reliability and accuracy of published papers, …

Author(s): Hartshorne, J. K., & Schachner, A.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Trainer Space for the Introduction to Open and Reproducible Research Workshop

Central location housing curriculum materials and planning tools for trainers of the COS Introduction to Open and Reproducible Research …

Author(s): Courtney K. Soderberg, Ian Sullivan, Jennifer Freeman Smith, Jolene Esposito, Matthew Spitzer, Natalie Meyers

Type of resources: Activity/Lab

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Education, Librarians, Reproducibility, Research Data Management Tools, Researchers

Link to resource

Transparency and Open Science Symposium GSA 2019

The past decade has seen rapid growth in conversations around and progress towards fostering a more transparent, open, and cumulative …

Author(s): Eileen K Graham, Gabrielle N, Jennifer Lodi-smith, Kendra Leigh Seaman, Rita M

Type of resources: Reading

Primary user(s):

Subject area(s): Life Science, Social Science

Tag(s): Aging, Aging Science, Gerontology, Open Science

Link to resource

Transparency of CHI Research Artifacts: Results of a Self-Reported Survey

Several fields of science are experiencing a “replication crisis” that has negatively impacted their credibility. Assessing …

Author(s): Chatchavan Wacharamanotham, Florian Echtler, Lukas Eisenring, Steve Haroz

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Life Science, Social Science

Tag(s): Data, Reproducibility

Link to resource

Transparency, replicability, and discovery in cognitive aging research: A computational modeling approach

Healthy aging is associated with deficits in performance on episodic memory tasks. Popular verbal theories of the mechanisms underlying …

Author(s): Kevin P. Darby, Per B. Sederberg

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Life Science, Social Science

Tag(s): Transparency, Replicability, Discovery, Cognitive Aging, Computational Modeling

Link to resource

Transparent and Open Social Science Research course

Demand is growing for evidence-based policy making, but there is also growing recognition in the social science community that limited …

Author(s): Berkeley Initiative for Transparency in the Social Sciences

Type of resources: Full Course, Lecture, Module, Video

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Course, Video, Reproducibility Knowledge

Link to resource

Transparent science: A more credible, reproducible, and publishable way to do science

This is an exciting time to be a psychological scientist. There is a major new movement that seeks to promote the credibility and …

Author(s): David Mellor, Simine Vazire and D. Lindsay

Type of resources: Primary Source, Reading, Chapter

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Transparent, Reproducible, and Open Science Practices of Published Literature in Dermatology Journals: Cross-Sectional Analysis

Background: Reproducible research is a foundational component for scientific advancements, yet little is known regarding the extent of …

Author(s): Andrew Niemann, Austin L. Johnson, Courtney Cook, Daniel Tritz, J. Michael Anderson, Matt Vassar

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Health, Medicine and Nursing, Biology, Genetics

Tag(s): Data, Dermatology, Reproducibility

Link to resource

Trial Publication after Registration in ClinicalTrials.Gov: A Cross-Sectional Analysis

Background ClinicalTrials.gov is a publicly accessible, Internet-based registry of clinical trials managed by the US National Library …

Author(s): Joseph S. Ross, Gregory K. Mulvey, Elizabeth M. Hines, Steven E. Nissen, Harlan M. Krumholz

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Clinical Trials, Citation Analysis, Lung and Intrathoracic Tumors, United States, Phase I Clinical Investigation, Safety Studies, Drug Administration, Medical Journals

Link to resource

Trial registration 10 years on

This month marks the tenth anniversary of the landmark decision by the International Committee of Medical Journal Editors to make …

Author(s): Wim E J Weber, José G Merino, Elizabeth Loder

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Clinical Research, Preregistration

Link to resource

Trust your science? Open your data and code.

A paper about Open Your Data and Code

Author(s): Victoria Stodden

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Tutorial/R code for creating funnel/forest plots

Meta-analyses are often accompanied by two popular forms of data visualization: forest plots and funnel plots. In this post, I’ll show …

Author(s): John K. Sakaluk

Type of resources: Reading, R code

Primary user(s): Student, Teacher

Subject area(s): Life Science, Math & Statistics, Social Science

Tag(s): Blog, Tutorial

Link to resource

Tutorial/R code for creating plots for 2-way interactions

ggplot2, as I’ve already made clear, is one of my favourite packages for R. And since that original post about ggplot2 remains one of …

Author(s): John K. Sakaluk

Type of resources: Reading, R code

Primary user(s): Student, Teacher

Subject area(s): Life Science, Math & Statistics, Social Science

Tag(s): Blog, Tutorial

Link to resource

Tutorial/R code for creating scree/parallel analysis plots

With this post, I’m going to be showing how you can use the psych package in conjunction with ggplot2 in order to create a prettier …

Author(s): John K Sakaluk

Type of resources: Reading, R code

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Blog, Tutorial

Link to resource

Two Years Later: Journals Are Not Yet Enforcing the ARRIVE Guidelines on Reporting Standards for Pre-Clinical Animal Studies

A study by David Baker and colleagues reveals poor quality of reporting in pre-clinical animal research and a failure of journals to …

Author(s): Ana Sottomayor, David Baker, Katie Lidster, Sandra Amor

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Health, Medicine and Nursing, Life Science

Tag(s): Analysis, Animal Models, Animal Studies, Data, Experimental Design, Publication Ethics, Publishing, Research Laboratories, Research Reporting Guidelines, Scientific Publishing, Statistical Data, Statistics

Link to resource

Tópicos Especiais em Biotecnologia: Planejamento e Otimização de Experimentos

A crise de confiança na ciência. Práticas e condutas questionáveis na pesquisa. Delineamento experimental adequado. Definição de …

Author(s): Caio Maximino and Monica Gomes Lima-Maximino

Type of resources: Syllabus

Primary user(s): Teacher

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

UKRN

The UKRN primer series is designed to introduce a broad audience to important topics in open and reproducible scholarship. Each primer …

Author(s): Emma Henderson, Jackie Thompson

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Reproducibility, Research Administration, Researchers

Link to resource

UKRN Open Research Primers

Open Research Action Plan, Data Sharing, Open Access, Open Code & Software, Open Resarch Awards, Preprints, Preregistration & …

Author(s): UKRN

Type of resources: Reading

Primary user(s):

Subject area(s): Education

Tag(s): Research

Link to resource

Ulysses' pact or Ulysses' raft: Using pre-analysis plans in experimental and nonexperimental research

Economists have recently adopted pre-analysis plans in response to concerns about robustness and transparency in research. The …

Author(s): Sarah A. Janzen, Jeffrey D. Michler

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Business and Communication, Social Science

Tag(s): Hypothesis Registry, Pre-Analysis Plan, Preregistration, Research Ethics, Transparency

Link to resource

Underreporting in Psychology Experiments: Evidence from a Study Registry.

Many scholars have raised concerns about the credibility of empirical findings in psychology, arguing that the proportion of false …

Author(s): Annie Franco, Neil Malhotra, and Gabor Simonovits

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Understanding Bayes: A Look at the Likelihood

A blog about bayesian statistics

Author(s): Alexander Etz

Type of resources: Reading

Primary user(s): Student

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Blog

Link to resource

Understanding Bayes: Visualization of the Bayes Factor

An abstract about Understanding Bayes and visualising Bayes Factor

Author(s): Alexander Etz

Type of resources: Diagram/Illustration, Reading, Student Guide, Teaching/Learning Strategy, R code

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Blog, R code

Link to resource

Understanding psychology as a science: An introduction to scientific and statistical inference

How can we objectively define categories of truth in scientific thinking? How can we reliably measure the results of research? In this …

Author(s): Zoltan Dienes

Type of resources: Textbook

Primary user(s): Student, Teacher, Researcher

Subject area(s): Applied Science, Life Science, Math & Statistics, Social Science

Tag(s): Book

Link to resource

Understanding Statistical Power and Significance Testing: an interactive visualization

Much has been said about significance testing – most of it negative. Methodologists constantly point out that researchers misinterpret …

Author(s): Kristoffer Magnusson

Type of resources: Interactive, Simulation, Student Guide, Teaching/Learning Strategy

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Blog, Interaction, Simulation, Tutorial

Link to resource

Unlikely Results

Why most published scientific research is probably false

Author(s): The Economist

Type of resources: Student Guide, Teaching/Learning Strategy, Video

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Math & Statistics, Social Science

Tag(s): Video

Link to resource

Up front and open, shrouded in secrecy, or somewhere in between? A Meta Research Systematic Review of Open Science Practices in Sport Medicine Research

OBJECTIVE: To investigate open science practices in research published in the top five sports medicine journals from 01 May 2022 and 01 …

Author(s): Garrett S. Bullock, Patrick Ward, Franco M. Impellizzeri, Stefan Kluzek, Tom Hughes, Charles Hillman, Brian R. Waterman, Kerry Danelson, Kaitlin Henry, Emily Barr, Kelsey Healey, Anu M. Räisänen, Christina Gomez, Garrett Fernandez, Jakob Wolf, Kristen F. Nicholson, Tim Sell, Ryan Zerega, Paula Dhiman, Richard D. Riley, Gary S Collins

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Open Access, Open Code, Study Protocol, Reporting Guidelines

Link to resource

Update on the endorsement of CONSORT by high impact factor journals: a survey of journal “Instructions to Authors” in 2014

The CONsolidated Standards Of Reporting Trials (CONSORT) Statement provides a minimum standard set of items to be reported in published …

Author(s): David Moher, Douglas G. Altman, Kenneth F. Schulz, Larissa Shamseer, Sally Hopewell

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Health, Medicine and Nursing

Tag(s): Data, Publishing, Reporting Guidelines

Link to resource

Update on Trial Registration 11 Years after the ICMJE Policy Was Established

Laws and policies to establish a global trial reporting system have greatly increased the transparency and accountability of the …

Author(s): Deborah A. Zarin, Tony Tse, Rebecca J. Williams, Thiyagu Rajakannan

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Trial Registration, Medicine, Preregistration

Link to resource

Use of the Journal Impact Factor in academic review, promotion, and tenure evaluations

The Journal Impact Factor (JIF) was originally designed to aid libraries in deciding which journals to index and purchase for their …

Author(s): Carol Muñoz Nieves, Erin C. McKiernan, Juan Pablo Alperin, Lesley A. Schimanski, Lisa Matthias, Meredith T. Niles

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Health, Medicine and Nursing, Information Science, Life Science, Social Science

Tag(s): And Tenure, Journal Impact Factor, Promotion, Publishing, Review

Link to resource

Using artificial intelligence to create diverse and inclusive medical case vignettes for education

Introduction Medical case vignettes play a crucial role in medical education, yet they often fail to authentically represent diverse …

Author(s): Michiel J. Bakkum, Mariëlle G. Hartjes, Joost D. Piët, Erik M. Donker, Robert Likic, Emilio Sanz, Fabrizio de Ponti, Petra Verdonk, Milan C. Richir, Michiel A. van Agtmael, Jelle Tichelaar

Type of resources: Reading

Primary user(s): Teacher, Librarian

Subject area(s): Applied Science, Life Science, Social Science

Tag(s): Artificial Intelligence, chatGPT, Diversity, Inclusivity

Link to resource

Using OSF to Share Data: A Step-by-Step Guide

Sharing data, materials, and analysis scripts with reviewers and readers is valued in psychological science. To facilitate this …

Author(s): Courtney K. Soderberg

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Using Pre-Analysis Plans in Qualitative Research

In the last decade, there has been a significant push for greater transparency in the social sciences. For example, epistemological and …

Author(s): Verónica Pérez Bentancur, Rafael Piñeiro Rodríguez, Fernando Rosenblattv

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Social Science

Tag(s): Pre-Analysis Plan, Preregistration, Qualitative Research

Link to resource

Using prediction markets to estimate the reproducibility of scientific research

Concerns about a lack of reproducibility of statistically significant results have recently been raised in many fields, and it has been …

Author(s): Anna Drebner et al.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Using R and lme/lmer to fit different two- and three- level longitudinal models

I often get asked how to fit different multilevel models (or individual growth models, hierarchical linear models or linear …

Author(s): Kristoffer Magnusson

Type of resources: Reading, R code

Primary user(s): Student, Teacher, Researcher

Subject area(s): Education, Life Science, Math & Statistics, Social Science

Tag(s): Blog, Tutorial

Link to resource

Using science and psychology to improve the dissemination and evaluation of scientific work

Here I outline some of what science can tell us about the problems in psychological publishing and how to best address those problems. …

Author(s): Brett T. Buttliere

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Version control with the OSF

This webinar will introduce the concept of version control and the version control features that are built into the Open Science …

Author(s): Center for Open Science

Type of resources: Lecture

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science

Tag(s): Analysis, Data, Education, Materials, Open Scholarship Tools and Technologies, OSF, Publishing, Reproducibility, Research Data Management Tools, Researchers

Link to resource

Visualizing a One-Way ANOVA using D3.js

A blog post and tutorial on visualizing one-way ANOVA

Author(s): Kristoffer Magnusson

Type of resources: Interactive, Reading, Simulation, Student Guide, Teaching/Learning Strategy

Primary user(s): Student, Teacher

Subject area(s): Math & Statistics

Tag(s): Blog, Interactive, Tutorial

Link to resource

Want to speed up scientific progress? First understand how science policy works

Researchers and policymakers often exist in different worlds and speak different languages. Here are three ways to bridge the divide.

Author(s): Matt Clancy, Dan Correa, Jordan Dworkin, Paul Niehaus, Caleb Watney & Heidi Williams

Type of resources: Primary Source, Reading

Primary user(s): Teacher, Administrator, Librarian

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Policy, Funding, Scientific Communication

Link to resource

We have to break up

Three mostly positive developments in academic psychology—the cognitive revolution, the virtual requirement for multiple study reports …

Author(s): Robert B. Cialdini

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

We Knew the Future All Along: Scientific Hypothesizing Is Much More Accurate Than Other Forms of Precognition-A Satire in One Part

A critique about Daryl Bem’s (2011) paper and reproducibility

Author(s): Arina K Bones

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Welcoming Quality in Non-Significance and Replication Work, but Moving Beyond the p-Value: Announcing New Editorial Policies for Quantitative Research in JOAA

The self-correcting nature of psychological and educational science has been seriously questioned. Recent special issues of …

Author(s): Matthew T. McBee, Michael S. Matthews

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

What does research reproducibility mean?

The language and conceptual framework of “research reproducibility” are nonstandard and unsettled across the sciences. In this …

Author(s): Goodman, S. N., Fanelli, D., & Ioannidis, J. P.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

What incentives increase data sharing in health and medical research? A systematic review

The foundation of health and medical research is data. Data sharing facilitates the progress of research and strengthens science. Data …

Author(s): Adrian G. Barnett, Anisa Rowhani-Farid, Michelle Allen

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Health, Medicine and Nursing

Tag(s): Data, Data Sharing, Open Data

Link to resource

What is a p-value?

Blog post going over the p value, misnomers and what p < .05 means

Author(s): Daniel Lakens

Type of resources: Reading, R Code

Primary user(s): Student, Teacher, Researcher

Subject area(s): Applied Science, Education, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Blog, Code, Reproducibility Crisis and Credibility Revolution

Link to resource

What is replication?

Replications are inevitably different from the original studies. How do we decide whether something is a replication? The answer shifts …

Author(s): Brian A. Nosek, Timothy M. Errington

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Reproducibility, Researchers

Link to resource

What is statistical power

This video is the first in a series of videos related to the basics of power analyses. All materials shown in the video, as well as …

Author(s): Center for Open Science

Type of resources: Lecture

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science

Tag(s): Analysis, Data, Education, Materials, Open Scholarship Tools and Technologies, OSF, Power, Publishing, Reproducibility, Research Data Management Tools, Researchers, Statistics

Link to resource

What is this thing called open science?

A general introduction to open scholarship.

Author(s): Tom Hardwicke

Type of resources: Lesson

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Data, Funders, History of Science, Metascience, Open Data, Open Scholarship Policy, Policy, Policy Makers, Publishing, Publishing Models, Reproducibility, Research Administration, Researcher Degrees of Freedom, Researchers, Research Integrity, Statistics, Transparency

Link to resource

What is Universal Design for Learning (UDL)?

Universal Design for Learning (UDL) is a way of thinking about teaching and learning that helps give all students an equal opportunity …

Author(s): Amanda Morin and Ace Parsi

Type of resources: Reading

Primary user(s): Student, Teacher, Parent

Subject area(s): Career and Technical Education, Education, Social Science

Tag(s): Diversity, Equity, Inclusion, Neurodiversity

Link to resource

What should a preregistration contain?

A large amount of variation exists in beliefs about the purpose and benefits of preregistration, making it difficult to implement and …

Author(s): Jonathon McPhetres

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Hypothesis Testing, Metascience, Open Science, Planning, Preregistration, Research Methods. Transparency

Link to resource

What You See Is What You Get? Enhancing Methodological Transparency in Management Research

We review the literature on evidence-based best practices on how to enhance methodological transparency, which is the degree of detail …

Author(s): Herman Aguinis, Ravi S. Ramani and Nawaf Alabduljader

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

What’s wrong with Psychology, anyway?

This chapter considers various factors that have been responsible for the comparatively slow development of psychology into a …

Author(s): David T. Lykken

Type of resources: Primary Source, Reading

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science, Transparency

Link to resource

What’s wrong with statistical tests – and where do we go from here?

This chapter considers problems with null hypothesis significance testing (NHST). The literature in this area is quite large. D. …

Author(s): R.B. Kline

Type of resources: Primary Source, Reading, Chapter

Primary user(s): Student

Subject area(s): Social Science

Tag(s):

Link to resource

When Does HARKing Hurt? Identifying When Different Types of Undisclosed Post Hoc Hypothesizing Harm Scientific Progress

Hypothesizing after the results are known, or HARKing, occurs when researchers check their research results and then add or remove …

Author(s): Mark Rubin

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Accommodation, Falsification, HARKing, Prediction, Replication Crisis

Link to resource

When Great Minds Think Unalike: Inside Science’s ‘Replication Crisis’

A podcast about replication crisis

Author(s): Shankar Vedantam and Maggie Penman

Type of resources: Student Guide, Teaching/Learning Strategy, Podcast

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Podcast, Reproducibility Knowledge, Reproducibility Crisis and Credibility Revolution

Link to resource

When is science (un)reliable?

In this course, we will explore the so‐called “reproducibility crisis” that has struck fields from psychology and economics to ecology …

Author(s): Tim Parker and Tom Armstrong

Type of resources: Syllabus

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution; Open Science; Conceptual and statistical knowledge

Link to resource

When power analyses based on pilot data are biased: Inaccurate effect size estimators and follow-up bias

When designing a study, the planned sample size is often based on power analyses. One way to choose an effect size for power analyses …

Author(s): Casper Albers and Daniel Lakens

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Which is the correct statistical test to use?

This paper explains how to select the correct statistical test for a research project, clinical trial, or other investigation. The …

Author(s): Evie McCrum-Gardner

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Who Re-Uses Data? A Bibliometric Analysis of Dataset Citations

Open data is receiving increased attention and support in academic environments, with one justification being that shared data may be …

Author(s): Geoff Krause, Madelaine Hare, Mike Smit, Philippe Mongeon

Type of resources: Reading

Primary user(s): Student, Teacher, Administrator, Librarian

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Open Data, Citations

Link to resource

Why an Entire Field of Psychology Is in Trouble

A video about psychology being trouble

Author(s): SciShow/Michael Aranda

Type of resources: Student Guide, Teaching/Learning Strategy, Video

Primary user(s): Student, Teacher

Subject area(s): Social Science

Tag(s): Video

Link to resource

Why and How to Use Pre-Analysis Plans

We describe what is a pre-analysis plan (PAP) and why you should use one. We emphasize the potential political uses of PAPs and, in …

Author(s): David Yokum, Jake Bowers

Type of resources: Reading

Primary user(s): Student, Teacher

Subject area(s): Social Science

Tag(s): Pre-analysis plans, Preregistration, Open Science, Motivated Reasoning, Policy

Link to resource

Why Hypothesis Testers Should Spend Less Time Testing Hypotheses

For almost half a century, Paul Meehl educated psychologists about how the mindless use of null-hypothesis significance tests made …

Author(s): Anne M. Scheel, Leonid Tiokhin, Peder M. Isager, Daniël Lakens

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Education, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Exploratory Research, Hypothesis Testing, Replication Crisis

Link to resource

Why Most Discovered True Associations Are Inflated

Newly discovered true (non-null) associations often have inflated effects compared with the true effect sizes. I discuss here the main …

Author(s): John P A Ioannidis

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Why most of psychology is statistically unfalsifiable

Low power in experimental psychology is an oft-discussed problem. We show in the context of the Replicability Project: Psychology (Open …

Author(s): Richard Morey and Daniel Lakens

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Social Science

Tag(s):

Link to resource

Why most published research findings are false

There is increasing concern that most current published research findings are false. The probability that a research claim is true may …

Author(s): John Ioannidis

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Why NASA and federal agencies are declaring this the Year of Open Science

I’m thrilled to be the Transform to Open Science lead for NASA, which has a 60-year legacy of pushing the limits of how science is used …

Author(s): Chelle Gentemann

Type of resources: Reading

Primary user(s): Student, Teacher, Administrator, Librarian, researcher

Subject area(s): Applied Science, Math & Statistics, Physical Science

Tag(s): Year of Open Science, NASA, Federal Research

Link to resource

why p = .048 should be rare (and why this feels counterintuitive)

This post discusses the why p value around .048 should be rare

Author(s): Simine Vazire

Type of resources: Reading

Primary user(s): Student, Teacher, Researcher

Subject area(s): Math & Statistics

Tag(s): Blog

Link to resource

Why Preregistration Makes Me Nervous

I must admit that when I first heard of the effort to get psychological scientists to preregister their studies (that is, to submit to …

Author(s): Susan Goldin-Meadow

Type of resources: Reading

Primary user(s): Student, Teacher, Librarian

Subject area(s): Applied Science, Arts and Humanities, Business and Communication, Career and Technical Education, Education, English Language Arts, History, Law, Life Science, Math & Statistics, Physical Science, Social Science

Tag(s): Preregistration, Badges

Link to resource

Why psychologists must change the way they analyze their data: The case of psi: Comment on Bem (2011).

Does psi exist? D. J. Bem (2011) conducted 9 studies with over 1,000 participants in an attempt to demonstrate that future events …

Author(s): Wagenmakers, E.-J., Wetzels, R., Borsboom, D., & van der Maas, H. L. J.

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Why Psychologists’ Food Fight Matters

“Important findings” haven’t been replicated, and science may have to change its ways.

Author(s): Michelle N.Meyer and Christopher Chabris

Type of resources: Blog

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Blog, Open Science, Reproducibility Crisis and Credibility Revolution

Link to resource

Why Summaries of Research on Psychological Theories are Often Uninterpretable

Null hypothesis testing of correlational predictions from weak substantive theories in soft psychology is subject to the influence of …

Author(s): Paul Meehl

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Wide-Open: Accelerating public data release by automating detection of overdue datasets

Open data is a vital pillar of open science and a key enabler for reproducibility, data reuse, and novel discoveries. Enforcement of …

Author(s): Bill Howe, Hoifung Poon, Maxim Grechkin

Type of resources: Reading

Primary user(s): student, teacher

Subject area(s): Biology

Tag(s): Archives, Biotechnology, Data, Data Mining, Gene Expression, Open Data, Sequence Databases, Text Mining, Web-based Applications

Link to resource

Willingness to share research data is related to the strength of the evidence and the quality of reporting of statistical results

Background: The widespread reluctance to share published research data is often hypothesized to be due to the authors’ fear that …

Author(s): Jelte M. Wicherts, Marjan Bakker, Dylan Molenaar

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Life Science, Social Science

Tag(s):

Link to resource

Workflow for Awarding Badges

Badges are a great way to signal that a journal values transparent research practices. Readers see the papers that have underlying data …

Author(s): Center for Open Science

Type of resources: Lecture

Primary user(s): student, teacher

Subject area(s): Computer Science, Information Science

Tag(s): Analysis, Data, Education, Materials, Open Scholarship Tools and Technologies, OSF, Publishing, Reproducibility, Research Data Management Tools, Researchers

Link to resource

Writing a Data Management Plan for Grant Applications

A class covering the basics of writing a successful data management plan for federal funding agencies such as the NEH, NSF, NIH, NASA, …

Author(s): Nick Wolf, Vicky Steeves

Type of resources: Activity/Lab

Primary user(s): student, teacher

Subject area(s): Applied Science, Life Science, Physical Science, Social Science

Tag(s): Data, Open Scholarship Guidelines, Research Data Management, Researchers

Link to resource

Writing reproducible geoscience papers using R Markdown, Docker, and GitLab

Reproducibility is unquestionably at the heart of science. Scientists face numerous challenges in this context, not least the lack of …

Author(s): Daniel Nüst, Edzer Pebesma, Markus Konkol, Rémi Rampin, Vicky Steeves

Type of resources: Activity/Lab

Primary user(s): student, teacher

Subject area(s): Physical Science

Tag(s): Analysis, Docker, Geosciences, GitLab, Open Scholarship Tools and Technologies, Publishing, R, Reproducibility, Research Data Management Tools, Researchers, RMarkdown, RStudio

Link to resource

You are not so smart

Psychology is working on the hardest problems in all of science. Physics, astronomy, geology — those are easy, by comparison. …

Author(s): David McRaney

Type of resources: Student Guide, Teaching/Learning Strategy, Podcast

Primary user(s): Student, Teacher

Subject area(s): Applied Science, Social Science

Tag(s): Podcast, Reproducibility Crisis and Credibility Revolution

Link to resource

You Cannot Step Into the Same River Twice: When Power Analyses Are Optimistic

Statistical power depends on the size of the effect of interest. However, effect sizes are rarely fixed in psychological research: …

Author(s): Blakeley B McShane, Ulf Böckenholt

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Math & Statistics

Tag(s):

Link to resource

Your Questions Answered: How to Retain Copyright While Others Distribute and Build Upon Your Work

In this webinar, a panel discusses licensing options, fundamentals in choosing a license for your research, and answers questions about …

Author(s): Joanna Schimizzi, Brandon Butler, and Becca Neel

Type of resources: Lesson

Primary user(s):

Subject area(s): Education

Tag(s): Licensing, Open Science, OSF, Scholarly Communication, Scholarship

Link to resource

‘‘Positive’’ Results Increase Down the Hierarchy of the Sciences

The hypothesis of a Hierarchy of the Sciences with physical sciences at the top, social sciences at the bottom, and biological sciences …

Author(s): Daniele Fanelli

Type of resources: Primary Source, Reading, Paper

Primary user(s): Student

Subject area(s): Applied Science, Social Science

Tag(s): Reproducibility Crisis and Credibility Revolution, Open Science

Link to resource

Missing a resource?

Fill out the form below or here is the direct link to open the Google form in a new window