Open Science & Reproducible Research Clusters — FORRT Taxonomy | FORRT - Framework for Open and Reproducible Research Training

1 Replication Crisis and Credibility Revolution

7 sub-clusters · 146 references

Attainment of foundational knowledge on the importance of reproducible and open research (i.e., grounding the motivations and theoretical underpinnings of Open and Reproducible Science). Integration with field specific content (i.e., grounded in the history of replicability). There are 7 sub-clusters which aim to further parse the learning and teaching process:

▸ History of the replication crisis & credibility revolution 26 / 26

In order to understand and weigh in on current developments, we need to first understand how the Open and Reproducible Science movement started, from its origins over the replicability/reproducibility crisis to the credibility revolution.

critique Paper

Retiring Popper: Critical realism, falsificationism, and the crisis of replication

This article offers a philosophical critique of the replication crisis debate in psychology, arguing that the field's focus on Popperian falsificationism lacks necessary ontological depth. It suggests that methodological reforms alone are insufficient without addressing the underlying assumptions about the nature of psychological phenomena.

evidence Paper

Quantitative Political Science Research Is Greatly Underpowered

This large-scale meta-research study provides empirical evidence that quantitative political science research is severely underpowered, with a median power of only 10% across thousands of tests. The findings demonstrate that only a small fraction of tests in the discipline meet the standard 80% power threshold required to detect consensus effects.

evidence Paper

1,500 scientists lift the lid on reproducibility

This resource presents the results of a comprehensive survey of 1,500 scientists to quantify their experiences with and attitudes toward the reproducibility crisis. It provides empirical evidence on how researchers across various fields perceive the severity of the issue and the specific factors they believe contribute most to failed replication attempts.

overview Paper

Is there a reproducibility crisis in science?

This resource explores the fundamental question of whether science is experiencing a reproducibility crisis by summarizing the key arguments and evidence. It provides an introductory survey of the topic suitable for those looking to understand the scope and implications of the replication debate across different scientific fields.

critique Paper

Understanding the Replication Crisis as a Base Rate Fallacy

This paper presents a theoretical critique of the standard narrative that the replication crisis is primarily caused by poor scientific conduct or questionable research practices. It uses the logic of the base rate fallacy to argue that high failure rates in replications are a predictable mathematical outcome in fields that investigate a large proportion of unlikely hypotheses.

advocacy Book

The Seven Deadly Sins of Psychology

This work identifies and analyzes systemic flaws in psychological science, such as publication bias and lack of transparency, that contribute to the replication crisis. It makes a strong case for institutional reform and the adoption of open science practices, such as Registered Reports, to improve the reliability of the field.

evidence Paper

Estimating the Reproducibility of Experimental Philosophy

This study reports the results of a large-scale collaboration to replicate 40 foundational findings in experimental philosophy to empirically assess the field's reproducibility. It provides specific data on replication success rates and effect sizes, offering a benchmark for the reliability of research within this philosophical sub-discipline.

overview Paper

Seven Easy Steps to Open Science

This article serves as an accessible introductory guide to open science, specifically tailored for students and researchers in psychological science. It uses a curated and annotated reading list of seven foundational papers to explain core concepts such as pre-registration, open access, and data sharing.

overview Paper

Academic Research in the 21st Century: Maintaining Scientific Integrity in a Climate of Perverse Incentives and Hypercompetition

This handbook offers a comprehensive survey of the field of academic integrity, addressing both theoretical research and practical applications in global educational settings. It serves as a foundational reference that bridges various perspectives on plagiarism, ethical conduct, and the systemic factors influencing scholarly honesty.

critique Paper

What is the Replication Crisis a Crisis Of?

This paper provides a theoretical analysis of the replication crisis in psychology, arguing that the debate must move beyond methodological fixes to address fundamental questions about how the subject matter is defined. It highlights the tension between seeking stable effects and acknowledging the inherent context sensitivity of psychological phenomena, suggesting that theoretical inadequacy is a central component of the crisis.

overview Preprint

Reproducibility Failure in Biomedical Research: Problems and Solutions

This review identifies specific failures in the research lifecycle—from reporting to peer review—that lead to irreproducibility in biomedicine. It evaluates the effectiveness of proposed reforms and emphasizes the need for evidence-based interventions to ensure scientific changes are both rigorous and beneficial.

advocacy Paper

The replication crisis has led to positive structural, procedural, and community changes

This resource reframes the replication crisis as a positive "credibility revolution" by cataloging the constructive structural and community-driven improvements it inspired. It provides a forward-looking roadmap for researchers to build upon these advancements to create a more robust and transparent scientific ecosystem.

overview Preprint

Concerns about Replicability, Theorizing, Applicability, Generalizability, and Methodology across Two Crises in Social Psychology

This paper provides a historical comparison between the two major crises of confidence in social psychology, demonstrating that many current methodological and theoretical concerns were already voiced in the 1960s and 70s. By analyzing recurring themes through direct quotes, it highlights the persistent structural challenges the field faces in achieving replicable and generalizable findings.

Leonelli, S. (2023). Philosophy of open science. Cambridge University Press. http://philsci-archive.pitt.edu/id/eprint/21986

overview Paper

The Matthew Effect in Science

This foundational paper introduces the concept of the Matthew effect to describe how disproportionate credit is awarded to eminent scientists compared to less-known researchers. It provides a sociological analysis of how the reward system in science reinforces existing advantages and influences the visibility of scientific contributions.

overview Paper

The Matthew Effect in Science, II: Cumulative Advantage and the Symbolism of Intellectual Property

This sequel to the original Matthew effect paper expands the analysis to include the concept of cumulative advantage and the symbolic value of intellectual property. It examines how institutionalized rewards and systemic structures create self-reinforcing cycles of inequality in scientific recognition and resource allocation.

advocacy Paper

A manifesto for reproducible science

This influential paper argues for a systemic overhaul of the scientific process to prioritize research reliability and efficiency over the mere volume of discovery. It proposes a comprehensive set of reforms across study design, reporting standards, and institutional incentive structures to address the structural causes of the reproducibility crisis.

overview Paper

Psychology's Renaissance

This resource provides a historical and methodological review of the transformative period in experimental psychology known as the 'Renaissance,' detailing how concerns shifted from traditional publication bias to the prevalence of p-hacking. It synthesizes the specific methodological reforms and reporting standards that emerged from this period of self-reflection to improve the field's reliability.

overview Paper

Replicability, Robustness, and Reproducibility in Psychological Science

This article clarifies the conceptual distinctions and relationships between replicability, robustness, and reproducibility within the context of psychological research. It frames the replication crisis as a productive opportunity for innovation and theory development, emphasizing how these practices help identify the boundaries of scientific knowledge.

overview Paper

How scientists fool themselves – and how they can stop

This resource explores the psychological factors that lead researchers to see patterns in noise and provides an overview of how cognitive biases affect data analysis. It introduces several preventative practices, including the use of blind analysis and the adoption of more formal pre-analysis planning to mitigate the risk of false discoveries.

Devezer, B., & Buzbas, E. O. (2025). Minimum viable experiment to replicate. PhilSci Archive. https://philsci-archive.pitt.edu/24738/

practice/tools Paper

A community-sourced glossary of open scholarship terms

This comprehensive, community-sourced glossary provides clear definitions for a wide range of terms used in the open scholarship movement. It serves as a practical reference tool designed to lower the barrier to entry for newcomers and facilitate consistent communication across diverse academic stakeholders.

overview Preprint

Renovating the Theatre of Persuasion. ManyLabs as Collaborative Prototypes for the Production of Credible Knowledge

This paper analyzes the organizational shift toward large-scale "ManyX" collaborative consortia, examining how they implement strict formalization, bureaucratization, and procedural hygiene. It explores how these collaborative prototypes redistribute scientific agency and attempt to institutionalize the production of credible knowledge through standardization.

overview Paper

Metascience as a Scientific Social Movement

This paper frames the rise of metascience as a scientific social movement aimed at reforming the institutional structures and normative practices of research. It provides a sociological perspective on how reformers organize to challenge established scientific standards and advocate for systemic cultural change across academia.

critique Paper

Positive Deviance Underlies Successful Science: Normative Methodologies Risk Throwing out the Baby With the Bathwater

This resource critiques the current movement toward "normative methodologies" in psychology, arguing that an over-emphasis on preventing research failure may inadvertently suppress the creative deviance necessary for breakthrough discoveries. It highlights the risk that rigid procedural rules like preregistration and openness badges prioritize data hygiene over the development of successful scientific theories.

overview Paper

Implications of the Credibility Revolution for Productivity, Creativity, and Progress

This article explores how the credibility revolution in psychology affects broader scientific goals like productivity, creativity, and progress. It evaluates the shift toward higher evidence standards and preregistration, framing these changes as empirical questions that must be studied to understand their long-term impact on the health of the discipline.

▸ Scientific Misconduct: Fabrication and Falsification 10 / 10

In order to understand and weigh in on how the Reproducibility Crisis started, we first need to understand scientific misconduct, especially data fabrication and falsification. These practices erode trust in science and distort the research record. Fabrication involves inventing data, participants, or outcomes; falsification involves altering materials, methods, measurements, images, or reporting so that findings are misrepresented. Because intent to mislead is central, these acts are distinct from questionable research practices and from honest mistakes. Recognizing the role of misconduct is therefore essential for understanding how unreliable or non-replicable studies entered the literature and contributed to the broader crisis.

advocacy Paper

How Many Scientists Fabricate and Falsify Research? A Systematic Review and Meta-Analysis of Survey Data

This paper articulates the fundamental necessity of research integrity for maintaining trust within the scientific community and society at large. It argues for the systematic adoption of open science practices as the primary mechanism for ensuring transparency, accountability, and the ability to verify research findings.

evidence Paper

Misconduct accounts for the majority of retracted scientific publications

This study provides empirical evidence that misconduct, rather than honest error, is the primary driver of retractions in the biomedical and life sciences. It reveals that previous estimates significantly underreported fraud due to uninformative retraction notices and highlights a historical increase in the rate of articles retracted for fraud.

evidence Paper

Explanations of Research Misconduct, and How They Hang Together

This meta-analysis provides the first standardized empirical estimate of the prevalence of fabrication and falsification among scientists based on survey data. It offers a critical quantitative baseline for understanding the frequency of research misconduct and highlights the methodological challenges in measuring self-reported ethical breaches.

advocacy Paper

Signaling the trustworthiness of science

This resource argues for the adoption of explicit signals of trustworthiness by scientists and journals to better communicate adherence to scientific norms to both peers and the public. It proposes specific article-level signals, such as transparent reporting and evidence of replication, to reinforce the credibility of the scientific enterprise.

critique Letter

Reply to Kornfeld and Titus: No distraction from misconduct

This scholarly reply defends the necessity of focusing on scientific misconduct and its signals against criticisms that such focus might be a distraction. It emphasizes the importance of maintaining rigorous standards and transparent signaling to prevent the erosion of scientific integrity.

policies Paper

Promoting Research Integrity in <scp>A</scp>frica: An African Voice of Concern on Research Misconduct and the Way Forward

This document establishes the official national code of conduct for research integrity within the Dutch academic system, outlining the principles and standards expected of all researchers. It serves as a regulatory framework for defining misconduct and promoting ethical research practices across all disciplines in the Netherlands.

advocacy Paper

Stop ignoring misconduct

This resource advocates for a shift from passive observation to active intervention regarding scientific misconduct. It emphasizes the responsibility of the scientific community to implement robust mechanisms for identifying and penalizing fabrication and falsification.

advocacy Letter

Signaling the trustworthiness of science should not be a substitute for direct action against research misconduct

This article argues that symbolic indicators of research quality, such as transparency badges, are insufficient for addressing research misconduct. It advocates for structural changes and direct investigative actions as more effective tools for ensuring scientific integrity.

evidence Paper

Scientific Misconduct and the Myth of Self-Correction in Science

This seminal study presents empirical evidence on the prevalence of questionable research practices among thousands of early- and mid-career scientists. It demonstrates that behaviors compromising research integrity are far more common than blatant fraud, shifting the focus from individual "bad apples" to systemic pressures within the scientific environment.

overview Journal Article

Publication Pressure and Scientific Misconduct in Medical Scientists

This scoping review maps and categorizes existing guidance documents and practices used by research organizations and funders to promote research integrity. It identifies common themes and gaps in current integrity promotion strategies, providing a comprehensive catalog of how research integrity is institutionalized across various organizations.

▸ Questionable research practices & their prevalence 13 / 13

Questionable research practices are actions which researchers take to increase the probability of their desired result. They can be done consciously and unconsciously, distinguishing them from deliberate scientific misconduct, but still compromise research integrity since they can lead to misleading conclusions. Examples of such behaviors include p-hacking, selective reporting, and HARK-ing (Hypothesizing After the Results are Known). The ways in which researchers engage in behaviors and decision-making that increase the probability of their (consciously or unconsciously) desired result.

overview Paper

p-Hacking: Its Costs and When It Is Warranted

This paper provides a precise conceptual definition of p-hacking and uses decision theory to evaluate its epistemic and practical consequences. It uniquely explores the theoretical conditions under which certain analytic flexibilities might be warranted while maintaining a focus on the risks of false positives.

Gelman, A., & Loken, E. (2013). The garden of forking paths: Why multiple comparisons can be a problem, even when there is no “fishing expedition” or “p-hacking” and the research hypothesis was posited ahead of time. Unpublished manuscript. http://www.stat.columbia.edu/~gelman/research/unpublished/p_hacking.pdf

evidence Preprint

Meta-Research: How problematic citing practices distort science

This meta-research study uses case studies to demonstrate how distorted or problematic citation practices persist even within the field of research integrity itself. It illustrates the specific ways in which these habits can misrepresent epistemic foundations and undermine the reliability of scientific communication across disciplines.

evidence Paper

The extent and causes of academic text recycling or ‘self-plagiarism’

This study presents empirical findings from a survey of academic researchers to demonstrate how perceptions of departmental research climate influence the prevalence of misconduct. The results suggest that the local organizational environment and prevailing norms are significant predictors of research misbehavior, highlighting the need for culture-focused institutional interventions.

overview Paper

Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling

This resource examines the phenomenon of academic text recycling (self-plagiarism), analyzing its prevalence and the regulatory uncertainties revealed by high-profile misconduct cases. It contributes to the field by clarifying the conditions for fair reuse of one's own work and identifying the remaining gray areas in academic policy.

evidence Paper

With Low Power Comes Low Credibility? Toward a Principled Critique of Results From Underpowered Tests

Employing a survey design with truth-telling incentives, this paper provides empirical data on the widespread prevalence of questionable research practices among psychologists. It reveals that researchers are significantly more likely to admit to behaviors they perceive as defensible, providing insight into the normalization of problematic methodologies within the discipline.

critique Paper

A brief review of research that questions the impact of questionable research practices

This review synthesizes research that challenges the prevailing consensus on the prevalence and negative impact of questionable research practices (QRPs) such as p-hacking and HARKing. It suggests that these practices may not be the primary drivers of the replication crisis and are not inherently problematic in all scientific contexts.

critique Paper

The Costs of HARKing

This resource provides a philosophical and critical evaluation of the twelve proposed costs of 'Hypothesizing After the Results are Known' (HARKing). It argues that many of these costs are either conceptually flawed or lack empirical evidence, suggesting that the negative impact of HARKing may be overestimated.

overview Paper

What is critical metascience and why is it important?

This article defines and establishes the scope of 'critical metascience,' a reflexive field that questions the underlying assumptions and potential biases within the metascience movement itself. It provides a conceptual framework for how critical inquiry can complement empirical meta-research to ensure scientific reforms are robust and self-correcting.

evidence Paper

False-Positive Psychology

This study provides empirical evidence of publication bias by comparing result outcomes in Registered Reports against a random sample of standard psychological studies. It quantifies the gap in reported positive findings between these formats, demonstrating how result-blind peer review significantly mitigates the selective reporting of statistically significant results.

critique Paper

The natural selection of bad science

This paper employs an evolutionary model to argue that scientific incentive structures, which prioritize publication volume, lead to the "natural selection" of poor research methods. It demonstrates that low-quality practices will inevitably proliferate in a system that rewards productivity over methodological rigor, regardless of individual scientists' intentions.

practice/tools Paper

Degrees of Freedom in Planning, Running, Analyzing, and Reporting Psychological Studies: A Checklist to Avoid p-Hacking

This resource provides an extensive checklist of 34 specific researcher degrees of freedom that can lead to p-hacking across various stages of the research process. It serves as a practical tool for psychologists to preemptively identify and minimize opportunistic choices during study planning, data collection, analysis, and reporting.

evidence Paper

Prevalence of Research Misconduct and Questionable Research Practices: A Systematic Review and Meta-Analysis

This resource presents data from a large-scale survey of researchers to estimate the frequency of observed but unreported scientific misconduct. It highlights the systemic failure of institutional reporting mechanisms and emphasizes the need for better protections and incentives to ensure that integrity breaches are properly surfaced and addressed.

▸ Collection of large scale replications 29 / 29

This is a collection of large scale replications that have been conducted estimating the rate of reproducibility of entire (sub)disciplines, offering a big-picture view of replication efforts and the current state of replicability across fields.

evidence Paper

Do economists replicate?

This resource provides an empirical analysis of the prevalence and nature of replication studies within the field of economics. It investigates how often economists conduct replications and identifies the factors that influence the likelihood of these studies being performed and published.

evidence Paper

Quantitative Political Science Research Is Greatly Underpowered

evidence Paper

A Review of Multisite Replication Projects in Social Psychology: Is It Viable to Sustain Any Confidence in Social Psychology’s Knowledge Base?

This article synthesizes findings from 36 multisite replication projects in social psychology to evaluate the success rate of the field's knowledge base. It identifies a low rate of successful replications and explores the theoretical and practical implications of these outcomes for the validity of established social psychological findings.

evidence Paper

Evaluating replicability of laboratory experiments in economics

This resource presents the results of a large-scale project that conducted high-powered, pre-registered replications of 18 laboratory experiments in economics. It provides precise data on replication success rates and reveals that replicated effect sizes in economics are, on average, about two-thirds of the original reported sizes.

evidence Paper

Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015

This study systematically evaluates the replicability of social science experiments published in the high-impact journals Nature and Science. It offers critical evidence on the reliability of high-profile findings across various social science disciplines, comparing original effect sizes with those found in rigorous, high-powered replication attempts.

evidence Paper

Is Economics Research Replicable? Sixty Published Papers from Thirteen Journals Say "Usually Not"

This study empirically evaluates the replicability of economics research by attempting to reproduce results from 67 papers published in 13 reputable journals. It highlights a significant gap in the availability of data and code files between journals with and without mandatory sharing policies, ultimately finding that most results could not be successfully replicated.

evidence Preprint

The Prevalence of Direct Replication Articles in Top-Ranking Psychology Journals

This meta-research assesses the prevalence of direct replication studies within top-ranking psychology journals to determine how often the field publishes self-correcting research. The authors use keyword searches and manual verification to provide an empirical indicator of the actual value and incentives provided for replication work in the published literature.

evidence Paper

Many Labs 3: Evaluating participant pool quality across the academic semester via replication

This large-scale crowdsourced project investigates whether the timing of data collection within an academic semester impacts the quality of behavioral research and the replicability of known effects. By testing 10 effects across 20 different university participant pools, it provides evidence that time-of-semester variations have little impact on data quality or the robustness of experimental findings.

evidence Paper

Many Labs 5: Testing Pre-Data-Collection Peer Review as an Intervention to Increase Replicability

This study evaluates whether involving original authors and experts in a pre-data-collection peer review process can improve the success rates of replication attempts in psychology. It specifically examines 10 studies that previously failed to replicate, testing the hypothesis that replication failures are often due to protocol deficiencies rather than the absence of the original phenomenon.

Errington, T. M., Mathur, M., Soderberg, C. K., Denis, A., Perfito, N., Iorns, E., & Nosek, B. A. (2021). Investigating the replicability of preclinical cancer biology. ELife, 10. CLOCKSS. https://doi.org/10.7554/eLife.71601

advocacy Paper

Replications in agricultural economics

This resource outlines the necessity of replication studies within agricultural economics and proposes a disciplinary framework to overcome institutional and practical barriers. It specifically targets the disconnect between policy-relevant recommendations and the current lack of verified research, urging a cultural shift among researchers and journal editors.

Christopherson, C. D., Hildebrandt, L., Adeyemi Adetula, Wiggins, B. J., McLaughlin, H., Hurst, M. A., IJzerman, H., Levitan, C., Legate, N., Pazda, A., Kaylis Hase, VanBenschoten, A., Fallon, M., LePine, S., Gervais, H., Lazarevic, L., Chartier, C. R., Corker, K. S., France, H., … Wagge, J. (2013). Collaborative Replications and Education Project (CREP). OSF. https://doi.org/10.17605/OSF.IO/WFC6U

evidence Paper

Estimating the Prevalence of Transparency and Reproducibility-Related Research Practices in Psychology (2014–2017)

This empirical study assesses the adoption of open science practices in psychology by manually auditing a random sample of published literature from 2014 to 2017. It establishes a quantitative baseline for tracking the impact of transparency and reproducibility initiatives within the discipline.

advocacy Paper

Replicating Empirical Research In Behavioral Ecology: How And Why It Should Be Done But Rarely Ever Is

This article highlights the historical neglect of replication in behavioral ecology and argues for its restoration as a fundamental scientific requirement. It identifies specific cultural and institutional obstacles, such as publication bias and editorial disdain, explaining why the field must adopt a more rigorous approach to verifying empirical evidence.

evidence Paper

Rate and success of study replication in ecology and evolution

This study measures the actual frequency of replication in ecology and evolutionary biology journals, identifying a strikingly low rate of published replication attempts. By providing these empirical figures, it exposes the disparity between scientific ideals and the actual publishing practices within these specific biological fields.

evidence Paper

Investigating Variation in Replicability

This foundational study provides empirical data on the replicability of 13 psychological effects by testing them across 36 independent labs and diverse participant pools. It contributes to the understanding of reproducibility by demonstrating that most of the tested effects were consistently reproducible regardless of whether the research was conducted in a lab, online, or in different geographical locations.

evidence Paper

Many Labs 2: Investigating Variation in Replicability Across Samples and Settings

This large-scale empirical study examines the replicability of 28 psychological findings across 125 diverse samples from 36 countries to investigate how variation in settings affects results. It provides critical meta-research evidence suggesting that the primary determinant of replication success is the strength of the original effect rather than the specific sample or context.

evidence Paper

Assessing the replication landscape in experimental linguistics

This research quantifies the prevalence of replication studies within the field of experimental linguistics by analyzing publication trends across nearly 100 journals. It provides a baseline assessment of the "replication gap" in the discipline, highlighting how infrequently direct replications are published compared to novel confirmatory research.

Makel, M. C., & Plucker, J. A. (2014). Facts Are More Important Than Novelty. Educational Researcher, 43(6), 304–316. https://doi.org/10.3102/0013189X14545513

evidence Paper

Replications in Psychology Research

This study provides a comprehensive historical audit of replication practices in psychology since 1900, measuring the frequency and nature of replication attempts across a century of literature. It establishes that a very small percentage of the field's output consists of replications and examines the publication outcomes of those that are attempted.

evidence Paper

Replication of Special Education Research

This study provides empirical data on the prevalence and success of replications within the field of special education by reviewing the complete publication history of 36 specialized journals. It identifies a low replication rate of 0.5% and examines authorship history to assess the status of scientific rigor in the discipline.

ManyPrimates. (n.d.). ManyPrimates. ManyPrimates. https://manyprimates.github.io/

evidence Paper

Replication in Second Language Research: Narrative and Systematic Reviews and Recommendations for the Field

This resource presents a systematic review of 67 self-labeled replication studies in second language (L2) research to estimate publication rates and analyze study characteristics. It also provides a narrative review of field-specific challenges, offering recommendations to bridge the gap between the perceived importance of replication and actual practice.

evidence Paper

Replication in criminology: A necessary practice

This study conducts a content analysis of leading criminology journals to measure how often replication studies are published compared to other social and natural sciences. The findings reveal a replication rate of approximately 2% in criminology, highlighting the need for more frequent confirmation of research results in the field.

evidence Paper

Replication studies in economics—How many and which papers are chosen for replication, and why?

This research investigates the frequency and types of replication studies published in the top 50 economics journals over a 40-year period. It distinguishes between narrow and scientific replications and identifies that higher-impact articles by authors from leading institutions are more likely to be selected for replication.

(2012). An Open, Large-Scale, Collaborative Effort to Estimate the Reproducibility of Psychological Science. Perspectives on Psychological Science, 7(6), 657–660. https://doi.org/10.1177/1745691612462588

evidence Paper

Estimating the reproducibility of psychological science

This foundational meta-research provides empirical evidence regarding the reproducibility of psychological science by attempting to replicate 100 experimental and correlational studies. The findings demonstrate a significant decline in effect sizes and statistical significance rates compared to the original publications.

evidence Paper

RETRACTED ARTICLE: High replicability of newly discovered social-behavioural findings is achievable

This study demonstrates that high replication rates are achievable for novel social-behavioural findings when laboratories utilize rigour-enhancing practices like preregistration, large sample sizes, and methodological transparency. It provides empirical evidence that the low reproducibility observed in other meta-research efforts may be a result of suboptimal methods rather than an inherent unreliability of the phenomena themselves.

evidence Paper

Contextual sensitivity in scientific reproducibility

This study presents empirical evidence on the role of "contextual sensitivity" in the reproducibility of psychological research by analyzing findings from the Reproducibility Project: Psychology. It demonstrates that findings judged to depend heavily on specific social, settings, or temporal contexts are significantly less likely to replicate than more generalizable findings.

▸ Proposed science improvement initiatives on statistics, measurement, teaching, data sharing, code sharing, pre-registration, & replication 22 / 22

Published checklists and other resources that can be used to shift behavior more toward improved practices.

critique Paper

Statistical Nonsignificance in Empirical Economics

This article critiques the standard practice in empirical economics of prioritizing statistically significant rejections over non-significant findings. It demonstrates that in the context of large economic datasets, the failure to reject a point null is often more scientifically informative than a rejection, challenging the traditional hierarchy of evidence.

advocacy Letter

Towards a culture of open scholarship: the role of pedagogical communities

This resource argues for the critical role of teaching and mentorship in establishing a sustainable culture of open scholarship and research integrity. It specifically calls on institutions and stakeholders to integrate open science principles into pedagogical practices to ensure that the next generation of researchers is equipped with the skills for reproducible research.

teaching/training Preprint

Introducing a Framework for Open and Reproducible Research Training (FORRT)

This publication introduces a pedagogical framework and a community-led initiative designed to help educators integrate open scholarship into their teaching. It aims to bridge the gap between the researcher-led adoption of open practices and the lack of structured training available for the next generation of scientists.

advocacy Paper

Behavioural science is unlikely to change the world without a heterogeneity revolution

This article argues that the impact of behavioral science on real-world problems is hindered by a neglect of treatment effect heterogeneity. It advocates for a shift in research priorities toward understanding how and why effects vary across different contexts and populations, proposing a framework to improve the generalizability of findings.

overview Other

Powering Reproducible Research

This resource highlights the fundamental importance of statistical power in ensuring the reproducibility of research results across scientific disciplines. It explains how low power not only reduces the chance of detecting true effects but also increases the probability that reported significant results are false positives.

overview Paper

Seven Easy Steps to Open Science

critique Paper

The case for formal methodology in scientific reform

The authors critique the current wave of scientific reform for being primarily heuristic and lacking formal mathematical rigor, which leads to over-generalized solutions. They propose a structured five-step framework for developing formal methodology to ensure that scientific reforms are theoretically sound and practically effective.

overview Paper

Open science interventions to improve reproducibility and replicability of research: a scoping review

This scoping review provides a comprehensive synthesis of existing literature regarding the effectiveness of open science interventions designed to improve research reproducibility. It systematically categorizes these practices and identifies specific gaps where empirical evidence of their actual impact is still needed.

critique Paper

Practical Methodological Reform Needs Good Theory

This article argues that the methodological reform movement in psychology currently lacks sufficient theoretical grounding, which may limit the long-term effectiveness of practical improvements. It provides established theoretical frameworks from other disciplines to help model the research process and guide more robust metascience initiatives.

practice/tools Paper

Valid replications require valid methods: Recommendations for best methodological practices with lab experiments.

This resource provides actionable methodological recommendations for conducting lab experiments to ensure they serve as a solid foundation for valid replications. It highlights specific practices in experimental design and implementation that are essential for producing reliable and reproducible findings.

evidence Paper

Publication and other reporting biases in cognitive sciences: detection, prevalence, and prevention

This publication examines the prevalence and impact of publication and reporting biases within the cognitive sciences, offering specific methods for their detection. It synthesizes evidence of how these biases distort the literature and suggests preventive strategies to improve the reliability of reported findings.

practice/tools Paper

Easing Into Open Science: A Guide for Graduate Students and Their Advisors

This guide offers a structured roadmap for graduate students and advisors to incrementally adopt open science practices by categorizing them by difficulty level. It provides practical advice on eight specific behaviors, ranging from starting journal clubs to submitting registered reports, to help researchers transition toward more transparent workflows.

Krähmer, D., Schächtele, L., & Auspurg, K. (2026). Code sharing and reproducibility in survey-based social research: evidence from a large-scale audit. Royal Society Open Science, 13(3). https://doi.org/10.1098/rsos.251997

evidence Paper

Many Labs 2: Investigating Variation in Replicability Across Samples and Settings

practice/tools Paper

Seven steps toward transparency and replicability in psychological science.

This resource provides a practical framework consisting of seven actionable steps that psychological researchers can implement to improve the transparency and reproducibility of their work. It offers a structured guide for transitioning to open science workflows, covering specific practices like preregistration and public data sharing.

advocacy Paper

A manifesto for reproducible science

advocacy Editorial

Checklists work to improve science

This resource makes the case for the adoption of structured checklists as a fundamental tool to ensure procedural consistency and reduce human error in research. It highlights how these simple interventions can effectively lead to more rigorous methodology and higher standards of transparent reporting across various scientific domains.

advocacy Paper

The Reproducibility Crisis in Science: A Statistical Counterattack

This resource identifies the lack of advanced statistical and analytical skills as a primary driver of the reproducibility crisis in science. It calls for a systemic shift in scientific training, arguing that rigorous statistical education is the most effective tool for ensuring research findings are replicable.

teaching/training Paper

Embedding open and reproducible science into teaching: A bank of lesson plans and resources.

This resource provides a curated collection of lesson plans and pedagogical materials designed to help educators integrate open scholarship into undergraduate and postgraduate curricula. It addresses the barrier of time and resource constraints by offering ready-to-use tools and sharing existing educational resources with the wider academic community.

overview Paper

Teaching open and reproducible scholarship: a critical review of the evidence base for current pedagogical methods and their outcomes

This resource provides a critical synthesis of existing research regarding the impact of open science training on undergraduate and postgraduate student outcomes. It evaluates the effectiveness of various pedagogical methods and identifies current gaps in the evidence base for teaching reproducible research practices.

critique Paper

Beyond Statistical Ritual: Theory in Psychological Science

This article critiques the current state of psychological science by arguing that an over-reliance on statistical significance has led to a decline in theoretical rigor. It suggests that the discipline’s focus on methodological fixes is insufficient without addressing the underlying 'theory crisis' where statistical rituals have replaced meaningful theoretical development.

practice/tools Preprint

A Guide for Social Science Journal Editors on Easing into Open Science

This guide provides actionable steps for social science journal editors to implement open science policies and practices within their publications. It consolidates existing resources into a structured pathway to help editors navigate the transition toward incentivizing transparency and data sharing.

▸ Ethical considerations for improved practices 13 / 13

Engaging in Open and Reproducible Science practices comes with ethical challenges that need to be sensitively navigated (e.g. when sharing data openly).

critique Paper

Open Science and Feminist Ethics: Promises and Challenges of Open Access

This article utilizes feminist ethics to critically evaluate the promises and challenges of the open access movement, particularly regarding power dynamics and data privacy. It offers an updated ethical framework to help researchers and policymakers navigate the intersection of transparency, social justice, and participant protection.

evidence Paper

The Matthew effect in science funding

This empirical study provides evidence for the "Matthew effect" in science funding by analyzing grant proposals and review scores to show how early success disproportionately benefits certain researchers. It demonstrates how funding gaps widen between winners and losers even when their initial qualifications are nearly identical.

overview Paper

Open Science: Challenges, Possible Solutions and the Way Forward

This review explores the multifaceted concept of open science, examining the barriers to accessible scientific communication such as high publication costs. It specifically evaluates the challenges of the open-access model and suggests various pathways for maintaining the accessibility of research outputs.

teaching/training Paper

How (and Whether) to Teach Undergraduates About the Replication Crisis in Psychological Science

This resource provides a validated one-hour lecture design and evaluation for introducing undergraduates to the replication crisis in psychology. It demonstrates that teaching these concepts can maintain student trust in science while improving their understanding of methodological improvements.

overview Paper

Academic Research in the 21st Century: Maintaining Scientific Integrity in a Climate of Perverse Incentives and Hypercompetition

evidence Paper

The Economic Impacts of Open Science: A Rapid Evidence Assessment

This study provides a systematic synthesis of research on the economic impacts of open science, identifying both positive and negative effects and the mechanisms through which they occur. It highlights the methodological challenges in tracking the usage of open outputs and offers evidence-based insights into how these economic benefits can be maximized.

Jacobs, A. M., Büthe, T., Arjona, A., Arriola, L. R., Bellin, E., Bennett, A., Björkman, L., Bleich, E., Elkins, Z., Fairfield, T., Gaikwad, N., Greitens, S. C., Hawkesworth, M., Herrera, V., Herrera, Y. M., Johnson, K. S., Karakoç, E., Koivu, K., Kreuzer, M., … Yashar, D. J. (2021). The Qualitative Transparency Deliberations: Insights and Implications. Perspectives on Politics, 19(1), 171–208. https://doi.org/10.1017/S1537592720001164

overview Paper

The concept of risk and responsible conduct of research

This resource explores how the definition of risk informs the ethical standards required for the responsible conduct of research. It provides a conceptual analysis of how researchers can navigate the balance between scientific advancement and their ethical obligations to minimize harm.

overview Paper

Fostering openness in open science: An ethical discussion of risks and benefits

This article examines the ethical tension between the benefits of open science—such as enhanced reliability and collaboration—and the potential risks associated with sharing sensitive information. It specifically highlights the 'dual-use dilemma' regarding security, confidentiality, and privacy concerns in an open research environment.

critique Paper

The challenges of open data sharing for qualitative researchers

This resource critiques the application of universal open science mandates to qualitative research, arguing that sharing full datasets for replication is epistemologically and ethically problematic. It highlights how standardized requirements fail to account for methodological differences and the specific risks involved in de-identifying complex qualitative narratives.

Lupia, A. (2020). Practical and Ethical Reasons for Pursuing a More Open Science. PS: Political Science & Politics, 54(2), 301–304. https://doi.org/10.1017/S1049096520000979

critique Editorial

Editorial Essay: The Tumult over Transparency: Decoupling Transparency from Replication in Establishing Trustworthy Qualitative Research

This editorial warns against the uncritical transfer of transparency and replication standards from psychology to qualitative management research. It argues for decoupling transparency from replication, suggesting that while transparency is necessary for trust, replication is often a poor fit for qualitative research goals.

critique Paper

Reflection over compliance: Critiquing mandatory data sharing policies for qualitative research

This resource critiques the 'Mandatory Inclusion of Raw Data' (MIRD) model, arguing that universal data-sharing mandates fail to account for the unique ethical and epistemological challenges of qualitative research. It provides a series of reflective questions to help researchers in health psychology and related fields navigate these policies while protecting participant confidentiality and methodological integrity.

▸ Ongoing debates (e.g., incentives for and against open science practices) 33 / 33

Open Science is not a monolith, and continued scrutiny of the proposed practices and reforms can be of value - whether to understand why there is resistance (and how to combat anti-open arguments) as well as pushing us to evaluate the potential positive and negative impacts of reforms.

critique Paper

Open Science Isn't Always Open to All Scientists

This article critiques the implementation of open science, highlighting how standard practices may inadvertently exclude researchers who lack specific resources, institutional support, or geographical advantages. It argues that for open science to be truly inclusive, the movement must address the diverse socio-economic and systemic challenges faced by scientists globally.

critique Preprint

Revisiting the replication crisis without false positives

This paper challenges the common assumption that the replication crisis is primarily a result of false positives caused by questionable research practices. By proposing alternative meta-scientific models, the authors demonstrate that low replicability can be explained by factors other than false positives, calling for a more nuanced understanding of the crisis.

overview Paper

Does Sociology Need Open Science?

This article provides a disciplinary overview of open science's relevance to sociology, connecting modern transparency standards with classical sociological theory. It serves as an introductory guide for sociologists to understand how open science principles intersect with the field's specific epistemological and methodological traditions.

critique Paper

Getting ontologically serious about the replication crisis in psychology.

This article applies analytic philosophy to argue that the replication crisis in psychology is deeply rooted in ontological confusion rather than just methodological failures. It suggests that until the field addresses the nature of psychological phenomena themselves, epistemological and procedural reforms will remain insufficient to solve the crisis.

advocacy Paper

Open is not enough

This resource uses the experiences of the high-energy physics community to argue that simple data openness is insufficient for ensuring true computational reproducibility. It advocates for the adoption of more comprehensive practices, such as preserving the entire computational environment and workflow, to make research findings genuinely verifiable by others.

advocacy Paper

An Agenda for Open Science in Communication

This paper outlines a seven-point agenda for integrating open science practices into communication research to address the discipline's replication crisis. It advocates for specific shifts in research culture, such as the publication of materials and code, to enhance the transparency and generalizability of communication studies.

Drummond, C. (2017). Reproducible research: a minority opinion. Journal of Experimental & Theoretical Artificial Intelligence, 30(1), 1–11. https://doi.org/10.1080/0952813X.2017.1413140

evidence Paper

US studies may overestimate effect sizes in softer research

This study empirically investigates the "US effect" by analyzing over a thousand primary outcomes from meta-analyses in genetics and psychiatry to demonstrate how institutional pressures can lead to overestimated effect sizes. It specifically identifies how productivity-driven career systems in the United States may incentivize more extreme research findings compared to other regions.

critique Paper

Is science really facing a reproducibility crisis, and do we need it to?

This resource disputes the prevailing "crisis" narrative in science, arguing that the focus on widespread unreliability is based on a misinterpretation of current evidence. It proposes a more constructive framework that views recent methodological shifts as a positive evolution and empowerment of the scientific community rather than a sign of systemic failure.

evidence Paper

The Economic Impacts of Open Science: A Rapid Evidence Assessment

critique Paper

Open Science for private Interests? How the Logic of Open Science Contributes to the Commercialization of Research

This article critically examines the intersection of the Open Science movement and the commercialization of research within the private sector. It argues that while Open Science promotes transparency and accountability, its current implementation may inadvertently serve private corporate interests rather than addressing social justice or public epistemic needs.

critique Paper

Science of psychological phenomena and their testing.

This paper argues that the 'replication crisis' in psychology stems from a misunderstanding of the inherent stability and variability of psychological phenomena rather than a failure of scientific rigor. It critiques the use of replication as a primary gatekeeper for scientific truth, suggesting that the field should instead focus on identifying stable patterns within variable data.

critique Paper

Campbell’s Law Explains the Replication Crisis: Pre-Registration Badges Are History Repeating

This article critiques the implementation of preregistration badges by framing them through the lens of Campbell’s Law, suggesting that high-stakes indicators can corrupt the very research processes they are meant to improve. It warns that mandating such practices may inadvertently prioritize the appearance of rigor over the actual quality of the science.

advocacy Paper

How to Produce, Identify, and Motivate Robust Psychological Science: A Roadmap and a Response to Vize et al.

This resource proposes a three-part vision for improving the robustness of psychological science by prioritizing methodological quality—such as sample size and valid measurement—over administrative mandates. It advocates for cultivating these foundational research practices as a more effective and less harmful path to scientific progress than relying on procedural requirements like preregistration.

advocacy Preprint

Open Science as Confused: Contradictory and Conflicting Discourses in Open Science Guidance to Researchers

This formal comment advocates for the integration of gender and diversity considerations into researcher assessment frameworks to improve institutional integrity and representation. It specifically argues that reshaping assessment criteria is a necessary step in fostering an inclusive and responsible research environment.

critique Paper

Low replicability can support robust and efficient science

This publication uses computational modeling of scientific communities to argue that a single-minded focus on high replicability may actually impede scientific progress and efficiency. It suggests that a certain level of non-replicable findings is an acceptable trade-off in a system that prioritizes discovery and the exploration of novel hypotheses.

advocacy Paper

Replication is relevant to qualitative research

This paper argues for the relevance and value of replication within qualitative research, suggesting it can address issues of transparency and transferability. It seeks to promote the adoption of replication as a fundamental building block of scholarship even in methodologies where it has traditionally been ignored.

critique Paper

The quantitative paradigm and the nature of the human mind. The replication crisis as an epistemological crisis of quantitative psychology in view of the ontic nature of the psyche

This paper frames the replication crisis in psychology as a fundamental epistemological mismatch between the complex nature of the human psyche and the quantitative methods used to measure it. It moves beyond statistical explanations to argue that the crisis stems from underlying philosophical and ontological assumptions that remain largely unaddressed in the field.

critique Paper

Conflicting Results and Statistical Malleability: Embracing Pluralism of Empirical Results

This article discusses how researcher degrees of freedom and methodological malleability naturally result in conflicting empirical findings, even in the absence of malpractice. It advocates for embracing a pluralistic view of results, where divergent outcomes are understood as a function of different but plausible design choices.

critique Paper

Breaking free

This resource provides a critical perspective on preregistration, arguing that it fails to solve the problems it targets while introducing new barriers for under-resourced researchers. It specifically highlights how the practice may stifle scientific creativity and impose disproportionate costs on junior scholars without guaranteeing ethical or novel outcomes.

overview Editorial

Reality check on reproducibility

This article offers a situational assessment of the reproducibility debate, aiming to provide a realistic perspective on the scale of the challenges facing scientific research. It contributes to the literature by encouraging a grounded evaluation of research practices to determine if the perceived 'crisis' matches empirical reality.

overview Paper

Is the Replicability Crisis Overblown? Three Arguments Examined

This publication examines and rebuts three common arguments used to downplay the significance of the replicability crisis, such as the sufficiency of low alpha levels and the role of conceptual replications. It clarifies statistical misunderstandings that lead some researchers to underestimate the prevalence of false-positive results in published literature.

critique Paper

Scandal in scientific reform: the breaking and remaking of science

This perspective analyzes how the narrative of 'scandal' and the claim that 'science is broken' have been used as rhetorical tools to drive the scientific reform movement. It highlights the power of these tactics in catalyzing institutional change while cautioning against their potential to undermine public trust and damage scientific careers.

critique Paper

Is There a Reproducibility Crisis? On the Need for Evidence-based Approaches

Through a systematic analysis of a UK parliamentary report, this resource argues that the label of a 'reproducibility crisis' lacks sufficient empirical evidence. It advocates for scientific reforms to be based on documented findings and evidence-based approaches rather than reacting to unverified claims of systemic failure.

critique Paper

Preregistration Is Neither Sufficient nor Necessary for Good Science

This article identifies several limitations and potential adverse effects of implementing a systematic preregistration system, specifically within the context of consumer research. It challenges the assumption that preregistration is a necessary condition for high-quality science by outlining implementation challenges and epistemological drawbacks.

critique Editorial

Editorial Essay: The Tumult over Transparency: Decoupling Transparency from Replication in Establishing Trustworthy Qualitative Research

Ràfols, I. (2025). Rethinking open science: Towards care for equity and inclusion. Global Dialogue. https://globaldialogue.isa-sociology.org/articles/rethinking-open-science-towards-care-for-equity-and-inclusion

critique Paper

A brief review of research that questions the impact of questionable research practices

critique Paper

Preregistration does not improve the transparent evaluation of severity in Popper’s philosophy of science or when deviations are allowed

This paper offers a philosophical critique of preregistration by demonstrating its limited utility within Popper’s theory-centric philosophy of science. It specifically argues that concerns regarding Type I error rate inflation are irrelevant in Popperian contexts and that preregistration fails to enhance the evaluation of test severity when deviations from plans occur.

critique Paper

The replication crisis is less of a “crisis” in Lakatos’ philosophy of science than it is in Popper’s

This article contrasts Popper’s and Lakatos’ philosophies to argue that the perception of a "replication crisis" is largely dependent on the philosophical framework one adopts. It suggests that moving toward a Lakatosian perspective can mitigate the sense of crisis by reframing how unexpected replication failures are interpreted within scientific programs.

critique Paper

There is no theory crisis in psychological science.

This article challenges the prevailing "theory crisis" narrative in psychology by arguing that theoretical limitations are persistent and rooted in the inherent complexity of human behavior. It provides a historical perspective that suggests current issues are not a temporary crisis but a fundamental, long-standing characteristic of the social and behavioral sciences.

critique Paper

Preregistration is not a panacea, but why? A rejoinder to “infusing preregistration into tourism research”

This rejoinder critiques the push for preregistration in tourism research by identifying it as an incomplete solution that may not address the field's underlying methodological challenges. It provides a cautionary perspective on the limitations of adopting open science practices without considering discipline-specific nuances and practical barriers.

overview Website

Epistemic replicability: A primer for psychological science and beyond

This resource introduces the concept of epistemic replicability, providing a conceptual framework for understanding the theoretical foundations of scientific reproducibility. It bridges the gap between statistical replication and broader knowledge accumulation across diverse scientific fields.

2 Conceptual and Statistical Knowledge

7 sub-clusters · 82 references

Attainment of a grounding in fundamental statistics, measurement, and its implications encompassing conceptual knowledge, application, interpretation and communication of statistical analyses. There are 8 sub-clusters which aim to further parse the learning and teaching process:

▸ Effect sizes, statistical power, simulations, & confidence intervals. 19 / 19

Statistics are more than p-values and we need to use other benchmarks to determine the statistical and practical relevance of an effect. Emphasizes effect size, confidence intervals, power, and simulations to design adequately powered studies and communicate practical significance.

evidence Editorial

Sample size estimation revisited

This editorial examines the prevalence and reproducibility of sample size estimations within the Journal of Sports Sciences, revealing that only a small minority of studies provide sufficient detail for calculation reproduction. It highlights a critical gap between established editorial guidelines and the actual reporting practices of researchers in the field.

practice/tools Editorial

Power, precision, and sample size estimation in sport and exercise science research

This resource offers a technical guide for sport and exercise scientists on determining sample sizes using both frequentist power and precision-based approaches. It acts as a practical primer for researchers to justify their study designs and align with rigorous statistical standards during the submission process.

evidence Paper

Quantitative Political Science Research Is Greatly Underpowered

advocacy Paper

Behavioural science is unlikely to change the world without a heterogeneity revolution

practice/tools Paper

Power Analysis and Effect Size in Mixed Effects Models: A Tutorial

This tutorial addresses the difficulty of conducting power analysis for experimental designs that include both participant and stimulus samples, common in cognitive psychology. It provides researchers with practical methods and literature reviews to accurately estimate power and effect sizes when using mixed-effects models.

practice/tools Preprint

Accuracy in Parameter Estimation and Simulation Approaches for Sample Size Planning with Multiple Stimuli

This resource introduces simulation-based approaches and Accuracy in Parameter Estimation (AIPE) as alternatives for sample size planning in research studies with multiple stimuli. It provides tools to determine necessary sample sizes for precise parameter estimation when traditional power formulas are insufficient or inapplicable to complex designs.

evidence Paper

Power failure: why small sample size undermines the reliability of neuroscience

This study presents a meta-research analysis quantifying the prevalence of low statistical power across the neuroscience literature and its role in undermining reproducibility. It demonstrates how underpowered studies lead to inflated effect sizes and waste resources, calling for systemic changes in how neuroscience research is conducted and reported.

Caldwell, A. R., Lakens, D., Parlett‑Pelleriti, C. M., Prochilo, G., & Aust, F. (2022). Power analysis with Superpower. https://aaroncaldwell.us/SuperpowerBook/

practice/tools Paper

Understanding Mixed-Effects Models Through Data Simulation

This tutorial provides a practical guide to using data simulation to better understand and interpret linear mixed-effects models that include random effects for both subjects and stimuli. By walking through R code and parameter interpretation, it helps researchers build intuition for complex models and correctly apply them to their own experimental data.

overview Paper

Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations

This resource clarifies common misconceptions regarding frequentist statistical indicators by identifying and correcting twenty prevalent misinterpretations of p-values, confidence intervals, and power. It serves as a rigorous pedagogical guide to help researchers avoid incorrect shortcut definitions that lead to invalid scientific conclusions.

practice/tools Paper

Conducting Simulation Studies in the R Programming Environment

This resource provides a practical tutorial for using the R programming environment to conduct simulation studies, making these techniques accessible to researchers without advanced programming backgrounds. It includes annotated code to help users estimate statistical power and assess the appropriateness of various analytical methods for their specific research questions.

overview Paper

Heterogeneity in effect size estimates

This resource proposes a framework that decomposes heterogeneity in effect sizes into three distinct sources: population, design, and analytical variation. It provides a theoretical foundation for understanding how these different forms of uncertainty limit the generalizability of research findings and affect the cumulative probability that a tested hypothesis is true.

practice/tools Paper

Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs

This primer provides step-by-step instructions for calculating and reporting various effect size measures for t-tests and ANOVAs, specifically distinguishing between metrics like Cohen’s d and partial eta-squared. It emphasizes how transparent effect size reporting directly enables cumulative science by facilitating accurate a-priori power analyses and the inclusion of findings in meta-analyses.

overview Paper

Sample Size Justification

This article details six different approaches for justifying sample sizes in quantitative research, moving beyond simple power analysis to include strategies based on accuracy planning, resource constraints, and point estimates. It provides researchers with a structured decision-making framework and standardized vocabulary to transparently communicate the rationale for their data collection plans.

evidence Paper

With Low Power Comes Low Credibility? Toward a Principled Critique of Results From Underpowered Tests

practice/tools Paper

Reporting effect sizes in original psychological research: A discussion and tutorial.

This tutorial offers practical guidance for psychological researchers on the reporting and interpretation of effect sizes and their associated confidence intervals. It specifically emphasizes the importance of unstandardized effect sizes and provides recommendations for selecting measures that best address specific research questions.

practice/tools Paper

Safeguard Power as a Protection Against Imprecise Power Estimates

This article introduces 'safeguard power analysis,' a practical method for sample size planning that accounts for the inherent uncertainty in effect size estimates. By using the lower bound of a confidence interval around an effect size, the tool helps researchers avoid the common problem of designing underpowered studies based on potentially inflated initial results.

critique Paper

Evaluating Research in Personality and Social Psychology: Considerations of Statistical Power and Concerns About False Findings

This article critiques the application of False Finding Rate (FFR) calculations as a primary criterion for evaluating the quality of research in personality and social psychology. It argues that the assumptions underlying these power-based evaluations often fail to reflect the practical realities of the discipline, potentially leading to the unfair dismissal of valid research.

evidence Paper

On the importance of modeling the invisible world of underlying effect sizes

This resource uses formal modeling and simulations to demonstrate that headline replication rates cannot be meaningfully interpreted without considering the underlying distribution of true effect sizes and statistical power. It provides a meta-research framework for understanding how observed replication failures can emerge from the mathematical properties of original study designs rather than solely from questionable research practices.

▸ Exploratory and confirmatory analyses 8 / 8

Confirmatory analyses test a priori hypotheses against a pre-specified analysis plan (ideally preregistered/Registered Report); any deviations are documented. Exploratory analyses probe patterns, generate hypotheses, and build models after seeing the data.

advocacy Book

The Seven Deadly Sins of Psychology

Feest, U., & Devezer, B. (2025). Toward a more accurate notion of exploratory research (and why it matters). PhilSci Archive. https://philsci-archive.pitt.edu/24482/

critique Paper

A critique of using the labels confirmatory and exploratory in modern psychological research

This paper critiques the binary categorization of research as either exploratory or confirmatory, arguing that these labels are too simplistic for modern psychological research involving complex statistical models. It highlights how these terms can mask the nuanced relationship between theory and data analysis, potentially obstructing methodological progress.

Lin, W., & Green, D. P. (2016). Standard Operating Procedures: A Safety Net for Pre-Analysis Plans. Political Science and Politics, 49(3), 495–500. https://doi.org/10.1017/S1049096516000810

critique Paper

Exploratory hypothesis tests can be more compelling than confirmatory hypothesis tests

This paper challenges the prevailing hierarchy that favors confirmatory testing over exploratory testing, arguing that the latter can often produce more compelling scientific insights. It provides a theoretical counterpoint to the idea that preregistration is the primary determinant of research quality or certainty.

critique Paper

Arrested Theory Development: The Misguided Distinction Between Exploratory and Confirmatory Research

This resource argues that psychology’s replicability crisis stems from "flexible theories" rather than a failure to distinguish between exploration and confirmation. It critiques current trends that prioritize methodological fixes like preregistration over the fundamental need for developing rigorous, "hard to vary" theories.

advocacy Paper

The Creativity-Verification Cycle in Psychological Science: New Methods to Combat Old Idols

This article advocates for the adoption of preregistration in psychological science as a necessary safeguard against pervasive cognitive biases like hindsight and confirmation bias. It argues for a clear structural separation between the 'creativity' of exploratory data analysis and the 'verification' of confirmatory hypothesis testing.

advocacy Paper

An Agenda for Purely Confirmatory Research

This resource advocates for the adoption of purely confirmatory research designs to prevent researchers from fine-tuning analyses to fit observed data. It highlights how the lack of pre-commitment to specific statistical tests undermines the validity of research claims in psychology.

▸ Limitations and benefits of NHST, Bayesian & Likelihood approaches. 11 / 11

Next to frequentist statistics, there are other quantitative approaches, each with different assumptions and goals. This subcluster summarizes benefits and limitations of each one.

advocacy Paper

The New Statistics

This publication advocates for the adoption of "the new statistics," urging researchers to move away from null-hypothesis significance testing in favor of estimation and effect size reporting. It presents a clear case for research integrity reforms, including the prespecification of studies and the active encouragement of replication to improve literature reliability.

practice/tools Paper

How to become a Bayesian in eight easy steps: An annotated reading list

This resource provides a curated and annotated reading list designed to guide researchers through the transition from frequentist to Bayesian statistical thinking. It offers a structured pathway for self-study by identifying foundational texts and explaining their significance in mastering Bayesian inference.

overview Paper

Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations

overview Paper

Bayes Factors

This paper provides a comprehensive review of the Bayes factor as a tool for quantifying scientific evidence in favor of a hypothesis. It discusses the practical application of Bayesian hypothesis testing across various research contexts and provides guidelines for interpreting the strength of evidence.

practice/tools Paper

Using Bayes factor hypothesis testing in neuroscience to establish evidence of absence

This resource demonstrates how Bayesian hypothesis testing can be applied specifically within neuroscience to distinguish between inconclusive results and genuine evidence for the absence of an effect. It provides a practical alternative to frequentist methods, which are inherently unable to provide statistical evidence in support of a null hypothesis.

overview Paper

Scientific method: Statistical errors

This resource provides an accessible overview of the common misinterpretations of p-values and how the rigid reliance on statistical significance thresholds fuels the reproducibility crisis. It explains the mathematical vulnerability of 'near-significant' results and suggests moving toward more nuanced statistical reporting that avoids binary thinking.

advocacy Paper

The Creativity-Verification Cycle in Psychological Science: New Methods to Combat Old Idols

critique Paper

Statistical Nonsignificance in Empirical Economics

policies Paper

Interrogating the “cargo cult science” metaphor

The Bonn PRINTEGER Statement provides a set of guidelines for research organizations to strengthen integrity by focusing on institutional responsibilities and the daily work environment. It contributes actionable advice on how management and governance can be adapted to proactively address the ethical challenges researchers face on the work-floor.

practice/tools Paper

Valid replications require valid methods: Recommendations for best methodological practices with lab experiments.

evidence Paper

With Low Power Comes Low Credibility? Toward a Principled Critique of Results From Underpowered Tests

▸ Philosophy of science 20 / 20

Approaches to assess the reliability of scientific theories, reasoning, and methods attempting to understand its ability to make predictions about the natural and social world. Introduces how differing philosophies (positivist, post-positivist, constructivist, etc.) influence what scientists consider valid evidence and how open science challenges some traditional norms.

critique Paper

Open Science From a Qualitative, Feminist Perspective: Epistemological Dogmas and a Call for Critical Examination

This article evaluates the alignment between open science frameworks and the priorities of qualitative and feminist research within the field of psychology. It specifically questions whether existing open science dogmas inadvertently marginalize transgressive research methods and calls for a critical examination of how these frameworks impact radical inquiry.

practice/tools Paper

Towards Open Science for the Qualitative Researcher: From a Positivist to an Open Interpretation

This resource provides a practical reflection on data handling and pseudonymization in qualitative research, detailed through a case study of custom software development. It bridges technical implementation with epistemological inquiry to demonstrate how open research data guidelines can be successfully adapted to qualitative workflows.

Feest, U., & Devezer, B. (2025). Toward a more accurate notion of exploratory research (and why it matters). PhilSci Archive. https://philsci-archive.pitt.edu/24482/

advocacy Preprint

Subjectivity is a Feature, not a Flaw: A Call to Unsilence the Human Element in Science

This resource advocates for the recognition of researcher subjectivity as an inherent and valuable component of science rather than a contaminant to be purged. It challenges the traditional myth of the detached scientist and encourages the explicit use of reflexivity to enhance scientific integrity.

advocacy Paper

How Computational Modeling Can Force Theory Building in Psychological Science

This article promotes the adoption of computational modeling as a vital tool for advancing theory building within psychological science. It demonstrates how formalizing theories into models forces researchers to clarify vague intuitions and specify assumptions that often remain unexamined in purely verbal theories.

overview Paper

What Makes a Good Theory, and How Do We Make a Theory Good?

This resource proposes a formal ontology of criteria, known as a metatheoretical calculus, to evaluate the quality and robustness of scientific theories. It specifically outlines categories such as metaphysical commitment and discursive survival to help researchers move beyond vague assessments and toward rigorous theoretical adjudication.

critique Paper

Approaching psychology’s current crises by exploring the vagueness of psychological concepts: Recommendations for advancing the discipline.

This resource argues that the replication, theory, and universality crises in psychology are fundamentally linked to the vagueness of psychological concepts. It suggests that advancing the discipline requires a focus on theoretical and philosophical refinement rather than just methodological or statistical changes.

advocacy Paper

Moving beyond 20 questions: We (still) need stronger psychological theory.

This resource argues that psychology continues to struggle with fragmented findings and emphasizes the persistent need for robust, unifying theories to replace the "20 questions" style of empirical research. It advocates for a shift in focus from isolated experimental effects toward the development of comprehensive theoretical frameworks.

critique Paper

Open Science and Epistemic Diversity: Friends or Foes?

This work explores how the current implementation of open science may marginalize diverse research traditions by privileging specific inquiry styles over others. It identifies four reference points—such as local specificity and data provenance—to help open science frameworks better accommodate epistemic diversity.

Leonelli, S. (2023). Philosophy of open science. Cambridge University Press. http://philsci-archive.pitt.edu/id/eprint/21986

Mackenzie, N., & Knipe, S. (2006). Research dilemmas: Paradigms, methods and methodology. Issues in Educational Research, 16(2), 193–205. http://www.iier.org.au/iier16/mackenzie.html

critique Paper

Metascience Is Not Enough – A Plea for Psychological Humanities in the Wake of the Replication Crisis

This article critiques the reliance on metascience as the primary solution to the replication crisis, arguing that it overlooks deep-seated epistemic problems within psychology. It advocates for integrating perspectives from the psychological humanities to address the conceptual and historical complexities that quantitative metascientific approaches may fail to capture.

critique Paper

The quantitative paradigm and the nature of the human mind. The replication crisis as an epistemological crisis of quantitative psychology in view of the ontic nature of the psyche

critique Paper

Theory-Testing in Psychology and Physics: A Methodological Paradox

This seminal paper identifies a methodological paradox where increased experimental precision in psychology, unlike in physics, actually makes theory corroboration more difficult when relying on null hypothesis significance testing. It critiques the logical foundations of how psychological theories are tested, arguing that 'statistical significance' is often an inadequate substitute for genuine theoretical progress.

critique Paper

Is replication possible in qualitative research? A response to Makel et al. (2022)

Serving as a direct rebuttal to advocacy pieces, this response highlights three core areas where the logic of replication conflicts with the goals of qualitative research. It provides a critical perspective on how open research practices developed for quantitative work may not be appropriate for educational or qualitative methodologies.

advocacy Paper

Building better theories

This resource argues that the replication crisis is fundamentally a crisis of theory, advocating for a shift in focus toward more rigorous theory construction and specification. It highlights how strengthening the theoretical foundations of psychological research is essential for creating more robust, falsifiable, and reproducible scientific findings.

critique Paper

Rethinking Transparency and Rigor from a Qualitative Open Science Perspective

This paper critiques the quantitative-centric definition of transparency in open science, arguing that current frameworks do not align with the epistemic goals of qualitative research. It proposes a broader perspective that emphasizes researcher reflexivity and contextual data interpretation as essential components of rigor.

critique Paper

Psychological models and their distractors

This paper critiques the current use of formal models in psychology, arguing that they often serve as 'distractors' that mask a lack of theoretical depth rather than resolving it. It challenges researchers to ensure that their mathematical models are genuinely grounded in coherent psychological theory rather than being used as mere technical window dressing.

Yarkoni, T. (2020). The generalizability crisis. Behavioral and Brain Sciences, 45. https://doi.org/10.1017/S0140525X20001685

Lakatos, I. (1978). The Methodology of Scientific Research Programmes. https://doi.org/10.1017/CBO9780511621123

▸ Questionable measurement practices (QMPs), validity & reliability issues. 8 / 8

The quality of our measures impacts the validity of our results, and offers another avenue for us to address potential questionable practices. Examines how measurement choices shape the credibility of findings. Addresses Questionable Measurement Practices (QMPs) like ad-hoc scale trimming, unvalidated instruments, poor reliability reporting, ignored measurement invariance, and their impact on construct validity, reliability, and generalizability.

practice/tools Paper

Measurement Schmeasurement: Questionable Measurement Practices and How to Avoid Them

This resource defines and categorizes Questionable Measurement Practices (QMPs), illustrating how hidden decisions in the measurement process can threaten the validity of scientific conclusions. It provides a practical framework for researchers to increase measurement transparency and offers guidance on how to avoid these common pitfalls during study design and reporting.

evidence Paper

Construct Validation in Social and Personality Research

The authors present empirical meta-research by auditing a representative sample of social and personality psychology papers to evaluate the state of construct validation. The study reveals a significant gap between the common use of latent variable measurement and the lack of rigorous, ongoing evidence provided by researchers to justify those measures.

critique Paper

Approaching psychology’s current crises by exploring the vagueness of psychological concepts: Recommendations for advancing the discipline.

evidence Paper

Hidden Invalidity Among 15 Commonly Used Measures in Social and Personality Psychology

This study presents empirical evidence of 'hidden invalidity' by showing that widely used psychological scales often fail structural validity tests despite having acceptable internal consistency. By analyzing a uniquely large dataset, it demonstrates that standard metrics like Cronbach's alpha often mask significant psychometric flaws in social and personality psychology measures.

Parsons, S. (2022). Exploring reliability heterogeneity with multiverse analyses: Data processing decisions unpredictably influence measurement reliability. Meta-Psychology, 6. https://doi.org/10.15626/MP.2020.2577

advocacy Paper

Psychological Science Needs a Standard Practice of Reporting the Reliability of Cognitive-Behavioral Measurements

This paper advocates for the establishment of a standard reporting practice for measurement reliability within cognitive-behavioral research to improve the robustness of psychological science. It argues that transparently reporting reliability is a necessary prerequisite for properly evaluating statistical inferences and ensuring that research findings are not merely artifacts of measurement error.

critique Paper

Unreliability as a threat to understanding psychopathology: The cautionary tale of attentional bias.

This resource critiques the reliance on unreliable behavioral measures in psychopathology research, using attentional bias as a primary example of how poor metrics hinder scientific progress. It highlights the specific threat that measurement error poses to major clinical initiatives, such as the RDoC, which depend on high-reliability measures for individual difference research and mediation analysis.

practice/tools Paper

Crowdsourcing multiverse analyses to explore the impact of different data-processing and analysis decisions: A tutorial.

This resource provides a practical tutorial on implementing multiverse analyses to test the robustness of research findings against various data-processing and analytical choices. It demonstrates how exploring multiple plausible analysis paths can reveal the sensitivity of results to arbitrary decisions, thereby improving the transparency and generalizability of empirical research.

▸ Research design, sampling methods, & its implications for inferences. 4 / 4

How design choices and sampling strategies shape bias, precision, and generalizability. Includes threats to validity (internal/external), power and sample-size planning, selection bias, clustering/design effects, weighting, and transparent reporting/preregistration. Design and sampling decisions determine the credibility and scope of statistical inference. This sub-cluster emphasizes adequate power and sample-size planning (e.g., safeguard power), transparent pre-analysis planning to constrain researcher degrees of freedom, and rigorous, valid methods as prerequisites for meaningful replication—reducing bias, increasing precision, and improving generalizability across lab and field work.

evidence Paper

A Powerful Nudge? Presenting Calculable Consequences of Underpowered Research Shifts Incentives Toward Adequately Powered Designs

This study uses a stylized thought experiment to empirically evaluate how researchers weigh statistical power versus individual productivity in hiring decisions. It demonstrates that explicitly presenting the scientific consequences of underpowered research can shift professional incentives toward favoring more robustly powered experimental designs.

practice/tools Paper

Valid replications require valid methods: Recommendations for best methodological practices with lab experiments.

practice/tools Paper

Safeguard Power as a Protection Against Imprecise Power Estimates

practice/tools Paper

Degrees of Freedom in Planning, Running, Analyzing, and Reporting Psychological Studies: A Checklist to Avoid p-Hacking

▸ The logic of null hypothesis testing, p-values, Type I and II errors (and when and why they might happen). 12 / 12

Frequentist statistics are typically the default in quantitative research. They come with certain assumptions and implications, as well as often being misinterpreted. Frequentist statistics are typically the default in much quantitative research, but they are often misinterpreted. This sub-cluster clarifies the logic of NHST, the meaning of p-values, and when and why Type I and Type II errors arise, extending to Type S and Type M errors. It links these to design and power choices and outlines practical steps for better inference and reporting.

Banerjee, A., Chitnis, U., Jadhav, S., Bhawalkar, J., & Chaudhury, S. (2009). Hypothesis testing, type I and type II errors. Industrial Psychiatry Journal, 18(2), 127. https://doi.org/10.4103%2F0972-6748.62274

critique Paper

Understanding the Replication Crisis as a Base Rate Fallacy

Cohen, J. (1994). The earth is round (p < .05). American Psychologist, 49(12), 997–1003. https://doi.org/10.1037/0003-066X.49.12.997

critique Paper

Type I Error Rates are Not Usually Inflated

This article challenges the conventional wisdom that questionable research practices like p-hacking necessarily inflate Type I error rates. It introduces nuanced distinctions between different types of statistical errors to argue that many criticized practices do not impact the error rates relevant to the researchers' specific hypotheses.

practice/tools Paper

Beyond Power Calculations

This paper proposes a move beyond traditional power analysis by introducing "design calculations" to estimate Type S (sign) and Type M (magnitude) errors. These metrics help researchers understand the risk of obtaining results that are either in the wrong direction or grossly exaggerated in magnitude, particularly in small-sample studies.

policies Paper

Interrogating the “cargo cult science” metaphor

critique Book Chapter

The Null Ritual: What You Always Wanted to Know About Significance Testing but Were Afraid to Ask

This publication critiques the institutionalized "null ritual," which it describes as an incoherent amalgamation of incompatible Fisherian and Neyman-Pearson statistical frameworks. It explains how this ritualized practice suppresses critical thinking and fosters the illusion that statistical significance is a substitute for scientific evidence and theoretical reasoning.

critique Paper

Mindless statistics

This article critiques the "null ritual" prevalent in the social sciences, where statistical procedures are applied mindlessly as a requirement for social group identification rather than scientific inquiry. It highlights how rigid adherence to significance levels leads to collective confusion among researchers and undermines the quality of statistical reasoning in scientific publications.

Lakens, D. Improving your statistical inferences. Online course. https://www.coursera.org/learn/statistical-inferences

evidence Paper

With Low Power Comes Low Credibility? Toward a Principled Critique of Results From Underpowered Tests

critique Paper

Inconsistent multiple testing corrections: The fallacy of using family-based error rates to make inferences about individual hypotheses

This resource highlights a specific logical inconsistency where researchers apply familywise error rate corrections to individual hypothesis tests rather than joint union hypotheses. It argues that this practice leads to inappropriate inferential conclusions and clarifies the intended purpose of alpha-level adjustments.

policies Paper

The ASA Statement on p -Values: Context, Process, and Purpose

This official statement from the American Statistical Association provides six principles to guide the use and interpretation of p-values in scientific research. It serves as a formal policy document intended to improve the transparency and reproducibility of statistical analysis across various disciplines.

3 Ways of Working

12 sub-clusters · 133 references

Attainment of an understanding of how research is conducted, managed and disseminated. There are 12 sub-clusters which aim to further parse the learning and teaching process[s]:

▸ Qualitative research 5 / 5

This section includes key introductory and methodological texts on designing and conducting qualitative research. These sources provide guidance on research planning, data collection, analysis techniques, and theoretical foundations relevant for beginners and experienced researchers alike.

teaching/training Book

Introducing Qualitative Research: A Student's Guide

This student-oriented guide provides a comprehensive framework for understanding and implementing qualitative research methods, from epistemological foundations to data generation and analysis. It serves as a pedagogical resource that maps out complex research designs, including action research and mixed methods, for those new to the field.

Braun, V., & Clarke, V. (2013). Successful qualitative research: A practical guide for beginners. SAGE Publications. ISBN: 9781847875815

Denzin, N. K., & Lincoln, Y. S. (Eds.). (2023). The SAGE handbook of qualitative research (6th ed.). SAGE Publications.

teaching/training Book

Designing Qualitative Research

This resource offers a systematic approach to the planning and execution of qualitative studies, focusing on the alignment between research questions and methodological choices. It addresses crucial aspects of the research process, including sampling strategies, quality criteria, and the integration of diverse data types through triangulation.

Kumar, R. (2019). Research methodology: A step-by-step guide for beginners (5th ed.). SAGE Publications.

▸ Adversarial collaborations 6 / 6

Adversarial collaborations typically include two (or more) groups of researchers addressing the same research question with conflicting hypotheses - e.g., one group expects an effect to exist, while another does not.

evidence Paper

Testing competing models of loss aversion: an adversarial collaboration

This resource provides an empirical demonstration of an adversarial collaboration used to resolve long-standing theoretical disagreements regarding models of loss aversion. It illustrates how researchers with conflicting hypotheses can jointly design and conduct experiments to move a niche research area forward despite fundamental differences in perspective.

advocacy Book Chapter

Adversarial Collaboration: The Next Science Reform

This publication advocates for the widespread adoption of adversarial collaboration as a foundational reform to improve scientific integrity and reduce confirmation bias across disciplines. It argues that pre-committing to research designs with intellectual opponents is a superior method for resolving scientific disputes compared to traditional adversarial debate.

overview Paper

The Tone Debate: Knowledge, Self, and Social Order

This resource explores the sociological and ethical dimensions of scientific critique, specifically focusing on the 'tone debate' that emerged during the replication crisis in psychology. It examines how researchers attempt to manage interpersonal conflict and professional standards through informal codes of conduct and social media engagement.

Kahneman, D. (2003). Experiences of Collaborative Research. American Psychologist, 58(9), 723–730. https://oce-ovid-com.utrechtuniversity.idm.oclc.org/article/00000487-200309000-00003/HTML

advocacy Paper

Make science more collegial: why the time for ‘adversarial collaboration’ has come

This piece makes a persuasive case for incorporating adversarial collaboration into standard research workflows to foster a more collegial and less polarized scientific environment. It highlights the strategic benefits of direct cooperation between rivals for increasing the credibility of research findings and accelerating scientific progress.

overview Journal Article

Publication Pressure and Scientific Misconduct in Medical Scientists

▸ Big team science 12 / 12

In big team science (sometimes referred to as “team science”), many researchers pool their resources to solve a problem or answer a research question together, usually resulting in one or several outputs that all researchers involved gain authorship on. Big team science projects can either be coordinated by an organisation (e.g. ManyBabies), or can be run independently by a group of researchers.

advocacy Letter

Towards a culture of open scholarship: the role of pedagogical communities

practice/tools Book

Research Without Borders: How to Identify and Overcome Potential Pitfalls in International Large-Team Online Research Projects

This case study identifies practical pitfalls and solutions associated with managing international, large-scale research collaborations via online surveys. It provides specific guidance for researchers on navigating cross-cultural collaboration and logistical challenges during the design and implementation phases of meta-scientific projects.

practice/tools Paper

How to build up big team science: a practical guide for large-scale collaborations

This resource provides a comprehensive practical framework for launching and managing big team science projects based on insights from diverse multi-disciplinary initiatives. It addresses essential organizational elements including team recruitment, leadership structures, governance, and the selection of collaborative digital tools.

advocacy Paper

‘Big team’ science challenges us to reconsider authorship

This publication highlights the limitations of traditional authorship models when applied to large-scale, collaborative research efforts. It advocates for a fundamental shift toward more nuanced credit systems that accurately recognize the diverse and distributed contributions inherent in big team science.

practice/tools Preprint

Promoting Civility in Formal And Informal Open Science Contexts

This resource provides evidence-based strategies for fostering interpersonal civility within both formal and informal open science environments. It addresses how incivility acts as a barrier to inclusion and offers practical solutions to help organizations maintain respectful and equitable collaborative spaces.

advocacy Paper

The Road We Must Take: Multidisciplinary Team Science

This article advocates for a multidisciplinary team science approach to overcome the complexities of translational research and find lasting solutions to healthcare problems. It highlights the need for academic institutions to go beyond providing facilities by actively fostering and sustaining collaborative research teams.

advocacy Paper

The Benefits, Barriers, and Risks of Big-Team Science

This article identifies five core challenges in psychology, such as replicability and generalizability, and argues that they result from an insufficient investment of resources in individual studies. It advocates for the adoption of 'big-team science' as a structural solution that allows researchers to pool intellectual and material resources. The authors evaluate the trade-offs of this model, highlighting how it can improve the robustness of psychological research.

Forscher, P. S., Wagenmakers, E. J., DeBruine, L., Coles, N., Silan, M. A., & IJzerman, H. (2020). A Manifesto for Team Science.

Lakens, D., & Lieck, D. (2022). An Overview of Team Science Projects in the Social Behavioral Sciences. OSF. https://doi.org/10.17605/OSF.IO/WX4ZD

advocacy Paper

Crowdsourced research: Many hands make tight work

This resource argues that crowdsourcing research is a valuable tool for validating scientific findings and ensuring balanced academic discourse. It explains how involving diverse groups of researchers can produce more reliable results that are better suited to inform public policy decisions.

teaching/training Paper

Publishing Research With Undergraduate Students via Replication Work: The Collaborative Replications and Education Project

This resource describes the Collaborative Replications and Education Project (CREP), which provides a structured framework for incorporating high-quality replication research into undergraduate education. It outlines how the project benefits students by providing publication opportunities and practical training in open science practices.

practice/tools Paper

Ten simple rules for socially responsible science

This paper provides a practical framework of ten guidelines aimed at helping researchers across disciplines minimize the indirect social harms caused by study design, reporting, and dissemination. It bridges the gap between traditional research ethics and broader social responsibility, offering actionable steps to prevent the stigmatization or marginalization of social groups.

▸ Community science 8 / 8

Community science, sometimes called “citizen science”, is scientific research conducted, in whole or in part, by amateur (or non-professional) scientists. Community science is sometimes described as "public participation in scientific research," participatory monitoring, and participatory action research whose outcomes are often advancements in scientific research by improving the scientific community's capacity, as well as increasing the public's understanding of science.

practice/tools Paper

Citizen Science: A Developing Tool for Expanding Science Knowledge and Scientific Literacy

This paper presents the nine-stage "Project Design and Implementation Model" for citizen science, offering a structured roadmap for developing large-scale public research collaborations. It provides practical insights derived from decades of experience at the Cornell Lab of Ornithology, making it an actionable resource for researchers looking to design their own citizen-led projects.

advocacy Paper

Next Steps for Citizen Science

This article highlights the necessity of strategic funding and organizational infrastructure to scale citizen science initiatives effectively to their full potential. It argues for moving beyond project-level thinking toward a coordinated framework that supports data standards and systemic investment in citizen-based research.

Cohn, J. P. (2008). Citizen Science: Can Volunteers Do Real Research? BioScience, 58(3), 192–197. https://doi.org/10.1641/B580303

overview Paper

Crowd science: The organization of scientific research in open collaborative projects

This paper provides a conceptual framework for understanding "crowd science," focusing on how open collaborative projects differ from traditional scientific organizational models. It characterizes projects based on their degrees of openness and participation, establishing a structured agenda for meta-research into these new modes of scientific production.

overview Book

Citzen Science

This overview positions citizen science as a critical component of the broader open science and innovation landscape, driven by both technological advancement and societal demand for participation. It emphasizes the field's potential to transform tertiary education and provides a high-level summary of the benefits of integrating the public into scientific knowledge production.

Hart, D.D., and Silka, L. (2020). Rebuilding the Ivory Tower: A Bottom-Up Experiment in Aligning Research with Societal Needs. Issues in Science and Technology, 36(3), 64–70. https://issues.org/aligning-research-with-societal-needs/

advocacy Letter

Fieldwork: institutions can make it more ethical

This resource advocates for institutional-level changes to ensure that fieldwork is conducted ethically and sustainably, emphasizing the responsibility of research bodies to provide structural support. It highlights the need for institutions to move beyond individual compliance and establish frameworks that protect both researchers and the communities they engage with.

overview Paper

Open Citizen Science: fostering open knowledge with participation

This publication explores the intersection of citizen science and open science, examining how participatory research can foster open knowledge through transparent methodologies. It discusses the varying intensities of community involvement and the importance of adhering to shared scientific standards to ensure the public credibility of volunteer-led research.

▸ Environmental sustainability (e.g. conference travel, high performance computing, etc.) 6 / 6

Examines the climate and environmental footprint of research workflows and infrastructures.

advocacy Paper

Spiral-scaling climate action: lessons from and for the academic flying less movement

This article analyzes the Academic Flying Less Movement to provide a conceptual framework for scaling individual environmental concerns into collective sectoral change. It draws on lessons from norm diffusion to suggest how scholarly communities can shift institutional practices away from high-carbon travel.

evidence Paper

An analysis of ways to decarbonize conference travel after COVID-19

This study provides an empirical analysis of strategies for decarbonizing conference travel in the post-pandemic era, evaluating different travel and meeting models. It contributes evidence-based pathways for academic organizations to reduce their carbon footprint while maintaining the essential benefits of scientific exchange.

advocacy Paper

The Need for Sustainability, Equity, and International Exchange: Perspectives of Early Career Environmental Psychologists on the Future of Conferences

This resource presents a perspective from early career researchers on the need to reform conference travel practices to align with the intrinsic values of environmental psychology. It argues that moving toward sustainable and equitable conference models is essential for maintaining the discipline's scientific credibility and ensuring inclusive international exchange.

practice/tools Paper

Carbon footprint estimation for computational research

This resource provides methodologies and frameworks for researchers to quantify the environmental impact of their data processing and high-performance computing tasks. It addresses the need for carbon transparency in computational science by offering specific estimation techniques to track the energy footprint of digital research workflows.

evidence Paper

The Carbon Footprint of Conference Papers

This study provides empirical evidence on the environmental cost of academic travel by calculating the CO2 emissions associated specifically with presenting papers at scientific conferences. It contextualizes the impact of professional travel by comparing conference-related emissions to both global totals and individual annual carbon footprints.

evidence Paper

Trend towards virtual and hybrid conferences may be an effective climate change mitigation strategy

This resource presents a comparative life cycle assessment of in-person, virtual, and hybrid conferences to identify the most effective strategies for mitigating climate change within the scientific community. It offers data-driven insights into the trade-offs between different event formats, highlighting the significant carbon reduction potential of transitioning to remote participation.

▸ Participatory research 12 / 12

Participatory research, sometimes also referred to as co-production, is an umbrella term for methods in which views and engagement of interested parties from relevant communities (academic or otherwise) are included throughout the research process, from conception to dissemination.

practice/tools Journal Article

Strengthening the Informed Consent Process in International Health Research through Community Engagement: The KEMRI-Wellcome Trust Research Programme Experience

This publication provides a practical case study on adapting informed consent processes to suit specific local, social, and cultural contexts in international health research. It offers actionable insights into how community engagement can be used to redesign consent forms and administration procedures to be more ethically robust.

practice/tools Paper

The Value and Challenges of Participatory Research: Strengthening Its Practice

This resource contributes an integrative practice framework consisting of five essential domains for designing, implementing, and evaluating participatory research partnerships. It provides a structured process for researchers to bridge the gap between academic study and community practice, focusing on shared control over health determinants and social justice.

practice/tools Paper

Inclusive Practices for Neurodevelopmental Research

This resource provides a practical framework for implementing inclusive research designs within the field of neurodevelopmental studies. It outlines six specific considerations for ethical and effective collaboration between researchers and the neurodivergent community, drawing on both theoretical models and real-world project exemplars.

advocacy Paper

Opening up understanding of neurodiversity: A call for applying participatory and open scholarship practices

This work advocates for the integration of participatory research and open scholarship practices to address traditional power imbalances that marginalize neurodivergent individuals in academia. It emphasizes how collaborative methodologies can lead to more representative and ethically grounded knowledge production regarding neurodevelopmental diversity.

policies Journal Article

Involving Research Stakeholders in Developing Policy on Sharing Public Health Research Data in Kenya

This document establishes the formal principles of the Cape Town Statement, providing a policy framework for promoting fairness, equity, and diversity within the global research ecosystem. It serves as a normative guideline for institutions and researchers to align their collaborative practices with ethical standards of equity.

evidence Journal Article

Research Stakeholders’ Views on Benefits and Challenges for Public Health Research Data Sharing in Kenya: The Importance of Trust and Social Relations

This study employs a deliberative qualitative approach to explore how research stakeholders in Kenya perceive the benefits and risks of sharing public health data. It identifies specific stakeholder concerns regarding fairness and interest protection, providing empirical evidence to inform data-sharing policies in low-to-middle income countries.

evidence Journal Article

Engaging Communities to Strengthen Research Ethics in Low‐Income Settings: Selection and Perceptions of Members of a Network of Representatives in Coastal <scp>K</scp>enya

This resource examines the relational dimensions of data sharing in Kenya, highlighting how structural inequities necessitate the building of trust between researchers and communities. It identifies practical ways to foster this trust, such as involving the public in policy development and creating partnerships between researchers and government health authorities.

practice/tools Paper

Interlinking open science and community-based participatory research for socio-environmental issues

This resource introduces a theoretical framework that interlinks Open Science with community-based participatory research to address socio-environmental issues. It provides four practical strategies, including the 'transcend method' and FAIR data visualization, designed to bridge gaps between diverse stakeholders and empower marginalized voices.

overview Journal Article

Consulting communities on feedback of genetic findings in international health research: sharing sickle cell disease and carrier information in coastal Kenya

This resource provides a conceptual exploration of the relationship between research integrity and research fairness, addressing stakeholder concerns that these two principles may be in conflict. It offers an analytical framework to harmonize these concepts, arguing for a more integrated understanding of responsible research conduct.

Martinez-Vargas, C. (2022). Democratising Participatory Research. https://doi.org/10.11647/OBP.0273

evidence Journal Article

What Are Fair Study Benefits in International Health Research? Consulting Community Members in Kenya

This study presents empirical findings from community consultations in Kenya to understand local perspectives on research benefits and participant compensation. The evidence suggests that concerns regarding 'undue inducement' are often secondary to local priorities of fairness and the addressing of structural poverty.

Pham, J., Perry-Wilson, T., Holmes, K., Schroeder, G., Reyes, A., & Pollok, M. (2025). The power of decolonizing research practices. The Professional Counselor, 15(1). https://tpcjournal.nbcc.org/the-power-of-decolonizing-research-practices

▸ Public and Private Partnerships 12 / 12

Academia is but one avenue for knowledge production. In fact, research happens in a variety of contexts. Open science practitioners who conduct research with public and private partners need to be aware of the challenges and opportunities that arise when working within these spaces.

evidence Preprint

Framing Power: Tracing Key Discourses in Open Science Policies

This meta-research study provides a content analysis of global Open Science policies to identify how dominant narratives may reinforce existing power imbalances. It highlights how institutional policy framing often serves the interests of established global powers while potentially marginalizing the knowledge systems of the Global South.

evidence Paper

The (Non)Academic Community Forming around Replications: Mapping the International Open Science space via its Replication Initiatives

This study maps the international landscape of replication initiatives to illustrate how the movement has evolved into a transdisciplinary community. It provides evidence of the diverse stakeholders involved, including non-academic actors and commercial publishers, showing how replication concerns have expanded beyond specific scientific fields.

overview Paper

Public–private partnerships and beyond: Potential for innovation and sustainable development

This article surveys the role of public-private partnerships as drivers of innovation and sustainable development, particularly within the European regulatory context. It examines how policy reforms and specialization strategies have been used to facilitate these collaborations during periods of public budget constraints.

overview Paper

How Will Open Science Impact on University-Industry Collaboration?

This publication examines the shifting dynamics between university researchers and industry partners in the context of the open science movement. It specifically analyzes how demands for open data and transparency challenge traditional knowledge exchange models based on closed partnerships and restrictive intellectual property rights.

advocacy Paper

The value of public-private collaborative real-world evidence platforms to monitor vaccine performance post authorization: DRIVE - a European initiative

This resource argues for the benefits of using public-private partnership platforms to generate real-world evidence for monitoring vaccine performance after authorization. It specifically calls for a structured debate to clarify stakeholder responsibilities and to address the underlying causes of institutional hesitancy toward these collaborative models.

practice/tools Preprint

An open toolkit for tracking open science partnership implementation and impact

This resource presents a practical toolkit designed to help institutions and firms track the implementation and socio-economic impact of open science partnerships. It provides a structured set of metrics to evaluate various open practices, such as data sharing and the removal of intellectual property barriers, across diverse research collaborations.

advocacy Paper

The fall of the innovation empire and its possible rise through open science

This resource argues that open science partnerships are a necessary mechanism to reverse the declining effectiveness and rising costs of the global innovation system. By analyzing a century of empirical literature on researcher productivity, it provides a strategic framework for collaborative research designed to restore science’s ability to generate broad social and economic benefits.

advocacy Paper

Open Science and Its Enemies: Challenges for a Sustainable Science–Society Social Contract

This resource analyzes the historical evolution of the social contract between science and society, positioning open science as a vital safeguard for scientific legitimacy and the public good. It identifies key systemic challenges to this contract and argues for a renewed institutional commitment to transparency to ensure the long-term sustainability of the scientific enterprise.

McKibban, A. R., & Steltenpohl, C. N. (2019). Community organizing, partnerships, and coalitions. In L. A. Jason, O. Glantsman, J. F. O’Brien, & K. N. Ramian (Eds.), Introduction to Community Psychology. Rebus. https://press.rebus.community/introductiontocommunitypsychology/chapter/community-organizing-partnerships-and-coalitions/

Okafor, I. A., Mbagwu, S. I., Chia, T., Hasim, Z., Udokanma, E. E., & Chandran, K. (2022). Institutionalizing open science in Africa: Limitations and prospects. Frontiers in Research Metrics and Analytics, 7, 855198. https://www.frontiersin.org/articles/10.3389/frma.2022.855198/full

evidence Paper

Open data partnerships between firms and universities: The role of boundary organizations

Through an inductive study of the Structural Genomics Consortium, this paper provides evidence on how 'boundary organizations' facilitate successful open data collaborations between universities and private firms. It details the specific organizational strategies used to resolve tensions between proprietary interests and academic motivations, offering a empirical model for pre-competitive R&D.

Shaw, D. L. (2017). Focus: Drug development: Is open science the future of drug development? The Yale Journal of Biology and Medicine, 90(1), 147. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5369032/

▸ Reflexivity and positionality 31 / 31

Examines how the social positions, values, and relationships shape research questions, design, data collection, analysis, and reporting.

critique Paper

Experimenter as automaton; experimenter as human: exploring the position of the researcher in scientific research

This paper critiques the push for mechanical objectivity in scientific research, which often associates researcher subjectivity with low quality. It argues for the importance of reflexivity in both quantitative and qualitative traditions to better account for the researcher's position in the production of knowledge.

advocacy Preprint

Subjectivity is a Feature, not a Flaw: A Call to Unsilence the Human Element in Science

critique Preprint

Open With Care! Consent, Context, and Co-production in Open Qualitative Research

This resource responds to scholarly critiques of the Netherlands Code of Conduct for Research Integrity, arguing that the existing code remains a justified and functional framework for ethical practice. It contributes to the broader debate on research ethics by clarifying the purpose and limitations of national integrity policies.

teaching/training Paper

Challenges of Critical Reflection: ‘Nothing Ventured, Nothing Gained’

This paper provides pedagogical reflections on the challenges and risks involved in teaching critical reflection to social work and health professionals. It examines how these cultural challenges can be managed by educators to turn potential resistance into productive learning opportunities.

evidence Paper

From understanding to insight: using reflexivity to promote students’ learning of qualitative research

This study provides empirical evidence on how students develop reflexive skills by analyzing journals from a qualitative research course. It identifies three distinct dimensions of reflexivity that facilitate the transition from theoretical understanding to deep, insightful learning.

overview Paper

Ethics, Reflexivity, and “Ethically Important Moments” in Research

This article distinguishes between "procedural ethics" and "ethics in practice," offering a conceptual framework for navigating unpredictable ethical moments that occur during the research process. It highlights reflexivity as a crucial bridge between formal institutional requirements and the daily realities of interacting with research participants.

practice/tools Paper

In search of a critical stance: Applying qualitative research practices for critical quantitative research in psychology

This paper describes how qualitative practices like memoing and positionality documentation can be applied to quantitative psychology to align the work with critical epistemological stances. It offers practical guidance for researchers to archive their decision-making processes and acknowledge their subjective influence on data interpretation.

practice/tools Paper

Reflexivity in quantitative research: A rationale and beginner's guide

This resource serves as a primer for quantitative researchers to integrate reflexivity into their workflow, explaining its importance for rigor while providing actionable steps for beginners. It bridges a traditional methodological gap by adapting self-reflection techniques for use in quantitative research designs.

overview Paper

Doing reflexivity in psychological research: What’s the point? What’s the practice?

This article provides an introductory guide to reflexivity within psychology, clarifying its definition and practical application for researchers new to qualitative methods. It distinguishes reflexive activity from other critical thinking practices and offers a framework based on perspectival location to improve the transparency of the research process.

teaching/training Paper

Embracing the Spiral

This resource reflects on a collaborative pedagogical process where graduate students and their supervisor developed the "reflexivity spiral" framework to interrogate their social locations. It illustrates how personal backgrounds and sociopolitical contexts dynamically shape research motivations and methodologies across various critical research traditions.

practice/tools Paper

Shedding the cloak of neutrality: A guide for reflexive practices to make the sciences more inclusive and just

This resource provides a practical guide for environmental scientists to implement reflexive practices, aimed at acknowledging how their personal positionality and social context influence knowledge production. It offers specific strategies for researchers to challenge the assumption of scientific neutrality and address epistemic oppression within their field.

practice/tools Paper

Ethical dilemmas and reflexivity in qualitative research

This resource provides actionable insights into the practice of reflexivity by sharing and debating the ethical challenges encountered during qualitative research projects. It offers a model for how researchers can navigate difficult decision-making moments and maintain ethical integrity through constant, critical self-reflection.

critique Paper

Experimenter as automaton; experimenter as human: exploring the position of the researcher in scientific research

advocacy Preprint

Subjectivity is a Feature, not a Flaw: A Call to Unsilence the Human Element in Science

critique Preprint

Open With Care! Consent, Context, and Co-production in Open Qualitative Research

teaching/training Paper

Challenges of Critical Reflection: ‘Nothing Ventured, Nothing Gained’

evidence Paper

From understanding to insight: using reflexivity to promote students’ learning of qualitative research

overview Paper

Ethics, Reflexivity, and “Ethically Important Moments” in Research

practice/tools Paper

In search of a critical stance: Applying qualitative research practices for critical quantitative research in psychology

practice/tools Paper

Social Identity Map: A Reflexivity Tool for Practicing Explicit Positionality in Critical Qualitative Research

The authors introduce the Social Identity Map as a visual tool to help qualitative researchers systematically identify and reflect on their specific social locations. This resource provides a structured method for translating the abstract concept of positionality into a tangible practice that informs data collection and analysis.

practice/tools Paper

Reflexivity in quantitative research: A rationale and beginner's guide

Kapiszewski, D., & Wood, E. J. (2021). Ethics, Epistemology, and Openness in Research with Human Participants. Perspectives on Politics, 20(3), 948–964. https://doi.org/10.1017/S1537592720004703

overview Paper

Doing reflexivity in psychological research: What’s the point? What’s the practice?

teaching/training Paper

Embracing the Spiral

policies Paper

POSITIONALITY STATEMENTS ARE JUST THE TIP OF THE ICEBERG: MOVING TOWARDS A REFLEXIVE PROCESS

This editorial presents and clarifies the Journal of Women and Minorities in Science and Engineering’s policy requiring positionality statements for all published research. It emphasizes that these statements are a necessary step toward a deeper reflexive process within STEM education, rather than a mere administrative requirement.

Prosser, A., (2025). Unheard, or unspoken? How listening to qualitative researcher voices will shape the future of open research [Plenary slides]. In Qualitative Open Science: Challenges, Opportunities, Tensions, and Synergies (Plenary session). Community of Practice for Naturally Occurring Data. https://osf.io/vnm7p

Prosser, A., (2025). Unheard, or unspoken? How listening to qualitative researcher voices will shape the future of open research [Video]. In Qualitative Open Science: Challenges, Opportunities, Tensions, and Synergies (Plenary session). Community of Practice for Naturally Occurring Data. https://osf.io/pxcy5

practice/tools Paper

Ethical dilemmas and reflexivity in qualitative research

critique Paper

Rethinking Transparency and Rigor from a Qualitative Open Science Perspective

▸ Research with students (under- and graduate) 17 / 17

Examines structured ways to involve undergraduates and postgraduates in research, course-based projects, lab apprenticeships, and multi-site replications/consortia. Covers pedagogy, supervision and authorship practices, training in open and reproducible methods, and evaluation of learning, equity, and research quality.

teaching/training Preprint

Teaching Constructive Replications in the Social Sciences

This resource outlines how constructive replications can be integrated into social science curricula as a learning-by-doing pedagogical tool. It emphasizes the dual benefit of improving students' methodological skills while contributing to the self-correcting nature of scientific research.

evidence Preprint

Eleven years of student replication projects provide evidence on the correlates of replicability in psychology

This paper provides a new dataset and empirical analysis of 176 student-led replication projects conducted over eleven years to identify correlates of replication success in psychology. It demonstrates that graduate-level course projects can generate high-quality meta-research data that helps address the difficulty of conducting large-scale independent replication studies.

practice/tools Paper

To Co-Author or Not to Co-Author: How to Write, Publish, and Negotiate Issues of Authorship with Undergraduate Research Students

This resource provides actionable strategies and practical advice for faculty on navigating the complexities of co-authoring and publishing research with undergraduate students. It addresses the pedagogical value of these collaborations while offering specific guidance on negotiating authorship and overcoming common logistical challenges.

teaching/training Paper

Grassroots Training for Reproducible Science: A Consortium-Based Approach to the Empirical Dissertation

This paper proposes a consortium-based model for undergraduate empirical dissertations designed to embed reproducible research practices and team science at the grassroots level of education. It details how aligning collaborative structures with student projects can shift research culture and improve the overall credibility of scientific training.

Button, K. S., Lawrence, N. S., Chambers, C. D., & Munafò, M. R. (2016). Instilling scientific rigour at the grassroots. The Psychologist, 29(3), 158-159. https://www.bps.org.uk/psychologist/instilling-scientific-rigour-grassroots

advocacy Paper

Reboot undergraduate courses for reproducibility

This resource argues for the fundamental restructuring of undergraduate curricula to prioritize reproducibility and open science principles from the very beginning of scientific training. It emphasizes the necessity of a cultural shift in how research methods are taught to ensure the next generation of researchers adopts robust technical and ethical standards.

teaching/training Paper

How (and Whether) to Teach Undergraduates About the Replication Crisis in Psychological Science

evidence Paper

High impact: Examining predictors of faculty-undergraduate coauthored publication and presentation in psychology

This study identifies institutional and faculty-level predictors that correlate with successful co-authored publications and conference presentations between faculty and undergraduates in psychology. It provides empirical data on how factors like rank, experience, and mentoring enjoyment influence research outcomes for students.

Bang Jensen, B., Bresee, B., Dreier, S. K., Farrokhi, R., Gade, E. K., Jeffers, W., Morris, M. H., Pabbaraju, C. S., Salehian, K., Sharifi, A., Schuett, A., Sirikupt, C., Thomas, E., & Villa, D. (2023). The Lab as a Classroom: Advancing Faculty Research Through Undergraduate Experiential Education. PS: Political Science & Politics, 56(4), 455–462. https://doi.org/10.1017/S1049096523000033

teaching/training Paper

Balancing Needs in Publishing With Undergraduate and Graduate Students at Doctoral Degree-Granting Universities

This resource addresses the complexities of co-publishing with students at different levels within research-intensive environments. It offers strategies for balancing the educational needs of students with the professional requirements of faculty publication.

evidence Paper

Increasing Research Productivity in Undergraduate Research Experiences: Exploring Predictors of Collaborative Faculty–Student Publications

This study examines predictors of collaborative faculty-student publications based on survey data from faculty at numerous research-intensive institutions. It identifies key factors such as faculty commitment and disciplinary differences that influence the success of mentored undergraduate research products.

Norcross, J. C. (2014). Getting involved in research as an undergraduate: Nuts and bolts. Psychology Student Network. The American Psychological Association. https://www.apa.org/ed/precollege/psn/2014/01/research-undergraduate.

teaching/training Paper

<scp>Co‐authoring</scp> with undergraduate students: An emerging process from the <scp>semi‐periphery</scp> of the world of science

This paper explores the pedagogical process of faculty-undergraduate co-authorship, specifically focusing on the unique challenges and opportunities within the 'semi-periphery' of the global scientific community. It highlights how these collaborative publishing practices can be institutionalized outside of dominant Western academic contexts to benefit both students and faculty.

overview Paper

Teaching open and reproducible scholarship: a critical review of the evidence base for current pedagogical methods and their outcomes

evidence Paper

UK Psychology PhD researchers’ knowledge, perceptions, and experiences of open science

This study presents empirical findings from a mixed-methods survey of UK-based psychology PhD researchers regarding their knowledge and adoption of open science practices. It identifies specific barriers and training needs, offering data-driven insights into how to better support the next generation of researchers in this field.

teaching/training Paper

Care-full and reproducible research: Teaching research skills and ethics to undergraduate researchers using critical replication studies

This article details a pedagogical framework for teaching research ethics and reproducibility through the implementation of course-based replication studies. It emphasizes a 'care-full' approach that encourages students to critically evaluate study motivations and generalizability while gaining hands-on experimental experience.

teaching/training Paper

Publishing Research With Undergraduate Students via Replication Work: The Collaborative Replications and Education Project

▸ Science communication and public outreach 6 / 6

We should not do science so it stays among scientists, we should do science so it reaches and impacts the general population, as well as funding agencies, community members, interested parties, and policy makers. Effective science communication builds trust in science and counteracts misinformation.

overview Book

Science Communication

This book provides a comprehensive overview of modern science communication practices through a series of diverse case studies and examples ranging from citizen science to social media. It explores the relationship between scientists and the public, reflecting on how communication shapes scientific culture and professional identity.

evidence Paper

From Open Access to Open Science: The Path From Scientific Reality to Open Scientific Communication

This publication reports on a self-experiment investigating the feasibility of making the entire development process of a doctoral thesis transparent and open-access. It evaluates the practical potential for researchers to use open licenses to ensure scientific work is traceable and freely accessible from inception to final form.

overview Book

The Oxford Handbook of the Science of Science Communication

This handbook provides a multi-disciplinary overview of the science of science communication, examining the factors that influence public understanding and the challenges facing the scientific community. It explores topics ranging from media structures and historical controversies to the integrity of the peer-review process and scientific retraction.

advocacy Paper

Bridging science communication and open science—Working inclusively toward the common good

This essay advocates for the integration of Open Science and Science Communication practices to strengthen the relationship between science and society. It proposes a conceptual framework to bridge the gap between these two historically disconnected communities, aiming to foster more inclusive and mutually beneficial engagement.

critique Paper

Forcing a Deterministic Frame on Probabilistic Phenomena: A Communication Blind Spot in Media Coverage of the “Replication Crisis”

This article critiques the deterministic way in which media outlets report on the replication crisis, arguing that such framing ignores the inherent statistical uncertainty of scientific results. It highlights how oversimplified success-or-failure narratives can erode public trust and calls for communication that emphasizes the cumulative, probabilistic nature of evidence.

overview Paper

Science communication and the issue of trust

This resource examines how the proliferation of diverse actors in the modern 'ecology of communication'—including PR experts, government bodies, and institutions—has complicated the landscape of science communication. It highlights the potential erosion of trust when scientific messaging is perceived to be influenced by special interests or institutional branding rather than pure research dissemination.

▸ Slow Science/Slow Scholarship 11 / 11

The scientific process is characterized by its methodical, deliberate nature, aimed at the comprehensive understanding of phenomena rather than the immediate resolution of societal issues. This approach, prioritizing the pursuit of knowledge over the fulfillment of performance targets, facilitates the development of trust between researchers and various stakeholders, including the academic community and the general public. It emphasizes the importance of inclusivity, ensuring that research outcomes are beneficial across diverse groups, particularly those that have been historically marginalized. The emphasis is placed on thoroughness and precision within the research process, rather than the rapidity of outcomes.

advocacy Letter

Taking time to savour the rewards of slow science

This piece advocates for the adoption of 'slow science' principles, emphasizing the qualitative rewards and scientific benefits of taking more time for research. It encourages researchers to resist the pressure of high-speed productivity in favor of professional fulfillment and thoroughness.

advocacy Paper

In support of slow science: Robust, open, and multidisciplinary

This resource makes the case for slow science by linking it to increased research robustness, openness, and the ability to conduct meaningful multidisciplinary work. It argues that slowing down allows for more rigorous methodology and better alignment with the core values of open science.

advocacy Paper

Fast Lane to Slow Science

This article discusses the transition from a high-pressure, 'fast' academic culture to a 'slow science' approach, focusing on how this shift benefits the scientific enterprise. It provides a perspective on balancing the systemic demands of modern academia with the need for thoughtful, high-quality research.

advocacy Paper

Slow ethnography: A hut with a view

This resource explores the 'liberatory potential' of slow ethnography, drawing on two decades of fieldwork in the Maya lowlands to illustrate the value of place-based research. It argues that a slower pace allows researchers to better apply frameworks for 'studying up, down, and sideways,' leading to deeper intellectual and personal insights.

evidence Paper

A Call for Slow Scholarship: A Case Study on the Intensification of Academic Life and Its Implications for Pedagogy

This resource uses survey data from academic staff to provide empirical evidence of how the intensification of academic life impacts teaching and professional well-being. It utilizes these findings to support a case for 'slow scholarship' as a necessary response to the pressures of the modern neoliberal university.

advocacy Paper

Amateur Science in Activist Performance: Towards a Slow Science

This article advocates for the integration of activist performance and the concept of the 'amateur' into scientific practice as a way to achieve a more reflective 'slow science.' It argues that artistic gestures and vocabularies can provide a critical counter-discourse to standard models of public engagement and citizen science.

practice/tools Paper

Slow Delphi: An investigation into information behaviour and the Slow Movement

The authors introduce the 'Slow Delphi,' a novel methodological tool designed to elicit deep qualitative insights on complex topics through a more paced and reflective expert consensus process. The resource demonstrates this tool's application within the field of information science to explore how 'slow' principles can be integrated into professional information behavior.

teaching/training Paper

Being ‘Lazy’ and Slowing Down: Toward decolonizing time, our body, and pedagogy

This article proposes a pedagogical shift that centers embodied learning and the disruption of Eurocentric, neoliberal temporalities in the classroom. It provides a theoretical framework for educators to use 'slowing down' as a deliberate decolonizing practice that reconnects the physical body with the intellectual mind.

critique Paper

Misinterpretation of ‘slow science’ and ‘academic productivism’ may obstruct science in developing countries

This publication critiques the potential misapplication of 'slow science' and 'academic productivism' critiques within the specific context of developing countries. It warns that these concepts, if misinterpreted, could inadvertently create barriers to scientific development and institutional growth in regions with different academic pressures than the Global North.

advocacy Paper

Writing Slow Ontology

This resource proposes the concept of "Slow Ontology" as a philosophical framework for rethinking scholarly existence in response to the escalating pace of academic production. It argues for a shift from merely slowing down productivity to adopting a more deliberate way of "scholarly being" that prioritizes quality and depth over efficiency.

evidence Paper

Academic life in the fast lane: The experience of time and speed in British academia

This article presents empirical findings on how academics in the United Kingdom experience time and speed, distinguishing between oppressive systemic acceleration and rare energizing moments. It contributes to the literature by exploring the specific psychological and professional ambivalence that arises from the high-pressure environment of contemporary academia.

▸ Types of academic, non-academic, & alt-academic positions 7 / 7

There are many interesting career options beyond academia for those who are currently in the academic system. These can be research based (e.g. research for non-profit and for-profit organisations), teaching based (e.g. in schools, applied higher-education, and universities), science communication roles, data related roles (e.g. data steward, FAIR advocates, data scientist), and beyond!

overview Paper

Fed up and burnt out: ‘quiet quitting’ hits academia

This piece examines the emergence of the 'quiet quitting' trend within the academic workforce as a response to systemic burnout and exhaustion. It discusses how researchers are increasingly setting boundaries on their labor as a survival mechanism against the industry's traditional culture of overwork.

practice/tools Paper

Lessons for psychology laboratories from industrial laboratories

This resource identifies specific practices from industrial laboratory settings that can be adapted to improve data collection and quality within psychology and cognitive neuroscience laboratories. It offers actionable suggestions for researchers to enhance the technical rigor of their experimental processes to help mitigate the field's 'crisis of confidence.'

advocacy Paper

Postdocs in crisis: science cannot risk losing the next generation

This publication makes a forceful case for addressing the systemic crises facing postdoctoral researchers to prevent a loss of talent that would jeopardize the future of science. It argues for the necessity of valuing and protecting the next generation of scholars by improving their career stability and working conditions.

advocacy Letter

The mental health of PhD researchers demands urgent attention

This resource advocates for systemic changes to improve the mental health and well-being of doctoral researchers across academia. It emphasizes that the sustainability of the research ecosystem depends on addressing the high levels of stress and burnout in PhD programs.

overview Paper

Seeking an ‘exit plan’ for leaving academia amid coronavirus worries

This article provides an overview of the factors influencing researchers to seek career opportunities outside of academia in the wake of the global pandemic. It details the increasing interest in "exit plans" as a response to institutional instability and the precarious nature of academic employment.

Nature. (2023, March 15). Careers advice from scientists in industry. https://www.nature.com/collections/cgahibcfad

advocacy Paper

Quality research needs good working conditions

This publication makes the case that high-quality, reproducible research is fundamentally tied to the structural working conditions and well-being of researchers. It calls for institutional reforms that prioritize labor stability and mental health as essential prerequisites for maintaining scientific integrity and rigor.

4 Pre-analysis Planning

5 sub-clusters · 42 references

Pre-analysis planning entails laying out a complete methodology and analysis before a study has been undertaken. This facilitates transparency and removes several potential QRPs. When teaching, students should attain knowledge regarding what a pre-registration entails, why it is important to remove potential QRPs and how to address deviations from preregistered plans. There are 5 sub-clusters which aim to further parse the learning and teaching process:

▸ Preregistration 14 / 14

When a researcher preregisters their work, they typically upload a detailed project plan before the start of data collection. This plan includes (but is not limited to) hypotheses, data collection procedures, measures and manipulated variables, and the analysis plan.

overview Preprint

Merits and Limits of Preregistration for Visualization Research

This resource provides an overview of how preregistration can be integrated into visualization research, discussing both its potential to reduce questionable research practices and its practical challenges. It specifically addresses the nuances of applying open science standards to user studies and data visualization methodologies.

evidence Paper

Evidence for prereg posters as a platform for preregistration

This study presents empirical evidence supporting the effectiveness of preregistration posters as a mechanism for improving research transparency and planning. It demonstrates how this specific format allows researchers to receive critical feedback on their designs before data collection, thereby strengthening the resulting study.

Centre for Open Science. (n.d.). Preregistration. Www.cos.io. https://www.cos.io/initiatives/prereg

DeHaven, A. (2017). 10 Tips for Making a Great Preregistration. www.cos.io. https://www.cos.io/blog/10-preregistration-tips

DeHaven, A. (2018). One Preregistration to Rule Them All? Www.cos.io. https://www.cos.io/blog/one-preregistration-rule-them-all

overview Paper

Preregistering qualitative research

This article explores how the concept of preregistration can be adapted for qualitative research by addressing the tension between data-driven interpretation and a priori planning. It provides a foundational framework for maintaining the inductive flexibility of qualitative work while leveraging preregistration to enhance research transparency and credibility.

evidence Paper

Preregistering Qualitative Research: A Delphi Study

This study presents empirical findings from a Delphi consensus process aimed at determining which components of preregistration templates are most useful for qualitative researchers. It contributes evidence-based recommendations for designing flexible preregistration forms that accommodate the diverse methodological needs of the qualitative community.

practice/tools Preprint

Making the black box transparent: A template and tutorial for (pre-)registration of studies using Experience Sampling Methods (ESM)

This resource offers a specialized template and tutorial designed to guide researchers through the preregistration process for studies using Experience Sampling Methods (ESM) and Ecological Momentary Assessment (EMA). It aims to reduce the "black box" of researcher degrees of freedom by providing a structured framework for documenting complex longitudinal designs.

critique Paper

Campbell’s Law Explains the Replication Crisis: Pre-Registration Badges Are History Repeating

advocacy Preprint

Preregistration of analyses of preexisting data

This resource addresses the unique challenges of applying preregistration to the analysis of preexisting data, a common practice in psychology that differs from original data collection. It makes the case for the utility of preregistering such studies to maintain transparency, even when researchers cannot control aspects like experimental manipulation or sample size.

advocacy Paper

The preregistration revolution

This paper argues for the fundamental importance of preregistration as a solution to the common human bias of mistaking postdiction for prediction. It advocates for the practice as a necessary step to protect the credibility of scientific findings by ensuring that research questions and analysis plans are defined before outcomes are observed.

evidence Preprint

Some Data Indicating that Editors and Reviewers Do Not Check Preregistrations during the Review Process

This empirical study investigates the behaviors of editors and reviewers by analyzing how often they engage with preregistration plans during the peer-review process. It contributes significant evidence of a 'compliance gap,' showing that even when studies are preregistered, the plans are rarely scrutinized during formal review.

Van den Akker, O. R., Weston, S., Campbell, L., Chopik, B., Damian, R., Davis-Kean, P., Hall, A., Kosie, J., Kruse, E., Olsen, J., Ritchie, S., Valentine, K., Van ’t Veer, A., & Bakker, M. (2021). Preregistration of secondary data analysis: A template and tutorial. Meta-Psychology, 5. https://doi.org/10.15626/MP.2020.2625

practice/tools Paper

Pre-registration in social psychology—A discussion and suggested template

This paper offers a specialized template and implementation guidelines for preregistration tailored to the methodological standards of social psychology. It discusses the relative advantages of reviewed and unreviewed preregistration models, providing researchers with actionable advice on how to structure their research plans for maximum transparency.

▸ Purpose of pre-analysis planning 10 / 10

Distinguishing exploratory and confirmatory analyses. One proposed benefit to this is transparency. FORRT does not propose one has more value over the other.

advocacy Paper

Making Prospective Registration of Observational Research a Reality

This article argues for the ethical and scientific necessity of prospectively registering observational studies in the health sciences, a domain where the practice is historically underutilized. It specifically identifies funders and ethics committees as the primary drivers for mandating the public disclosure of study protocols and analysis plans.

practice/tools Paper

How can we make sound replication decisions?

This perspective piece introduces a conceptual framework to guide researchers and institutions in making strategic decisions about which findings should be prioritized for replication. It provides actionable criteria for weighing scientific values against practical constraints to ensure that limited research resources are allocated effectively.

Graf, C. (2017). 8 Answers About Registered Reports and Research Preregistration. Wiley Library. www.wiley.com. https://www.wiley.com/network/researchers/being-a-peer-reviewer/8-answers-about-registered-reports-research-preregistration-and-why-both-are-important

advocacy Preprint

The Value of Preregistration for Psychological Science: A Conceptual Analysis

This article provides a philosophical defense of preregistration in psychological science by applying the principles of error statistics. It argues that preregistration is essential for maintaining the severity of a test, ensuring that the probability of finding support for a theory is not artificially inflated by post-hoc analytical choices.

Lin, W., & Green, D. P. (2016). Standard Operating Procedures: A Safety Net for Pre-Analysis Plans. Political Science and Politics, 49(3), 495–500. https://doi.org/10.1017/S1049096516000810

policies Paper

Registered Reports

This resource introduces the Registered Reports publishing model, which shifts peer review to the stage before data collection to combat publication bias and the file-drawer problem. It presents a collection of social psychology replications as evidence of how this format successfully prioritizes methodological rigor and open science practices over the direction of the results.

advocacy Paper

The preregistration revolution

overview Paper

How scientists fool themselves – and how they can stop

Wagenmakers, E. J., & Dutilh, G. (2016). Seven selfish reasons for preregistration. APS Observer, 29(9). https://www.psychologicalscience.org/observer/seven-selfish-reasons-for-preregistration

practice/tools Paper

Degrees of Freedom in Planning, Running, Analyzing, and Reporting Psychological Studies: A Checklist to Avoid p-Hacking

▸ Registered reports 10 / 10

A publishing format consisting of a preregistration with a specific journal that undergoes peer review. Specifically, the journal reviews the introduction and methodology and upon in-principle acceptance (IPA, in stage 1), it agrees to publish the study assuming the preregistration is followed and deviations are reported (stage 2 acceptance).

evidence Paper

Registered Replication Report

This resource provides empirical data from a large-scale multi-lab replication effort to estimate the true effect size of the verbal overshadowing phenomenon. It addresses discrepancies between original findings and subsequent research by using a pre-registered, standardized protocol across multiple sites to ensure a high-powered and unbiased assessment.

Anon. (n.d.). Wiley’s Reviewing Registered Reports. https://authorservices.wiley.com/Reviewers/journal-reviewers/how-to-perform-a-peer-review/reviewing-registered-reports.html

policies Editorial

Registered Reports: A new publishing initiative at Cortex

This article introduces the "Registered Report" article format at the journal Cortex, explaining the policy of peer-reviewing research protocols before data collection. It details how this editorial shift aims to prioritize the quality of research questions and methods over the statistical significance of the results.

advocacy Editorial

Registered Reports: Realigning incentives in scientific publishing

This foundational paper argues for Registered Reports as a mechanism to eliminate publication bias and p-hacking by decoupling the decision to publish from the study results. It makes the case for realigning scientific incentives so that professional success is determined by the quality of research questions and methods rather than the direction of findings.

D. Chambers, C., Feredoes, E., D. Muthukumaraswamy, S., & J. Etchells, P. (2014). Instead of “playing the game” it is time to change the rules: Registered Reports at AIMS Neuroscience and beyond. AIMS Neuroscience, 1(1), 4–17. https://doi.org/10.3934/Neuroscience.2014.1.4

advocacy Editorial

Registered Reports: Realigning incentives in scientific publishing

overview Paper

The past, present and future of Registered Reports

This article provides a comprehensive overview of the Registered Reports publication model, tracing its historical development and evaluating its effectiveness in mitigating publication bias. It serves as both a theoretical reflection on the format's impact and a practical guide for researchers, editors, and reviewers navigating the pre-acceptance process.

practice/tools Editorial

Registered Reports: A new chapter at Ecology & Evolution

This article provides a walkthrough of the Registered Report process within the field of ecology and evolution by documenting the experiences of an editor, author, and student. It serves as a practical guide for researchers in these disciplines to understand the logistical and pedagogical implications of adopting pre-accepted study protocols.

advocacy Paper

Registered reports for qualitative research

This resource advocates for the adoption of the Registered Reports publishing format within qualitative research to reduce publication bias and increase the visibility of non-quantitative studies. It explores how the model of pre-data collection peer review can be effectively tailored to support the rigor of qualitative research designs.

Zotero. (n.d.). OSF Collection of Registered Reports. Zotero. https://www.zotero.org/groups/479248/osf/collections/KEJP68G9/search/right/titleCreatorYear

▸ Study Design 1 / 1

Design elements should be before data collection so that confirmatory claims are credible and deviations are interpretable. This includes choosing appropriate sample sizes, conditions, and measures prior to registration, and planning how to handle deviations

practice/tools Preprint

When and How to Deviate from a Preregistration

This resource offers a principled approach to handling changes made after a study has been preregistered, focusing on how to report these deviations without undermining research integrity. It provides specific examples of how to document deviations and evaluate their impact on the severity of statistical tests and the validity of scientific inferences.

▸ Understanding the types of preregistration and writing one. 7 / 7

Practical materials and resources to move beyond the theory of preregistration, into conducting a preregistered study.

Call, M., DeHaven, A. C., Pfeiffer, N., Mellor, D. T., & Lowrey, O. (2023, October 19). Templates of OSF registration forms. OSF. https://osf.io/zab38/

Centre for Open Science. (n.d.). Registered Reports: Peer review before results are known to align scientific values and practices. Www.cos.io. https://www.cos.io/initiatives/registered-reports

practice/tools Preprint

When and How to Deviate from a Preregistration

advocacy Preprint

Preregistration of analyses of preexisting data

advocacy Paper

The preregistration revolution

overview Preprint

Pre-registration and Registered Reports: a Primer from UKRN

This primer serves as an introductory guide to preregistration and Registered Reports, explaining how these practices help mitigate selective reporting and questionable research practices. It provides a foundational overview for researchers looking to incorporate these transparency-enhancing methodologies into their standard workflow.

practice/tools Letter

Increasing the transparency of systematic reviews: presenting a generalized registration form

This resource provides a comprehensive and flexible registration form designed to enhance transparency in systematic reviews regardless of the discipline or review type. By prioritizing broad applicability over niche requirements, it allows researchers in diverse fields like economics, law, or physics to document their protocols effectively even when standard forms are not suitable for their specific data types.

5 Transparency and reproducibility in computation and analysis

5 sub-clusters · 54 references

Attainment of the how-to basics of reproducible reports and analyses. It requires students to move towards transparent and scripted analysis practices in quantitative research. There are 5 sub-clusters which aim to further parse the learning and teaching process:

▸ Analysis and reporting in qualitative research 6 / 6

By documenting and reporting research processes in qualitative research, transparency and credibility in qualitative research reports is ensured. Topics include using agreed reporting standards, demonstrating methodological rigor, and recent calls to integrate qualitative methods into the open science movement. The emphasis is on making qualitative research as trustworthy and open as context permits, without forcing inappropriate replication model

Anon. (n.d.). OSL Open Stats Lab. Trinity Sites. https://sites.trinity.edu/osl/

Anon. (n.d.). Software carpentry. Software Carpentry. https://software-carpentry.org/

Anon. (n.d.). Teaching integrity in empirical research. Project TIER | Teaching Integrity in Empirical Research. https://www.projecttier.org/

practice/tools Paper

Navigating the messy swamp of qualitative research: Are generic reporting standards the answer?A review essay of the book Reporting Qualitative Research in Psychology: How to Meet APA Style Journal Article Reporting Standards, Revised Edition, by Heidi M. Levitt, Washington, DC, American Psychological Association, 2020, 173pp., $29.99 (paperback), ISBN: 978-1-4338-3343-4

This article identifies and corrects ten common pitfalls in published reflexive thematic analysis research to clarify what constitutes high-quality practice. It specifically challenges the application of quantitative-style metrics like inter-rater reliability, arguing for quality standards that align with the qualitative and reflexive nature of the method.

O’Brien, B. C., Harris, I. B., Beckman, T. J., Reed, D. A., & Cook, D. A. (2014). Standards for Reporting Qualitative Research. Academic Medicine, 89(9), 1245–1251. https://doi.org/10.1097/ACM.0000000000000388

practice/tools Paper

Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups

This resource provides the COREQ checklist, a highly specific 32-item tool dedicated to improving the reporting quality of studies specifically using interviews and focus groups. It ensures researchers transparently document critical details about the research team, study design, and data analysis process to facilitate better appraisal and synthesis of qualitative work.

▸ Computational reproducibility 35 / 35

Making sure anyone can reproduce quantitative analyses through things like well-commented scripts, writing codebooks, version control, literate programming (e.g. Quarto), reproducible computational environment (containers, package managers), and automated data pipelines.

advocacy Paper

Open is not enough

critique Paper

Determining Validity in Qualitative Inquiry

This review essay evaluates the American Psychological Association’s reporting standards for qualitative research in psychology. It critiques the utility of standardized, generic templates in a diverse methodological field, questioning whether such standards can accommodate the inherent complexity and messiness of qualitative inquiry.

Data Carpentry. (n.d.). Reproducible research in R workshop overview. https://datacarpentry.org/rr-workshop/

practice/tools Paper

Qualitative Research and the Question of Rigor

This resource provides a framework for selecting qualitative validity procedures by cross-referencing a researcher's philosophical lens with their adopted paradigm. It details nine specific strategies—such as triangulation, member checking, and thick description—to help researchers ensure and demonstrate the rigor of their findings.

Fox, N. (2018). Writing reproducible scientific papers in R. YouTube. https://www.youtube.com/playlist?list=PLmvNihjFsoM5hpQdqoI7onL4oXDSQ0ym8

practice/tools Preprint

A Transparency Checklist for Qualitative Research

This resource introduces and compares open-source software tools for qualitative data analysis, specifically the 'qcoder' R package and 'Taguette' application. It discusses how free and extensible tools can improve equity in research and outlines the benefits of adopting open-source workflows to increase the transparency of qualitative inquiry.

Gandrud, C. (2016). Reproducible research with R and R Studio. CRC Press.

practice/tools Paper

Valid replications require valid methods: Recommendations for best methodological practices with lab experiments.

Harrison, P., Barugahare, A., & Tsyganov, K., (2020). Reproducible research in R. Monash Data Fluency. https://monashdatafluency.github.io/r-rep-res/index.html

practice/tools Paper

Integrating Qualitative Methods and Open Science: Five Principles for More Trustworthy Research*

This article provides specific recommendations for improving the methodological rigor of laboratory experiments to ensure they are robust enough for valid replication. It argues that the success of the open science movement depends not only on statistical transparency but also on the fundamental validity of the experimental methods employed.

overview Paper

TIER2: enhancing Trust, Integrity and Efficiency in Research through next-level Reproducibility

This resource describes TIER2, a European Commission-funded project designed to investigate and improve research reproducibility across the social, life, and computer sciences. It outlines a systematic approach to developing tools and guidelines for diverse stakeholders to foster a more robust and trustworthy scientific ecosystem.

practice/tools Paper

Primer on Reproducible Research in R: Enhancing Transparency and Scientific Rigor

This publication provides a practical tutorial on using the R programming language to achieve computational reproducibility, specifically targeting researchers who may have limited coding experience. It offers a step-by-step primer on documenting research procedures and contexts more comprehensively to overcome the transparency limitations inherent in traditional publication practices.

practice/tools Paper

Best Practices for Computational Science: Software Infrastructure and Environments for Reproducible and Extensible Research

This article synthesizes a formalized set of best practice recommendations for managing software infrastructure and computational environments to ensure research is reproducible and extensible. It provides a practical framework for computational scientists across disciplines to standardize the dissemination of their code and data.

overview Paper

A Review of the Quality Indicators of Rigor in Qualitative Research

This resource synthesizes key indicators of rigor and quality specifically tailored for qualitative research within the health professions education field. It maps out best practices across the entire research process—from question formulation to final reporting—to minimize researcher bias and enhance the overall trustworthiness of findings.

Kapiszewski, D., & Karcher, S. (2020). Transparency in Practice in Qualitative Research. PS: Political Science & Politics, 54(2), 285–291. https://doi.org/10.1017/S1049096520000955

overview Paper

Journal article reporting standards for qualitative primary, qualitative meta-analytic, and mixed methods research in psychology: The APA Publications and Communications Board task force report.

This resource provides a review of rigor attributes and best practices for qualitative research design specifically within the context of health professions education. It outlines how a strong conceptual framework and iterative data analysis can minimize bias and enhance the trustworthiness of qualitative findings.

Levitt, H. M. (2020). Reporting qualitative research in psychology: How to meet APA style journal article reporting standards. American Psychological Association.

overview Paper

But is it rigorous? Trustworthiness and authenticity in naturalistic evaluation

This resource introduces foundational criteria for rigor in naturalistic inquiry, specifically proposing the concepts of trustworthiness and authenticity as alternatives to traditional quantitative standards. It provides a conceptual framework that allows qualitative researchers to demonstrate the quality and integrity of their work while remaining consistent with the paradigm’s unique assumptions.

advocacy Paper

Replication is relevant to qualitative research

Malterud, K. (2001). Qualitative research: standards, challenges, and guidelines. The Lancet, 358(9280), 483–488. https://doi.org/10.1016/S0140-6736(01)05627-6

practice/tools Paper

AI chatbots can boost scientific coding

This article explores how AI chatbots can be integrated into scientific workflows to assist with programming tasks, debugging, and documentation. It provides actionable insights into how these tools can lower the barrier to entry for computational methods and enhance the reproducibility of research code.

overview Paper

Establishing methodological rigour in international qualitative nursing research: a case study from Ghana

This resource surveys the fundamental standards and common challenges inherent in conducting high-quality qualitative research across various fields. It provides general guidelines to help researchers navigate the complexities of qualitative methodology while maintaining rigor and transparency.

overview Book

Denzin, Norman

This resource offers an integrated account of how a researcher’s ethical obligations and epistemological commitments should shape their approach to research openness. It argues for a flexible understanding of transparency in political science that accounts for different ways of knowing and the specificities of research with human participants.

Navarro, D. (2019, January 11). A tutorial for psychology students and other beginners. (version 0.6.1). Learning Statistics with R. https://learningstatisticswithr.com/book/

critique Editorial

Editorial Essay: The Tumult over Transparency: Decoupling Transparency from Replication in Establishing Trustworthy Qualitative Research

overview Paper

Reproducible Research in Computational Science

This article proposes reproducibility as a necessary minimum standard for evaluating scientific claims within computational science, especially when independent replication is unfeasible. It highlights how the increasing complexity of computational work necessitates transparent sharing of code and data to ensure the validity of published findings.

PsyTeachR. (2021). Data Skills for Reproducible Science. PsyTeachR. https://psyteachr.github.io/msc-data-skills/

evidence Paper

Conducting secondary analysis of qualitative data: Should we, can we, and how?

This critical interpretive synthesis analyzes 71 published articles to provide empirical evidence on the current state and methodologies of qualitative secondary data analysis. It systematically maps how researchers navigate methodological and ethical concerns, offering a data-driven look at the prevalence and execution of these practices across disciplines.

teaching/training Editorial

Enhancing the quality and transparency of qualitative research methods in health psychology

This resource introduces a digital research environment built on Semantic MediaWiki designed to facilitate collaborative analysis using the method of objective hermeneutics. It specifically explores how this platform enhances transparency and supports student learning within research-based university seminars.

Stahl, N. A., & King, J. R. (2020). Expanding approaches for research: Understanding and using trustworthiness in qualitative research. Journal of Developmental Education, 44(1), 26–28. https://files.eric.ed.gov/fulltext/EJ1320570.pdf

critique Paper

Rethinking Transparency and Rigor from a Qualitative Open Science Perspective

Stodden, V., Miguez, S. (2014). Best Practices for Computational Science: Software Infrastructure and Environments for Reproducible and Extensible Research. Journal of Open Research Software (1), e21. https://openresearchsoftware.metajnl.com/articles/10.5334/jors.ay

Tamminen, K. A., Bundon, A., Smith, B., McDonough, M. H., Poucher, Z. A., & Atkinson, M. (2021). Considerations for making informed choices about engaging in open qualitative research. Qualitative Research in Sport, Exercise and Health, 13(5), 864–886. https://doi.org/10.1080/2159676X.2021.1901138

practice/tools Website

The Turing Way: A Handbook for Reproducible Data Science

This collaboratively written handbook serves as a comprehensive guide to reproducible, ethical, and collaborative data science practices throughout the research lifecycle. It offers an extensive collection of actionable tutorials, checklists, and templates designed to make open science practices achievable for researchers across all fields.

practice/tools Preprint

Good enough practices in scientific computing

This resource outlines a minimum set of practical, "good enough" computing habits designed to improve data management and analysis reproducibility for researchers across all scientific disciplines. It focuses on accessible entry points for scientists without formal computer science training, covering topics like file organization, documentation, and versioning.

▸ Free and open source software 5 / 5

Free and open source software is a foundation for reproducible research: open tooling lowers access barriers, enables community review, and supports longevity through transparent code, issue tracking, and forking.

Chao, L. (2009). Utilizing open source tools for online teaching and learning Information Science. Information Science Reference.

Huang, R. (2016). RQDA: R-based qualitative data analysis (Version 0.2-8) [Computer software]. R Project. http://rqda.r-forge.r-project.org/

practice/tools Paper

A Quick Guide to Software Licensing for the Scientist-Programmer

This guide simplifies the complex landscape of software licensing for researchers who develop their own tools, helping them choose appropriate legal frameworks for their work. It provides practical advice on navigating institutional requirements to ensure that scientific software can be legally shared and reused.

practice/tools Paper

<scp>OpenSAFELY</scp>: A platform for analysing electronic health records designed for reproducible research

This paper introduces OpenSAFELY, a secure software platform designed to enable reproducible and transparent analysis of electronic health records (EHRs) at scale. It addresses technical and privacy barriers in health data research by providing a framework where analysis code is automatically shared and execution is standardized.

advocacy Book

The cathedral and the bazaar

This seminal essay contrasts two models of software development—the closed "cathedral" and the collaborative "bazaar"—to explain the success of the Linux kernel. It articulates the fundamental principles of peer review and decentralized community iteration that underpin the effectiveness of the open-source movement.

▸ Research software engineering 2 / 2

Covers the emerging role of Research Software Engineers, professionals who develop software for research purposes. Emphasizes best practices in coding (testing, version control, documentation) as integral to research transparency. Also discusses how RSEs bridge the gap between traditional IT and academic science, ensuring that scientific software is reliable and sustainable

evidence Paper

A survey of the state of the practice for research software in the United States

This study presents empirical data from a large-scale survey of researchers to identify the structural and technical challenges involved in maintaining sustainable research software. It highlights a widespread, unmet need for formalized software training and identifies critical gaps in institutional support for the researchers who build these tools.

Lamprecht, A.-L., Garcia, L., Kuzak, M., Martinez, C., Arcila, R., Martin Del Pico, E., Dominguez Del Angel, V., van de Sandt, S., Ison, J., Martinez, P. A., McQuilton, P., Valencia, A., Harrow, J., Psomopoulos, F., Gelpi, J. Ll., Chue Hong, N., Goble, C., & Capella-Gutierrez, S. (2019). Towards FAIR principles for research software. Data Science, 3(1), 37–59. https://doi.org/10.3233/DS-190026

▸ Tools to check yourself and others 6 / 6

Detecting errors in the literature, and preventing them from entering the literature by checking your own work. Includes tools such as statcheck.io, GRIM, and SPRITE to detect errors in reporting of statistics.

practice/tools Paper

The GRIM Test

This paper introduces the Granularity-Related Inconsistency of Means (GRIM) test, a simple mathematical technique used to detect reporting errors in summary statistics derived from integer-based data like Likert scales. It provides a low-threshold method for researchers and peer reviewers to verify the mathematical consistency of means reported in psychological research.

practice/tools Paper

How quality control could save your science

This resource outlines practical quality control measures researchers can implement to prevent common errors and improve the reproducibility of their work. It highlights specific steps such as better documentation, blinding, and validating reagents to bolster scientific rigor.

Nuijten, M. B., Van Assen, M. A. L. M., Hartgerink, C. H. J., Epskamp, S., & Wicherts, J. M. (2017). The validity of the tool “statcheck” in discovering statistical reporting inconsistencies. https://psyarxiv.com/tcxaj/.

overview Paper

Primer on Reproducibility: Trends, Tools, and Some Tips and Tricks

This primer provides a comprehensive introduction to the reproducibility crisis, detailing its multifaceted causes and implications for the scientific community. It offers a curated collection of trends and practical tools designed to help researchers enhance the reproducibility of their own work.

teaching/training Preprint

Error Tight: Exercises for Lab Groups to Prevent Research Mistakes

This resource introduces a structured set of exercises based on human factors research to help lab groups identify and prevent common procedural errors. It focuses on building a collaborative lab culture that prioritizes error detection through systematic workflow audits and collective accountability.

evidence Paper

Statistical heartburn: an attempt to digest four pizza publications from the Cornell Food and Brand Lab

This article demonstrates the application of simple statistical checks to identify inconsistencies and potential errors within a specific set of published research papers. By detailing a forensic analysis of these case studies, it provides a model for how researchers can independently verify the plausibility and accuracy of reported data.

6 FAIR data and materials

7 sub-clusters · 82 references

Students will learn about FAIR data (and education/research materials) principles that address Findability, Accessibility, Interoperability, and Reusability; engage with reasons to share data, the initiatives designed to increase scientific openness; as well as of possible privacy and security considerations together with anonymization procedures. There are 7 sub-clusters which aim to further parse the learning and teaching process:

▸ Reasons to share data and materials 18 / 18

Sharing data and research materials is beneficial for science and society. Open data can enable validation of results, inspire new discoveries through re-use, increase researcher credit (e.g., via data citations), and promote transparency that enhances trust. Key studies have shown that papers with shared data receive more citations and foster broader collaboration.

overview Paper

Revisiting Qualitative Data Reuse

This article evaluates the rapid growth and mainstream acceptance of qualitative data reuse in the United Kingdom, identifying the specific policy and cultural drivers behind this shift. It explores how secondary analysis provides opportunities to gain new methodological insights and substantive findings from existing raw research materials.

evidence Paper

The citation advantage of linking publications to research data

This study empirically quantifies the citation premium associated with articles that provide data availability statements and functional links to external data repositories. By analyzing over half a million articles using an automated system, it provides large-scale evidence that transparent data sharing is linked to increased scientific impact across disciplines.

advocacy Paper

An Agenda for Open Science in Communication

DuBois, J. M., Strait, M., & Walsh, H. (2018). Is it time to share qualitative research data? Qualitative Psychology, 5(3), 380–393. https://doi.org/10.1037%2Fqup0000076

evidence Paper

Many Labs 2: Investigating Variation in Replicability Across Samples and Settings

evidence Paper

Investigating Variation in Replicability

practice/tools Paper

Data: Sharing Is Caring

This resource provides actionable guidance on implementing data management best practices throughout the research lifecycle to facilitate easier data sharing. It emphasizes the importance of formal citations for data to ensure researchers receive professional recognition for the intellectual labor involved in data curation.

overview Paper

Transparency: The Emerging Third Dimension of Open Science and Open Data

This paper proposes a three-dimensional model of open science by incorporating transparency as a distinct axis alongside open access and open data. It provides a conceptual framework that links transparency to research integrity and policy contexts, helping to clarify its role within the broader open science movement.

McKiernan, E. C., Bourne, P. E., Brown, C. T., Buck, S., Kenall, A., Lin, J., McDougall, D., Nosek, B. A., Ram, K., Soderberg, C. K., Spies, J. R., Thaney, K., Updegrove, A., Woo, K. H., & Yarkoni, T. (2016). How open science helps researchers succeed. ELife, 5. CLOCKSS. https://doi.org/10.7554/eLife.16800

advocacy Paper

The Reproducibility Crisis in Science: A Statistical Counterattack

evidence Paper

Data reuse and the open data citation advantage

This study provides a statistically rigorous analysis of the 'citation advantage' associated with making research datasets publicly available. It contributes high-powered empirical evidence that documents the link between data sharing and increased scholarly impact, while also examining the temporal patterns of data reuse.

Rosenberg, J. M., Borchers, C., Burchfield, M. A., Anderson, D., Stegenga, S. M., & Fischer, C. (2022). Posts About Students on Facebook: A Data Ethics Perspective. Educational Researcher, 51(8), 547–550. https://doi.org/10.3102/0013189X221120538

practice/tools Paper

The what, why, and how of born-open data

This resource introduces the concept of "born-open" data, which involves automating the archiving of datasets to public repositories at the moment of creation. It provides a practical workflow to reduce the time and effort required for data sharing while ensuring complete transparency from the pilot phase through to final analysis.

Stodden, V. C. (2011). Trust Your Science? Open Your Data and Code. Columbia University. https://doi.org/10.7916/D8CJ8Q0P

evidence Preprint

The academic, economic and societal impacts of Open Access: an evidence-based review

This comprehensive review evaluates the academic, economic, and societal consequences of open access, providing a data-driven foundation for ongoing policy debates. It synthesizes empirical findings from a wide range of meta-research to highlight the tangible benefits and potential challenges of transitioning away from paywalled research.

critique Paper

The Value of Openness in Open Science

This paper offers a philosophical caution against uncritical openness, arguing that transparency initiatives can sometimes have unintended detrimental effects on scientific epistemology. It proposes a framework where openness is treated as one governing value among many that must be carefully balanced to preserve the overall integrity of the scientific process.

overview Paper

Open science, data sharing and solidarity: who benefits?

This resource summarizes multidisciplinary discussions regarding the intersection of data sharing, ethics, and solidarity within the life sciences. It specifically addresses the distribution of benefits in open science, questioning how the movement impacts different global populations and who truly gains from the sharing of personal and research data.

overview Paper

Promises and pitfalls of data sharing in qualitative research

This resource provides a balanced overview of the advantages and ethical complexities associated with data sharing in qualitative research. It specifically examines the tension between the push for research transparency and the need to protect sensitive information, outlining the unique risks and responsibilities qualitative researchers must manage when sharing their work.

▸ Reasons not to share: Privacy and security considerations 27 / 27

Open sharing of data sometimes poses legitimate privacy and security concerns. These include protecting participant privacy, honoring cultural ownership of data, and security risks. It emphasizes that not all data can or should be open, and ethical frameworks guide decisions in these cases.

practice/tools Paper

A context-consent meta-framework for designing open (qualitative) data studies

This article introduces a context-consent meta-framework designed to assist researchers in the ethical planning and execution of studies involving open qualitative data. Based on interviews with qualitative psychologists, the resource specifically recommends conducting 'archaeologies of context and consent' to evaluate the suitability of datasets for secondary use.

overview Paper

Open Data in Qualitative Research

This resource surveys the evolving landscape of open data in qualitative research, discussing the impact of technological changes and funder mandates on research practices. It contrasts the benefits of transparency and accountability with the unique challenges posed by the non-standardized nature of qualitative data.

critique Preprint

Open With Care! Consent, Context, and Co-production in Open Qualitative Research

evidence Paper

Participation in patient support forums may put rare disease patient data at risk of re-identification

This study provides empirical evidence of re-identification risks for rare disease patients who participate in online support forums. It identifies how specific health data patterns and ICD codes act as quasi-identifiers, and proposes practical adjustments to forum guidelines and organizational privacy measures to protect patient anonymity.

overview Paper

Aspects of Data Ethics in a Changing World: Where Are We Now?

This paper surveys the current ethical landscape of data science, emphasizing how the ubiquity of data and advanced extraction tools creates a pressing need for oversight. It specifically examines the tension between the potential for technological progress and the risks of misuse in a data-saturated world.

practice/tools Paper

Applying the ‘CARE Principles for Indigenous Data Governance’ to ecology and biodiversity research

This article provides actionable guidance on integrating the CARE Principles for Indigenous Data Governance into ecology and biodiversity research. It explains how researchers can move beyond standard open data practices to prioritize Indigenous sovereignty, benefit-sharing, and ethical stewardship of biodiversity data.

policies Journal Article

Involving Research Stakeholders in Developing Policy on Sharing Public Health Research Data in Kenya

evidence Journal Article

Research Stakeholders’ Views on Benefits and Challenges for Public Health Research Data Sharing in Kenya: The Importance of Trust and Social Relations

practice/tools Paper

Open Sharing of Data on Close Relationships and Other Sensitive Social Psychological Topics: Challenges, Tools, and Future Directions

This article details an adversarial collaboration that navigated the complexities of sharing sensitive social psychological data from a speed-dating study. It provides a reflective case study of the ethical decision-making process, highlighting how even shared data can prompt retroactive concerns about participant confidentiality.

critique Paper

A dataset without a code book: ethnography and open science

Drawing on research in sexual violence, this paper argues that the ontological and epistemic foundations of open science are often incompatible with ethnographic practice. It identifies three distinct logics of open data—epistemic, political-economic, and regulatory—to demonstrate how standardized data-sharing expectations can conflict with the nature of knowledge production in ethnography.

overview Paper

Rethinking Data Sharing and Human Participant Protection in Social Science Research: Applications from the Qualitative Realm

This article evaluates how traditional participant protection models designed for quantitative data fail to address the unique privacy requirements of qualitative research. It proposes a rethinking of data-sharing frameworks to better accommodate the specific epistemic and ethical nuances of the qualitative social science realm.

advocacy Preprint

Open Science as Confused: Contradictory and Conflicting Discourses in Open Science Guidance to Researchers

critique Paper

The challenges of open data sharing for qualitative researchers

practice/tools Paper

Secondary analysis of qualitative data: a valuable method for exploring sensitive issues with an elusive population?

This publication provides a procedural demonstration of how to conduct secondary analysis on qualitative datasets, focusing on the often-undocumented 'how-to' of the research process. It specifically illustrates the value of this methodology for exploring sensitive issues and reaching elusive populations by repurposing existing longitudinal transcripts.

McGrath, C., & Nilsonne, G. (2018). Data sharing in qualitative research: Opportunities and concerns. MedEdPublish, 7(255), 255.

critique Preprint

Adversarial reanalysis and the challenge of open data in regulatory science

This paper examines the risks associated with open data mandates in the specific context of environmental regulatory science, where transparency requirements can be used as 'Trojan Horses' to undermine scientific evidence. It distinguishes between replication and reanalysis to highlight how adversarial reanalysis can be weaponized to exclude critical studies from the policy-making process.

evidence Paper

Research Participant Views regarding Qualitative Data Sharing

This study explores the under-researched area of participant attitudes toward qualitative data sharing through interviews with individuals involved in sensitive research. It highlights that participants value the potential impact of their data but demand strong protections regarding confidentiality and the prevention of data misuse by secondary users.

practice/tools Paper

#MeToo Online Disclosures: A Survivor-Informed Approach to Open Science Practices and Ethical Use of Social Media Data

This resource proposes a survivor-informed framework for the ethical collection, storage, and sharing of sensitive social media data, specifically focusing on sexual violence disclosures. It addresses the unique ethical challenges of applying open science principles to public digital data that may carry significant safety and privacy risks for the individuals who posted them.

critique Paper

Reflection over compliance: Critiquing mandatory data sharing policies for qualitative research

critique Paper

When open data closes the door: A critical examination of the past, present and the potential future for open data guidelines in journals

This paper provides a critical evaluation of how current journal open data guidelines often overlook the specific ethical and theoretical nuances of qualitative research. It uses a content analysis of 261 journals to demonstrate the disconnect between broad open-science mandates and the practical realities of qualitative social psychology.

overview Paper

Ethical aspects of data sharing and research participant protections.

This article provides a comprehensive overview of how open data practices intersect with established ethical frameworks for human subjects research. It explores the reinterpretation of respect, beneficence, and justice in the context of protecting participant privacy while maximizing the value of shared research data.

evidence Paper

Authorial and institutional stratification in open access publishing: the case of global health research

This research identifies socioeconomic inequalities in open access publishing by showing how institutional prestige and funding levels correlate with the type of OA model authors can afford. It demonstrates that authors from lower-ranked institutions are disproportionately excluded from gold OA options, potentially exacerbating global disparities in research visibility.

advocacy Paper

Generating evidence on privacy outcomes to inform privacy risk management: A way forward?

This resource advocates for the development of empirical evidence to inform privacy risk management strategies in data sharing. It proposes a shift toward a more systematic, evidence-based approach to privacy assessments to ensure that data protection measures are effective without being unnecessarily restrictive.

evidence Paper

Qualitative Data Sharing: Participant Understanding, Motivation, and Consent

This empirical study investigates how research participants in a sensitive context—abortion reporting—understand and perceive the sharing of their qualitative data. By analyzing participant motivations and responses to consent requests, it provides evidence-based insights into the complexities of obtaining informed consent for data reuse.

evidence Preprint

Diffusion of ethical governance policy on sharing of biological materials and related data for biomedical research

This study examines the diffusion of ethical norms regarding the sharing of human biological materials by analyzing national policy frameworks in Guinea, Argentina, India, and Malawi. It uses an analytical framework to demonstrate how international expert communities influence the adoption and adaptation of global governance standards at the domestic level.

overview Paper

Enabling Open-Science Initiatives in Clinical Psychology and Psychiatry Without Sacrificing Patients’ Privacy: Current Practices and Future Challenges

This article reviews the unique privacy risks associated with open data sharing in the fields of clinical psychology and psychiatry. It evaluates current governance mechanisms and technical strategies that allow researchers to participate in open science initiatives while maintaining necessary patient confidentiality.

▸ Licenses and reuse 11 / 11

Licensing determines how others may access, cite, remix, and redistribute your work. This section orients you to data/code/materials licenses (e.g., CC BY/CC0), data-use agreements, and rights/obligations that shape ethical, legally sound reuse, especially for qualitative and sensitive data

Anon. (n.d.). Can I reuse someone else’s research data? OpenAIRE. https://www.openaire.eu/can-i-reuse-someone-else-research-data

overview Paper

Getting the most from archived qualitative data: epistemological, practical and professional obstacles

This resource surveys the landscape of qualitative secondary analysis, identifying the specific epistemological, professional, and practical hurdles that hinder its wider adoption. It examines the evolving status of qualitative research and the barriers researchers face when attempting to maximize the utility of archived data.

overview Paper

Creative Commons licenses and the non-commercial condition: Implications for the re-use of biodiversity information

This analysis investigates the impact of the Creative Commons 'non-commercial' (NC) license condition on the interoperability and reuse of biodiversity information. It clarifies common misconceptions regarding the legal definition of the NC clause and highlights how its use can inadvertently create barriers to scientific progress.

evidence Paper

Data Sharing in Psychology: A Survey on Barriers and Preconditions

This study provides empirical findings from a survey of 600 psychology authors regarding their data sharing habits and perceived obstacles. It highlights specific barriers such as the lack of established disciplinary norms, the effort required for curation, and the preference for sharing data only upon individual request.

evidence Paper

Many Labs 2: Investigating Variation in Replicability Across Samples and Settings

evidence Paper

Investigating Variation in Replicability

evidence Paper

Barriers and facilitators to qualitative data sharing in the United States: A survey of qualitative researchers

This study provides empirical evidence on the current state of qualitative data sharing by surveying 425 US-based researchers about their experiences and concerns. It identifies specific barriers, such as a lack of participant permission and concerns over sensitive data, that contribute to the low prevalence of repository-based sharing.

Stodden, V. (2009). The Legal Framework for Reproducible Scientific Research: Licensing and Copyright. Computing in Science & Engineering, 11(1), 35–40. https://doi.org/10.1109/MCSE.2009.19

Wicherts, J. M., Borsboom, D., Kats, J., & Molenaar, D. (2006). The poor availability of psychological research data for reanalysis. American Psychologist, 61(7), 726–728. https://doi.org/10.1037/0003-066X.61.7.726

Whylly, K. E., Karcher, S., & Renbarger, R. (2023, January 25). Data sharing for qualitative research: Webinar and panel. Center for Open Science. https://youtu.be/eWZvmSIfhQY

practice/tools Paper

Degrees of Freedom in Planning, Running, Analyzing, and Reporting Psychological Studies: A Checklist to Avoid p-Hacking

▸ Metadata standards 4 / 4

Reusable research starts with good, machine-actionable metadata. This sub-cluster points to field-tested schemas and “minimum information” checklists so teams can capture provenance, methods, and context consistently across datasets, code, and teaching materials.

Anon. (n.d.). Fair cookbook. FAIR Cookbook. https://faircookbook.elixir-europe.org/content/home.html

Anon. (n.d.). Project metadata. LDbase. https://www.ldbase.org/resources/user-guide/information-to-gather/project-metadata

practice/tools Paper

The FAIR Cookbook - the essential resource for and by FAIR doers

This resource provides a collection of actionable, community-curated "recipes" that translate abstract FAIR principles into concrete technical workflows for data management. It helps researchers and data stewards bridge the gap between high-level policy requirements and the actual technical implementation needed to make research data findable, accessible, interoperable, and reusable.

overview Paper

The FAIR Guiding Principles for scientific data management and stewardship

This seminal paper establishes the FAIR Data Principles as a formal framework for improving the machine-actionability of scholarly data. It provides the foundational criteria meant to guide diverse stakeholders in enhancing the long-term stewardship and reuse of digital research assets across all scientific disciplines.

▸ Repositories 9 / 9

Trusted places to deposit datasets, code, and teaching materials so they remain findable, citable, and preserved.

Anon. (2019). Home. Open Science Framework. https://osf.io/

Anon. (n.d.). Figshare - credit for all your research. Figshare.com. https://figshare.com

Anon. (n.d.). LDbase: A Learning & Development Data Repository. www.ldbase.org. https://www.ldbase.org/

Anon. (n.d.). Zenodo - Research. Shared. Zenodo.org. https://zenodo.org

practice/tools Paper

Practical Solutions for Sharing Data and Materials From Psychological Research

This article offers practical workflows and solutions tailored to the specific challenges of sharing complex research materials in psychology, such as sensitive video and audio data. It highlights tools like Databrary.org to demonstrate how researchers can navigate privacy concerns and technical barriers to make their research products more accessible.

Github. (2013). Build software better, together. GitHub. https://github.com

Qualitative Data Repository (n.d.). Guidance and resources. Qualitative Data Repository. https://qdr.syr.edu/guidance

practice/tools Paper

The what, why, and how of born-open data

practice/tools Paper

Using OSF to Share Data: A Step-by-Step Guide

This tutorial provides a practical, step-by-step workflow for utilizing the Open Science Framework (OSF) to share research materials, data, and scripts. It specifically addresses the needs of researchers in psychological science by detailing how to establish stable storage and generate unique identifiers for their digital objects.

▸ Research data management 9 / 9

Introduces the planning and processes for managing research data through its lifecycle, from organizing files and documenting data (so that you and others can understand it later) to storing it securely and preparing it for sharing or archiving. Good RDM underpins the ability to be FAIR.

practice/tools Paper

Introducing the Qualitative Data Repository's Curation Handbook

This practice-oriented paper presents the Qualitative Data Repository’s Curation Handbook, which outlines formalized procedures for structuring and archiving qualitative and multi-method research. The handbook serves as a guide for both researchers and data curators to ensure that shared data is discoverable, accessible, and meaningful for secondary analysis.

practice/tools Paper

Valid replications require valid methods: Recommendations for best methodological practices with lab experiments.

critique Paper

Misidentified biomedical resources: Journal guidelines are not a quick fix

This resource critiques the effectiveness of current journal policies in mitigating the use of misidentified biological materials, such as contaminated cell lines or non-specific antibodies. It argues that simple policy mandates are insufficient to address these deep-seated biomedical research quality issues without more comprehensive institutional interventions.

advocacy Paper

How Data Curation Enables Epistemically Responsible Reuse of Qualitative Data

This paper presents a conceptual argument for the 'epistemically responsible reuse' of qualitative data, countering common skepticism regarding the ethical and epistemological validity of secondary analysis. It highlights the vital role of meticulous data curation in preserving the necessary context and integrity for meaningful reuse in qualitative inquiry.

practice/tools Paper

Qualitative Data Sharing: Data Repositories and Academic Libraries as Key Partners in Addressing Challenges

This resource outlines the collaborative roles that data repositories and academic libraries can play in overcoming practical hurdles to qualitative data sharing. It specifically addresses how these institutions can partner with researchers to manage informed consent, de-identification, and data access challenges.

practice/tools Paper

Ten Simple Rules for Creating a Good Data Management Plan

This resource provides a practical framework for developing comprehensive Data Management Plans required by research funders. It breaks down the complex data life cycle into ten actionable steps to ensure research data is properly discovered, collected, and preserved.

practice/tools Paper

The what, why, and how of born-open data

UK Data Service. (n.d.). Research data management., https://ukdataservice.ac.uk/learning-hub/research-data-management/

practice/tools Website

De-Identification When Making Data Sets Findable, Accessible, Interoperable and Reusable

This resource offers technical and ethical guidance for the de-identification of research datasets to enable sharing in accordance with FAIR principles. It identifies common pitfalls in the anonymization process and provides strategies to balance data utility with participant privacy protection.

▸ FAIR principles applied to Education & Training 4 / 4

FAIR isn’t only for datasets, syllabi, slides, and assignments can be findable, accessible, interoperable, and reusable too. This section offers institutional and practical roadmaps to make FAIR-by-design teaching materials the default.

teaching/training Report

D2.2 Methodology for FAIR-by-Design Training Materials

This resource outlines a six-stage methodology for creating educational materials that are FAIR by design, using a backward instructional design process. It provides practical checklists and guidance on metadata and granularity to help trainers produce learning resources that are discoverable and reusable across different contexts.

practice/tools Paper

Ten quick tips for navigating intellectual property in FAIR educational resources

This guide provides ten practical tips for educators to navigate the intersection of intellectual property law and FAIR principles when creating Open Educational Resources. It offers actionable advice on copyright and licensing to ensure that materials are both legally compliant and easily shareable.

Kohrs, F. E., Auer, S., Bannach-Brown, A., Fiedler, S., Haven, T. L., Heise, V., Holman, C., Azevedo, F., Bernard, R., Bleier, A., Bössel, N., Cahill, B. P., Castro, L. J., Ehrenhofer, A., Eichel, K., Frank, M., Frick, C., Friese, M., Gärtner, A., … Weissgerber, T. L. (2023). Eleven strategies for making reproducible research and open science training the norm at research institutions. ELife, 12. CLOCKSS. https://doi.org/10.7554/eLife.89736

practice/tools Paper

Towards FAIRification of learning resources and catalogues—lessons learnt from research communities

This publication synthesizes community-led efforts to apply FAIR principles to educational materials, offering practical rules for enhancing the findability and reusability of learning resources. It highlights lessons learned across different research communities to guide the development of interoperable training catalogues and shared educational infrastructures.

7 Publication Sharing

7 sub-clusters · 51 references

This cluster has 6 sub-clusters:

▸ Why open access? 4 / 4

Open access accelerates knowledge circulation, enables global access and increases visibility and reuse.

advocacy Paper

The future of academic publishing

This collaborative piece explores the evolving landscape of scholarly communication, advocating for structural changes to improve the equity and accessibility of research dissemination. It provides a forward-looking perspective on how academic publishing can better serve global societal needs by moving toward more open and inclusive models.

overview Book

Open Access

This resource offers a definitive primer on open access, outlining its core principles and dismantling common myths regarding its cost, legality, and impact. It serves as a foundational text for understanding how digital infrastructure and copyright-holder consent can be leveraged to share scholarly knowledge freely and without restriction.

evidence Preprint

The academic, economic and societal impacts of Open Access: an evidence-based review

Willinsky, J. (2006). The access principle : the case for open access to research and scholarship. MIT Press. https://mitpress.mit.edu/9780262512664/the-access-principle

▸ Alternatives to legacy journals 8 / 8

Introduces new publishing models beyond the traditional subscription journals. Examples include publishing platforms like Octopus (which breaks papers into smaller units), modular publishing on ResearchEquals, community peer-review outlets like Peer Community In, and overlay journals that curate arXiv preprints. These alternatives aim to make publishing faster, fairer, and more transparent by decoupling the functions of journals.

Anon. (n.d.). Free, fast and fair: The Global Primary Research Record where researchers publish their work in full detail. Octopus. https://www.octopus.ac/

Anon. (n.d.). Researchequals.com. ResearchEquals.com. https://www.researchequals.com/

Anon. (2023). Free Peer Review & Validation of preprints of articles. Peer Community In. https://peercommunityin.org/

overview Preprint

Understanding the Publish-Review-Curate (PRC) Model of Scholarly Communication

This resource introduces the Publish-Review-Curate (PRC) model of scholarly communication, explaining how it decouples the dissemination of research from traditional peer review and curation. It provides stakeholders with an overview of the model's mechanics and current data on its adoption across the research ecosystem.

Currie, G. (2024). Open science: What is publish, review, curate? Inside eLife. https://elifesciences.org/inside-elife/dc24a9cd/open-science-what-is-publish-review-curate

MetaROR. (2024, November 21). Publish–Review–Curate. MetaROR. https://cms.metaror.org/publish-review-curate

overview Paper

Biomedical publishing: Past historic, present continuous, future conditional

This resource provides a historical perspective on how academic publishing evolved into its current state and evaluates the limitations of traditional peer review and impact factors. It explores a future model where publishing is decoupled from evaluation through the increased use of preprints and post-publication peer review.

Wikimedia Foundation. (2023, September 1). Overlay Journal. Wikipedia. https://en.wikipedia.org/wiki/Overlay_journal

▸ Different shades of open access 7 / 7

We usually hear about GOLD open access (journal archives openly) for hefty fees. There are other options that may better fit your needs (and budget / reluctance to fund for-profit publishers); GREEN (self-archive), DIAMOND (journal archives openly for free), and more.

Anon. An introduction to open access. Jisc. (n.d.). https://www.jisc.ac.uk/guides/an-introduction-to-open-access

Anon. (n.d.). OA books toolkit. Green, gold, diamond – different models for open access books | OA Books Toolkit. https://oabooks-toolkit.org/lifecycle/article/13868103-green-gold-diamond-different-models-for-open-access-books

evidence Paper

The oligopoly’s shift to open access: How the big five academic publishers profit from article processing charges

This study provides a data-driven estimation of the massive financial revenue generated by the five largest commercial publishers through Article Processing Charges (APCs) between 2015 and 2018. It highlights how major publishers have successfully shifted their profit models to capture the growing open access market.

evidence Paper

Data availability, reusability, and analytic reproducibility: evaluating the impact of a mandatory open data policy at the journal Cognition

This empirical study evaluates the effectiveness of a mandatory open data policy at a specific journal by measuring changes in data availability and the success rate of independent analytic reproduction. It demonstrates that while such policies significantly increase data sharing, technical and procedural barriers to actually reproducing results remain a major challenge.

practice/tools Paper

The what, why, and how of born-open data

evidence Paper

Authorial and institutional stratification in open access publishing: the case of global health research

evidence Paper

The state of OA: a large-scale analysis of the prevalence and impact of Open Access articles

This study provides large-scale empirical evidence on the prevalence and citation impact of open access articles across the scholarly landscape. It quantifies the growth of different types of open access and provides a data-driven assessment of the citation advantage associated with making research freely available.

▸ Open peer review 26 / 26

Open peer review typically refers to sharing reviewer reports and/or reviewers signing reviews. It is important to understand which definition is being used to understand the pros and cons.

Anon. (n.d.). About Meta-Psychology. Meta-Psychology. https://open.lnu.se/index.php/metapsychology/about

Carlsson, R., Lakens, D., van Assen, M.A.L.M., Heene, M., Innes-Ker, A., Schönbrodt, F., Danielsson, H., DeBruine, L., Buchanan, E.M., Kalmendal, A., Holcombe A.O., & Batinovic, L. (2023, March 14). Meta-psychology. OSF. https://osf.io/3m4z3/

Ford, E. (2013). Defining and Characterizing Open Peer Review: A Review of the Literature. Journal of Scholarly Publishing 44(4), 311-326. https://muse.jhu.edu/article/513250.

overview Paper

What are innovations in peer review and editorial assessment for?

This resource provides a conceptual analysis of the diverse goals and rationales driving recent innovations in the peer review process. It helps readers understand the specific problems that different editorial models, such as open or transparent review, are intended to solve.

policies Book Chapter

Designing journal peer review: diverse expectations, procedures and concerns

This paper presents a foundational, consensus-based definition of predatory publishing reached by an international panel of experts to provide a clear standard for the academic community. It serves as a definitive reference for stakeholders to distinguish legitimate journals from deceptive ones by establishing clear criteria for predatory behavior.

evidence Review Article

The changing forms and expectations of peer review

This resource provides an empirical quantification of the scale of scientific literature contaminated by the use of misidentified cell lines, identifying tens of thousands of affected papers. It highlights the persistence of 'ghost' data in the research record and the systemic failure of scholarly publishing to correct known errors over time.

overview Journal Article

The ability of different peer review procedures to flag problematic publications

This resource traces the historical evolution of peer review, examining how it transitioned from a mechanism for quality assessment to a modern gatekeeper of scientific integrity. It contextualizes the current debate over scientific self-regulation by highlighting how the expectation for peer review to detect fraud is a relatively recent development.

evidence Editorial

Innovating editorial practices: academic publishers at work

This research analyzes the internal decision-making processes and structural barriers within commercial publishing houses that affect the adoption of open science editorial practices. It highlights the specific practical considerations and organizational hurdles that publishers weigh when deciding whether to implement innovative peer review technologies.

evidence Editorial

Journal Peer Review and Editorial Evaluation: Cautious Innovator or Sleepy Giant?

This study provides empirical evidence on the effectiveness of various peer review models by correlating specific procedures with retraction rates in the Retraction Watch database. It offers a data-driven comparison of how different review innovations perform in their primary task of flagging problematic or fraudulent research.

evidence Journal Article

Changing peer review practices: transforming roles and future challenges

This research investigates the adoption and implementation rates of innovative peer review procedures across a wide range of scientific journals. It identifies which innovations are gaining traction among editors and which remain theoretical, highlighting the current state of quality management in academic publishing.

overview Letter

Hundreds of journals’ editorial practices captured in database

This publication provides a broad survey of the shifting landscape of peer review, focusing on how the roles of stakeholders are being transformed. It identifies emerging challenges and outlines future directions for the evolution of review practices within the scholarly ecosystem.

practice/tools Editorial

The Platform for Responsible Editorial Policies: An initiative to foster editorial transparency in scholarly publishing

This resource announces a comprehensive database documenting the editorial practices of hundreds of scholarly journals. It provides a searchable repository that allows researchers to compare transparency levels and procedural standards across a vast array of publications.

Horbach, S. P. J. M. (2021). How the pandemic changed editorial peer review – and why we should wonder whether that’s desirable. Impact of Social Sciences (LSE). https://blogs.lse.ac.uk/impactofsocialsciences/2021/02/10/how-the-pandemic-changed-editorial-peer-review-and-why-we-should-wonder-whether-thats-desirable/

Horbach, S. P. J. M., Ochsner, M., & Kaltenbrunner, W. (2022). Reflections on guest editing a Frontiers journal [Blog post]. Leiden Madtrics. https://www.leidenmadtrics.nl/articles/reflections-on-guest-editing-a-frontiers-journal?

Horbach, S. P. J. M., Ross-Hellauer, T., & Waltman, L. (2022). Sunlight not shadows: Double-anonymized peer review is not the progressive answer to status bias. OSF https://osf.io/preprints/osf/fqb5c_v1

evidence Paper

Published peer review reports have higher informative content than unpublished reports

This empirical study analyzes over 250,000 peer review reports from medical journals to compare the quality and informativeness of published versus unpublished reports. It provides evidence that open peer review is associated with more informative content and identifies specific demographic differences in reviewer behavior.

evidence Paper

Should psychologists sign their reviews? Some thoughts and some data.

This article evaluates the costs and benefits of signed peer reviews in psychology by combining a theoretical discussion of accountability and retaliation with empirical data. It specifically addresses how transparency in the review process aligns with the broader open science movement while acknowledging the practical risks to junior researchers.

Mahoney, M. J. (1977). Publication prejudices: An experimental study of confirmatory bias in the peer review system. Cognitive Therapy and Research, 1(2), 161–175. https://doi.org/10.1007/BF01173636

advocacy Paper

The Peer Reviewers' Openness Initiative: incentivizing open research practices through peer review

This paper introduces a collective initiative designed to incentivize open research by encouraging reviewers to make data and material sharing a prerequisite for comprehensive review. It offers a strategic framework to resolve the social dilemma of open science by shifting the responsibility for transparency from authors to the peer review process itself.

evidence Paper

Single-blind vs Double-blind Peer Review in the Setting of Author Prestige

This empirical study investigates the impact of author prestige on the peer review process by comparing the outcomes of single-blind and double-blind review models. It provides evidence on whether masking author identities can effectively reduce status-based bias and improve the fairness of manuscript evaluations in medical publishing.

evidence Journal Article

Open peer review urgently requires evidence: A call to action

This scoping review synthesizes recent empirical research on various components of Open Peer Review to assess its effectiveness and how it meets community expectations. It identifies specific gaps in the current literature, highlighting where further experimentation is required to justify broader adoption of open review practices.

advocacy Review Article

Additional experiments required: A scoping review of recent evidence on key aspects of Open Peer Review

This resource argues that despite the increasing number of retractions, the current rate is still insufficient to address the volume of flawed or fraudulent research in the literature. It makes the case for more proactive correction of the scientific record and greater transparency from publishers regarding the reasons for retractions.

advocacy Review Article

What is open peer review? A systematic review

This paper issues a call to action for the scholarly community to generate more rigorous empirical evidence regarding the impact and implementation of Open Peer Review. It provides a preliminary research agenda designed to guide future studies so that the move toward open review can be based on evidence rather than intuition.

practice/tools Preprint

Ten considerations for open peer review

This resource provides actionable guidance for authors, reviewers, and editors by outlining ten key considerations for navigating the transition to open peer review. It serves as a pragmatic, hands-on introduction that helps stakeholders identify potential pitfalls and opportunities when implementing more transparent review processes.

Tvina, A., Spellecy, R., & Palatnik, A. (2019). Bias in the Peer Review Process. Obstetrics & Gynecology, 133(6), 1081–1083. https://doi.org/10.1097/AOG.0000000000003260

evidence Paper

Effect of open peer review on quality of reviews and on reviewers'recommendations: a randomised trial

This publication presents empirical findings from a randomized trial investigating how revealing reviewer identities to authors impacts the quality and recommendations of peer reviews. It provides specific evidence that while identity disclosure does not significantly alter the quality of the report, it does impact the willingness of individuals to participate in the review process.

▸ Preprints and postprints 3 / 3

To circumvent paywalls and inaccessible scientific work, pre- and postprints can be published on open repositories in order to make the work accessible to all.

Anon. (n.d.). What are the differences between preprint and postprint versions?. ZB MED - Informationszentrum Lebenswissenschaften. https://www.publisso.de/en/advice/publishing-advice-faqs/preprint-and-postprint

overview Paper

Accelerating scholarly communication: The transformative role of preprints

This report examines the rapidly changing landscape of preprints, detailing the drivers, impediments, and practices that influence their adoption across scholarly disciplines. It provides a comprehensive analysis of how sharing research in pre-review form is transforming scholarly communication and identifies key trends in article dissemination.

advocacy Paper

Preprints: An underutilized mechanism to accelerate outbreak science

This essay advocates for the use of preprints specifically within the field of outbreak science to facilitate the rapid sharing of life-saving data. It emphasizes how preprints can bypass traditional publication bottlenecks to accelerate scientific responses during public health emergencies.

▸ Rights retention strategies 1 / 1

To whom does the paper belong?

Anon. (n.d.). Plan S rights retention strategy. Plan S. https://www.coalition-s.org/rights-retention-strategy

▸ SPARC Europe - Rights Retention Helper: 2 / 2

https://sparceurope.org/what-we-do/open-access/copyright/project-retain/rights-retention-helper/

Suber, Peter. Methods of Rights Retention: Methods of rights retention

8 Replication and meta-research

7 sub-clusters · 82 references

Attainment of a grounding in 'replication research', which takes a variety of forms, each with a different purpose and contribution. Replicable science requires replication research. When teaching, students should understand the purpose and need of replications in its variety of forms and be able to conduct (and join) replication projects. There are 6 sub-clusters which aim to further parse the learning and teaching process:[ag][ah]

▸ Conducting replication studies; challenges, limitations, and comparisons with the original study 21 / 21

A replication study seeks to repeat findings of previous research using identical or similar methods to determine if consistent results can be obtained. Limits can arise from protocol drift, differences in context or measurement, and low power.

evidence Paper

Estimating the reproducibility of psychological science

overview Paper

Toward a more credible assessment of the credibility of science by many-analyst studies

This paper introduces the "many-analyst" research design as a method for assessing the robustness of scientific findings. It argues that observing the variation in results when different researchers analyze the same dataset provides a more rigorous measure of uncertainty and credibility than individual replication attempts.

evidence Paper

The (Non)Academic Community Forming around Replications: Mapping the International Open Science space via its Replication Initiatives

practice/tools Paper

How can we make sound replication decisions?

Devezer, B., & Buzbas, E. O. (2025). Minimum viable experiment to replicate. PhilSci Archive. https://philsci-archive.pitt.edu/24738/

Errington, T. M. (2024). Building reproducible bridges to cross the “valley of death.” Journal of Clinical Investigation, 134(1). https://doi.org/10.1172/JCI177383

teaching/training Paper

Teaching Replication

This resource advocates for a pedagogical model where students in laboratory courses perform direct replications of published findings as a core part of their training. It outlines how this approach addresses the 'replication crisis' by generating needed data while providing students with authentic, high-stakes experience in scientific methodology.

critique Letter

Comment on “Estimating the reproducibility of psychological science”

This paper provides a statistical critique of the landmark 2015 Open Science Collaboration study, arguing that its conclusions regarding the low reproducibility of psychology are based on flawed analysis. It offers a re-evaluation of the original data to suggest that the reproducibility of psychological science may actually be quite high.

Grahe, J. E., Brandt, M. J., Wagge, J. R., Legate, N., Wiggins, B. J., Christopherson, C. D., Weisberg, Y., Corker, K.S., Chartier, C.R., Fallon, M., Hildebrandt, L., Hurst, M.A., Lazarevic, L., Levitan, C., McFall, J., McLaughlin, H., Pazda, A., Ijzerman, H., Nosek, B.A., … & France, H. (2018). Collaborative Replications and Education Project (CREP). https://osf.io/wfc6u/

teaching/training Paper

Harnessing the Undiscovered Resource of Student Research Projects

This resource introduces the "question-list paradigm" as a framework for leveraging the high volume of undergraduate research projects to advance the science of psychology. It explains how these student projects can be coordinated to test the replicability of established findings across diverse geographic locations and institutional populations.

practice/tools Paper

Valid replications require valid methods: Recommendations for best methodological practices with lab experiments.

Horbach, S. P. J. M., Cole, N. L., Kopeinik, S., Leitner, B., Ross-Hellauer, T., & Tijdink, J. (2025). How to get there from here? Barriers and enablers on the road towards reproducibility in research [Manuscript]. OSF. https://osf.io/n28sg/

practice/tools Preprint

Pre-replication: anything goes, once

This commentary introduces the concept of "pre-replication," a structured preparatory exercise to be performed before conducting a replication study. It provides a framework for analyzing the epistemic and analytic goals of the replication to ensure the work is conceptually sound and its outcomes are interpretable.

King, G. (1995). Replication, replication. PS: Political Science & Politics, 28(3), 444-452. https://gking.harvard.edu/files/replication.pdf

overview Paper

Growth From Uncertainty: Understanding the Replication ‘Crisis’ in Infant Cognition

This article applies the philosophical concept of "epistemic iteration" to the field of infant cognition to reframe the replication crisis as a productive part of scientific growth. It argues that failed replications should be viewed as opportunities for theoretical refinement rather than evidence of untrustworthy research.

Lenne & Mann (2016). CREP project report. https://osf.io/sdj7e/

critique Preprint

Adversarial reanalysis and the challenge of open data in regulatory science

critique Editorial

Editorial Essay: The Tumult over Transparency: Decoupling Transparency from Replication in Establishing Trustworthy Qualitative Research

overview Paper

TIER2: enhancing Trust, Integrity and Efficiency in Research through next-level Reproducibility

evidence Paper

Expectations for Replications

This resource uses computer simulations to demonstrate how random measurement error alone makes high replication success rates statistically unlikely, even under ideal conditions. It argues that many perceived failures in psychological science stem from mathematically unreasonable expectations regarding the consistency of experimental results.

teaching/training Paper

Publishing Research With Undergraduate Students via Replication Work: The Collaborative Replications and Education Project

▸ Direct vs. conceptual replications 7 / 7

Direct replications use the exact same methods and materials, while conceptual replications test the same concept but with different methods, materials, or both. There is an ongoing debate as to how “direct” a replication can possibly be.

critique Paper

Kinds of Replication: Examining the Meanings of “Conceptual Replication” and “Direct Replication”

This article critiques the theoretical foundations of the replication crisis by examining how "direct" and "conceptual" replications are defined and understood. It proposes that the discipline should move beyond a discovery-oriented philosophy of science to consider the way research practices actively shape psychological phenomena.

overview Paper

Reconceptualizing replication as a sequence of different studies: A replication typology

This resource synthesizes existing literature on replication to propose a comprehensive typology that views replication as a sequential process rather than a binary outcome. It provides a structured framework for researchers to transition from simple direct replications to studies that systematically test alternative explanations and real-world applicability.

critique Paper

Approaching psychology’s current crises by exploring the vagueness of psychological concepts: Recommendations for advancing the discipline.

evidence Paper

Internal conceptual replications do not increase independent replication success

This meta-research study evaluates whether the inclusion of internal conceptual replications in an original paper increases the likelihood of successful independent direct replications. The findings suggest that internal replications do not predict future reproducibility, calling into question their role as indicators of robust psychological effects.

advocacy Paper

The Value of Direct Replication

This commentary advocates for direct replication as the primary mechanism for verifying the reliability of scientific effects across laboratories. It critiques the assumptions of those who devalue replication, arguing that direct repetition is the only rigorous way to confirm the validity of a research finding.

evidence Paper

Contextual sensitivity in scientific reproducibility

Zwaan, R. A., Etz, A., Lucas, R. E., & Donnellan, M. B. (2017). Making replication mainstream. Behavioral and Brain Sciences, 41. https://doi.org/10.1017/S0140525X17001972

▸ Meta-analyses 16 / 16

Meta-analysis pools estimates to show the bigger picture. Careful work starts with a prespecified plan, aligns effect sizes, and checks bias and sensitivity. It reports heterogeneity and uses prediction intervals to show what a future study might find. Data and code are shared so the synthesis can be audited and updated.

evidence Paper

Meta-analyses in psychology often overestimate evidence for and size of effects

This study provides empirical meta-research demonstrating that meta-analyses in psychology frequently overestimate effect sizes and evidence strength due to inadequate publication bias adjustments. It evaluates the effectiveness of various adjustment methods and illustrates the systematic distortion present in current psychological literature.

practice/tools Paper

Robust Bayesian meta‐analysis: Model‐averaging across complementary publication bias adjustment methods

This paper introduces Robust Bayesian Meta-Analysis (RoBMA), a methodological framework that uses model-averaging to account for publication bias. It provides a practical solution for researchers to synthesize data more reliably without having to choose a single adjustment method that may not fit their specific data conditions.

evidence Paper

Footprint of publication selection bias on meta‐analyses in medicine, environmental sciences, psychology, and economics

This large-scale meta-epidemiological study assesses the prevalence of publication selection bias across medicine, environmental sciences, psychology, and economics by analyzing over 700,000 effect size estimates. It provides a comparative analysis that reveals how bias varies by discipline, identifying economics and environmental sciences as having particularly high levels of contamination.

evidence Paper

Correcting for Bias in Psychology: A Comparison of Meta-Analytic Methods

This resource uses a comprehensive simulation study to evaluate how different meta-analytic methods designed to correct for bias perform on data patterns specifically common to psychology. It identifies which statistical techniques are most effective at recovering true effect sizes when research is affected by questionable research practices or publication bias.

evidence Paper

Meta-assessment of bias in science

This meta-assessment offers a global view of bias prevalence by analyzing a large random sample of meta-analyses drawn from all scientific disciplines. It identifies specific risk factors for effect size overestimation—such as small sample size and early publication—while demonstrating that although certain bias patterns are consistent, their magnitude varies significantly across fields.

evidence Paper

“Positive” Results Increase Down the Hierarchy of the Sciences

This study empirically tests the "Hierarchy of the Sciences" hypothesis by analyzing the proportion of positive results reported in over 2,400 papers across all disciplines. It demonstrates that the frequency of results supporting a tested hypothesis increases as one moves from the physical to the social sciences, suggesting that the "hardness" of a field influences its susceptibility to non-cognitive biases.

Hong, S., & Reed, W. R. (2020). Using Monte Carlo experiments to select meta‐analytic estimators. Research Synthesis Methods, 12(2), 192–215. Portico. https://doi.org/https://doi.org/10.1002/jrsm.1467

evidence Paper

The Power of Bias in Economics Research

This research evaluates the credibility of empirical economics by quantifying statistical power and publication bias across 159 different research literatures. It reveals that the majority of economics studies are severely underpowered and that reported effects are frequently exaggerated, providing a stark assessment of the structural challenges facing the reliability of economic findings.

practice/tools Paper

Advice for improving the reproducibility of data extraction in meta‐analysis

This resource provides practical steps and R-based software recommendations to improve the transparency and reproducibility of data extraction in meta-analyses. It specifically addresses the lack of guidance for making this crucial phase of evidence synthesis shareable and verifiable by other researchers.

evidence Paper

Comparing meta-analyses and preregistered multiple-laboratory replication projects

This study provides empirical evidence comparing the effect sizes found in traditional meta-analyses versus those from large-scale, preregistered multi-laboratory replication projects. It highlights a significant discrepancy where traditional meta-analyses tend to report much larger effect sizes, suggesting they may be more susceptible to systematic biases.

critique Paper

The case of the misleading funnel plot

This resource critiques the reliance on funnel plots for detecting publication bias, explaining how factors like study heterogeneity and chance can produce misleading asymmetry. It warns researchers that visual evidence in these plots is often ambiguous and should not be used as a definitive diagnostic tool for bias.

evidence Paper

Footprint of publication selection bias on meta‐analyses in medicine, environmental sciences, psychology, and economics

evidence Paper

Assessing treatment effects and publication bias across different specialties in medicine: a meta-epidemiological study

This study examines the impact of treatment effect inflation and publication bias across various medical specialties. It demonstrates that large effects reported in small, low-powered studies contribute significantly to evidence distortion and argues for shifting institutional incentives toward research quality rather than the extremity of results.

evidence Paper

What meta-analyses reveal about the replicability of psychological research.

This study provides a large-scale empirical assessment of replicability in psychology by analyzing over 12,000 effect sizes from 200 meta-analyses. It quantifies the prevalence of low statistical power and evaluates how bias and heterogeneity contribute to the field's replication challenges.

Topor, M., Pickering, J. S., Barbosa Mendes, A., Bishop, D. V. M., Büttner, F., Elsherif, M. M., Evans, T. R., Henderson, E. L., Kalandadze, T., Nitschke, F. T., Staaks, J. P. C., Van den Akker, O. R., Yeung, S. K., Zaneva, M., Lam, A., Madan, C. R., Moreau, D., O’Mahony, A., Parker, A. J., … Westwood, S. J. (2023). An integrative framework for planning and conducting Non-Intervention, Reproducible, and Open Systematic Reviews (NIRO-SR). Meta-Psychology, 7. https://doi.org/10.15626/MP.2021.2840

evidence Paper

Publication bias examined in meta-analyses from psychology and medicine: A meta-meta-analysis

This meta-meta-analysis empirically investigates the extent of publication bias by comparing hundreds of systematic reviews across the fields of psychology and medicine. The research highlights how this bias leads to overestimated effects, providing a comparative perspective on how different disciplines struggle with the credibility of published findings.

▸ Meta-research 23 / 23

Meta-research studies how research is done. It maps power, bias, reporting quality, and the uptake of open practices. It tests which interventions improve credibility and efficiency. The aim is practical guidance that helps fields do better work and waste less effort. Findings are shared openly so policies, training, and incentives can respond.

critique Letter

Claims about scientific rigour require rigour

This resource critiques a high-profile study on rigour-enhancing practices, arguing that the study itself failed to adhere to the preregistration and transparency standards it advocated. It serves as a methodological cautionary tale, emphasizing that meta-research about scientific rigour must be held to the same rigorous standards it promotes.

critique Preprint

Revisiting the replication crisis without false positives

evidence Paper

The (Non)Academic Community Forming around Replications: Mapping the International Open Science space via its Replication Initiatives

evidence Paper

Comparing dream to reality: an assessment of adherence of the first generation of preregistered studies

This empirical study assesses the adherence of psychology researchers to their preregistration plans and the transparency of their reporting regarding deviations. It identifies a significant gap between initial research designs and final publications, emphasizing the importance of disclosing changes made during the data collection and analysis process.

practice/tools Paper

How can we make sound replication decisions?

Devezer, B., & Buzbas, E. O. (2025). Minimum viable experiment to replicate. PhilSci Archive. https://philsci-archive.pitt.edu/24738/

overview Paper

Open science interventions to improve reproducibility and replicability of research: a scoping review

critique Paper

Can a Good Theory Be Built Using Bad Ingredients?

This paper examines how the replication crisis impacts theory development, arguing that the consequences of replication failures depend on whether a theory aims to explain, predict, or unify. It provides a nuanced theoretical analysis of why replicability is more foundational for explanatory theories than for those focused primarily on predictive outcomes.

overview Paper

Promoting Virtue or Punishing Fraud: Mapping Contrasts in the Language of ‘Scientific Integrity’

This paper maps the diverse and often conflicting meanings of "research integrity" across different stakeholders, from narrow definitions focused on misconduct to broader ethical frameworks. It highlights the subtle linguistic and conceptual differences in how integrity is understood by researchers, policymakers, and the public.

overview Journal Article

The ghosts of HeLa: How cell line misidentification contaminates the scientific literature

This resource explores the intersection of Open Science practices and generative AI, identifying how these technologies both facilitate and complicate goals of transparency and accessibility. It specifically analyzes the tensions between generative AI's black-box nature and foundational open principles, offering an exploratory framework for navigating these emerging challenges.

evidence Review Article

The changing forms and expectations of peer review

overview Journal Article

The ability of different peer review procedures to flag problematic publications

evidence Paper

The extent and causes of academic text recycling or ‘self-plagiarism’

evidence Editorial

Journal Peer Review and Editorial Evaluation: Cautious Innovator or Sleepy Giant?

evidence Journal Article

No time for that now! Qualitative changes in manuscript peer review during the Covid-19 pandemic

This study evaluates the impact of the COVID-19 pandemic on the quality of scholarly peer review by analyzing changes in review reports and editorial decision letters. It specifically investigates whether the rapid acceleration of the publication process during the pandemic led to a decrease in the depth and rigor of critical evaluation.

critique Paper

Why Most Published Research Findings Are False

This landmark theoretical paper uses a mathematical framework to argue that the majority of published research findings are likely false due to factors like low statistical power, small effect sizes, and researcher bias. It serves as a fundamental critique of the prevailing incentive structures and methodological standards in modern science.

evidence Paper

With Low Power Comes Low Credibility? Toward a Principled Critique of Results From Underpowered Tests

evidence Paper

Evaluating the Pedagogical Effectiveness of Study Preregistration in the Undergraduate Dissertation

This study uses a quasi-experimental design to empirically evaluate the effectiveness of preregistration as a pedagogical tool for undergraduate psychology students. It demonstrates how early exposure to open science practices can influence student learning outcomes and help mitigate questionable research practices during initial research training.

overview Paper

What is critical metascience and why is it important?

evidence Paper

Registered report: Survey on attitudes and experiences regarding preregistration in psychological research

This study provides empirical evidence regarding the attitudes and practical experiences of psychology researchers concerning preregistration. It identifies specific systemic and individual obstacles that hinder the adoption of the practice, offering insights into how to better foster open science workflows.

Syed, M. (2023). Some Data Indicating that Editors and Reviewers Do Not Check Preregistrations during the Review Process. https://osf.io/nh7qw/

critique Preprint

The social replication of replication: Moving replication through epistemic communities

This resource analyzes the "replication drive" and how the practice of replication is being moved across various epistemic communities. It warns against the indiscriminate promotion of replication as a universal standard for quality, highlighting the social and institutional pressures that shape this culture change.

evidence Paper

Preregistration in practice: A comparison of preregistered and non-preregistered studies in psychology

This research evaluates the practical impact of preregistration by comparing matched sets of preregistered and non-preregistered studies in psychology. It contributes critical data by showing that preregistration did not necessarily lead to fewer positive results or smaller effect sizes in this sample, challenging theoretical expectations about its immediate effects on the literature.

▸ Purposes of replication attempts - what is a ‘failed’ replication? 7 / 7

Explains the diverse aims of replication and clarifies that “failure” is not a verdict on truth but evidence about effect size, robustness, and conditions. Encourages nuanced interpretation (e.g., meta-analytic and design-aware) over binary success/failure narratives and highlights responsible communication of discrepant findings.

Fidler, F., & Wilcox, J. (2021). Reproducibility of scientific results. In E. N. Zalta (Ed.), The Stanford encyclopedia of philosophy. https://plato.stanford.edu/archives/sum2021/entries/scientific-reproducibility

teaching/training Paper

Teaching Replication

overview Paper

Replication and the Manufacture of Scientific Inferences: A Formal Approach

This resource introduces a formal theoretical framework for replication using "replication causal diagrams" (r-dags) and Bayesian inference to categorize replication types and evaluate their evidentiary value. It moves beyond simple typologies by providing a mathematical basis for updating beliefs about natural phenomena based on specific study procedures and outcomes.

overview Paper

What is replication?

This paper proposes a conceptual shift in the definition of replication, moving away from procedural similarity toward a focus on the diagnostic evidence provided for a prior claim. It argues that a study's status as a replication is determined by whether its possible outcomes are informative regarding the validity of the original finding.

critique Preprint

A Problem in Theory and More: Measuring the Moderating Role of Culture in Many Labs 2

This article critiques the methodology of the Many Labs 2 project, specifically challenging its conclusions regarding the role of cultural variability in replication success. It identifies significant theoretical flaws in the project's design, such as the selection of effects that lacked theoretical reasons to vary by culture and the use of sample sites with insufficient cultural contrast.

evidence Paper

Contextual sensitivity in scientific reproducibility

Zwaan, R. A., Etz, A., Lucas, R. E., & Donnellan, M. B. (2017). Making replication mainstream. Behavioral and Brain Sciences, 41. https://doi.org/10.1017/S0140525X17001972

▸ Registered Replication Reports 5 / 5

Registered Reports are studies that are peer-reviewed prior to data collection, with an agreement between the journal and the author(s) that it will be published regardless of outcome as long as the preregistered methods are reasonably followed. Registered REPLICATION Reports are a special category of these that only include replications.

evidence Paper

Registered Replication Report

Anon. (n.d.). Ongoing Replication Projects. Association for Psychological Science - APS. https://www.psychologicalscience.org/publications/replication/ongoing-projects

overview Paper

The past, present and future of Registered Reports

evidence Letter

Registered Replication Report

This resource reports the results of a multi-laboratory replication attempt focused on the psychological effect of grammatical aspect on perceptions of intentionality. It contributes empirical evidence to determine the robustness and generalizability of previously reported findings in the field of psycholinguistics and social cognition.

overview Paper

An Introduction to Registered Replication Reports at Perspectives on Psychological Science

This article introduces the Registered Replication Report (RRR) as a new article format within the journal Perspectives on Psychological Science, explaining its purpose in addressing the replication crisis. It details the editorial philosophy and procedural framework intended to encourage high-quality, multi-lab replications of foundationally important psychological findings.

▸ The politics of replicating famous studies 3 / 3

Sometimes responses to replication research can be negative. Failed replications of famous work, most notably power posing, ego depletion, stereotype threat, and facial feedback[ai][aj], have received a lot of attention.

evidence Paper

Behavioral Priming: It's All in the Mind, but Whose Mind?

This research presents two experiments that failed to replicate a seminal social priming study while specifically investigating the role of experimenter expectancy effects. It contributes to the meta-research discussion by suggesting that unconscious experimenter cues, rather than direct priming of participants, may account for previously observed behavioral effects.

Neuliep, J. W., & Crandall, R. (1990). Editorial bias against replication research. Journal of Social Behavior & Personality, 5(4), 85-90.

Neuliep, J. W., & Crandall, R. (1993). Reviewer bias against replication research. Journal of Social Behavior & Personality, 8(6), 21-29.

9 Academic Structures and Institutions

19 sub-clusters · 305 references

Attainment of a grounding in topics related to academia and academics. Students should understand how individuals, teams, institutions, and academic culture work together to promote (or hinder) openness, inclusion, diversity, equity, and transparency. Gathering perspectives on navigating scientific and academic life. Learning the challenges and rewards in the academic setting, the “hidden curriculum” in academic life. There are 17 sub-clusters which aim to further parse the learning and teaching process:

▸ Accessibility 8 / 8

Accessibility refers to making data, research environments, teaching and research outputs usable (e.g. Universal Design and hybrid participation options) by as many people as possible.

teaching/training Paper

Next‐generation field courses: Integrating Open Science and online learning

This resource explores how field courses can serve as a strategic pedagogical platform for training early career researchers in Open Science practices. It describes a specific approach to integrating these workflows within the practical constraints of environmental and biological fieldwork in both physical and online formats.

teaching/training Paper

Compassionate pedagogy for neurodiversity in higher education: A conceptual analysis

This resource provides a conceptual framework for applying "compassionate pedagogy" in higher education to support neurodivergent students. It promotes a shift from pathologizing models toward a perspective that recognizes cognitive variation as natural biodiversity, offering a basis for more inclusive teaching practices.

evidence Paper

Developing the inclusive curriculum: Is supplementary lecture recording an effective approach in supporting students with Specific Learning Difficulties (SpLDs)?

This study evaluates the impact of supplementary lecture recordings on the learning experiences of students with Specific Learning Difficulties (SpLDs). It provides empirical evidence regarding the effectiveness of lecture capture as an inclusive tool for reducing learning barriers in a traditional curriculum.

evidence Paper

Lights, camera, active! appreciation of active learning predicts positive attitudes towards lecture capture

This pre-registered study investigates how instructors' pedagogical beliefs about active learning influence their willingness to adopt and support lecture capture technology. The findings specifically address and challenge common concerns regarding the perceived negative impact of recordings on classroom attendance and student engagement.

Nordmann, E., Hutchison, J., MacKay, J.R.D. (2021). Lecture rapture: the place and case for lectures in the new normal. Teaching in Higher Education, 27(5), 709-716. https:doi.org/10.1080/13562517.2021.2015755

practice/tools Paper

Leveraging Campus Collaboration to Promote Sustainable, Inclusive Writing Support for All Students

This resource provides a model for sustainable campus collaboration to support writing and literacy development for students with intellectual disabilities. It details how cross-departmental partnerships can provide specialized interventions that would otherwise be prohibitively resource-intensive for individual academic programs.

practice/tools Paper

Guidelines to improve internationalization in the psychological sciences

This resource provides a set of actionable best practices and guidelines aimed at increasing international representation and diversity within the field of psychology. It specifically addresses how to bridge publication and data gaps across different world regions to move beyond the dominance of Western, industrialized contexts.

Steinfeld, E., & Maisel, J. (2012). Universal design: Creating inclusive environments. John Wiley & Sons. https://content.e-bookshelf.de/media/reading/L-592705-5ce358b130.pdf

▸ Citation Politics & Practices 17 / 17

Citation is fundamental for academic writing to acknowledge the ideas and work of authors influenced and informed our own writing. However, the politics of citations perpetuates power imbalance especially for race, gender and neurodivergence. Under-represented individuals (e.g. ethnic minority individuals, women, LGBTQIA2S+ and neurodivergent individuals) tend to be ignored in scholarship, while a small group of white, heterosexual, neurotypical men dominate the scholar discourse and cite themselves. This section provides information detailing the under-represented groups that are under-cited and how a citation diversity statement can be a solution to reduce inequality in citation politics.

advocacy Paper

A Manifesto for Rewarding and Recognizing Team Infrastructure Roles

This manifesto advocates for the formal recognition and reward of Team Infrastructure Roles (TIRs), such as data stewards and software engineers, within the scientific research ecosystem. It highlights how current evaluation systems overlook these essential contributors and proposes a shift in institutional culture to value diverse specialized skills beyond project leadership.

critique Paper

Scientific reform, citation politics and the bureaucracy of oblivion

This article examines how the open science movement may inadvertently create a 'bureaucracy of oblivion' by using transparency and replicability as exclusionary filters in citation practices. It warns against uncritical interpretations of scientific reform that may unfairly devalue research conducted before or outside of the current reformist framework.

critique Paper

The Gollum Effect: The Issue of Research Opportunity Guarding in Academia

This article introduces the "Gollum effect" to describe the possessive guarding of research resources, such as specific study sites or species, within the field of conservation biology. It highlights how this gatekeeping behavior stifles collaboration and prevents new researchers from entering specific niche fields or accessing critical data.

evidence Preprint

Meta-Research: How problematic citing practices distort science

critique Paper

Automated citation recommendation tools encourage questionable citations

This article analyzes the risks associated with automated citation recommendation systems, arguing that they may inadvertently promote questionable citing practices by reinforcing existing social reward structures. It highlights how these tools can prioritize convenience and popularity over epistemic rigor, calling for a more cautious and transparent approach to their implementation in scientific publishing.

evidence Paper

Men Set Their Own Cites High: Gender and Self-citation across Fields and over Time

This study provides large-scale empirical evidence of a significant gender gap in self-citation practices by analyzing over a million research papers across multiple disciplines. It quantifies how men cite their own work substantially more than women do and demonstrates that this disparity has persisted or even increased over several decades.

advocacy Preprint

Citing decisions in psychology: A roadblock to cumulative and inclusive science

This resource advocates for a more intentional and systematic approach to citation within psychological science to support cumulative research and diversity, equity, and inclusion. It identifies current citation behaviors as barriers to representative science and encourages authors to adopt more conscientious decision-making processes when acknowledging intellectual contributions.

evidence Paper

A critical analysis of plant science literature reveals ongoing inequities

This empirical analysis quantifies systemic geographical and gender-based inequities within the plant science literature by examining roughly 300,000 publications. It highlights significant disparities in citation impact and publication footprint, revealing how affluent nations dominate the field while research from the Global South remains under-cited despite high biodiversity in those regions.

critique Paper

On misogynoir: citation, erasure, and plagiarism

This article provides a critical analysis of citation erasure and plagiarism, specifically regarding the intellectual labor of Black women in coining and developing the term 'misogynoir.' It exposes how academic and public discourse often detaches concepts from their originators, serving as a critique of the structural anti-Blackness and misogyny embedded in citation politics.

Opara, I. (2022). How to protect research ideas as a junior scientist. Nature. https://www.nature.com/articles/d41586-022-03750-0

evidence Paper

Women are credited less in science than men

This study provides empirical evidence that the perceived productivity gap in science is partially driven by women's contributions being systematically unacknowledged as formal authorship. Using multiple data sources, it demonstrates that women in research teams are significantly less likely than their male counterparts to receive credit for their work.

Smith, C. A. (2018). Cite Black women: A critical praxis. www.citeblackwomencollective.org/our-praxis.html

advocacy Paper

Cite Black Women: A Critical Praxis (A Statement)

This collective statement outlines the intellectual genealogy and core principles of the Cite Black Women movement, framing citation as a critical praxis. It serves as a call to action for scholars to intentionally recognize and engage with the intellectual contributions of Black women across all academic fields.

overview Paper

The rise of citational justice: how scholars are making references fairer

This resource provides an overview of the emerging movement toward citational justice, examining how scholars are identifying and correcting biases in academic referencing. It surveys various strategies and motivations behind current efforts to make citation practices more equitable and representative.

evidence Paper

Disabled academics: a case study in Canadian universities

This qualitative study explores the lived experiences of disabled faculty members in Canadian universities, highlighting the tensions between neoliberal performance standards and institutional accommodation policies. It contributes empirical insights into how systemic academic structures and medicalized concepts of disability marginalize scholars.

evidence Paper

Gendered citation practices in the field of communication

This research analyzes citation patterns within 14 communication journals to quantify how gender influences referencing practices in the field. The findings reveal a significant under-citation of women-led research, primarily driven by the citation choices of male authors and the structure of their professional networks.

practice/tools Paper

The Citation Diversity Statement: A Practice of Transparency, A Way of Life

This resource introduces and defines the 'Citation Diversity Statement' as a practical tool for researchers to quantify and transparently disclose the demographic diversity of their reference lists. It provides a framework for integrating reflexive citation practices into the research workflow to help mitigate systemic biases in scholarly attribution.

▸ Decolonizing Research Practices 26 / 26

advocacy Paper

Decolonizing Psychological Science: Introduction to the Special Thematic Section

This article advocates for the decolonization of psychology, arguing that the field currently overrepresents Euro-American interests and colonial structures. It serves as a call to action for the discipline to center critical voices and the interests of the global majority to ensure research sustainability and social relevance.

evidence Website

Framing power: Tracing key discourses in open science policies

This paper conducts a critical analysis of open science policies to uncover how power dynamics and colonial legacies are embedded within their language and objectives. It highlights how current policy frameworks may inadvertently reinforce global inequalities while appearing to promote openness and equity.

Ahmed, S. (2017). Living a feminist life. Duke University Press.

policies Paper

The CARE Principles for Indigenous Data Governance

This resource establishes the CARE Principles for Indigenous Data Governance (Collective Benefit, Authority to Control, Responsibility, and Ethics), providing a global policy framework for the ethical management of Indigenous data. It complements technical data standards by focusing on the rights, interests, and sovereignty of Indigenous peoples within the open science landscape.

Chilisa, B. (2012). Indigenous research methodologies. Sage.

Coulthard, G. S. (2014). Red skin, white masks: Rejecting the colonial politics of recognition. University of Minnesota Press.

Denzin, N. K., Lincoln, Y. S., & Smith, L. T. (Eds.). (2008). Handbook of critical and indigenous methodologies. Sage.

Fricker, M. (2007). Epistemic injustice: Power and the ethics of knowing. Oxford University Press.

practice/tools Paper

The Application of Participatory Action-Research in Latin America

This resource details the methodology and specific techniques of Participatory Action-Research (PAR), emphasizing the integration of theory, social action, and participant involvement. It offers a practical framework for conducting research that serves marginalized groups and challenges traditional academic hierarchies in the social sciences.

Fanon, F. (1963). The wretched of the earth (C. Farrington, Trans.). Grove Press.

Freire, P. (2000). Pedagogy of the oppressed (30th anniversary ed.). Continuum. (Original work published 1970)

Quijano, A. (2000). Coloniality of power, Eurocentrism, and Latin America. Nepantla: Views from South, 1(3), 533–580.

Kovach, M. (2009). Indigenous Methodologies. University of Toronto Press.

Lugones, M. (2007). Heterosexualism and the colonial/modern gender system. Hypatia, 22(1), 186-219.

Mbembe, A. (2001). On the Postcolony. University of California Press.

Mignolo, W. D. (2011). The darker side of Western modernity: Global futures, decolonial options. Duke University Press.

Mills, C. W. (1997). The Racial Contract. Cornell University Press.

Santos, B. de S. (2014). Epistemologies of the South: Justice against epistemicide. Routledge.

overview Paper

An introduction to decolonial research

This publication provides a comprehensive introductory survey of the motivations and theoretical foundations of decolonial research. It serves as an accessible guide for researchers to understand how coloniality shapes knowledge production and how to apply decolonial perspectives to their own investigative practices.

Smith, L. T. (2012). Decolonizing methodologies: Research and Indigenous peoples (2nd ed.). Zed Books.

Spivak, G. C. (1988). Can the subaltern speak? In C. Nelson & L. Grossberg (Eds.), Marxism and the interpretation of culture (pp. 271–313). University of Illinois Press.

Patel, L. (2015). Decolonizing educational research: From ownership to answerability. Routledge.

Simpson, L. B. (2017). As we have always done: Indigenous freedom through radical resistance.

Tuck, E., & Yang, K. W. (2012). Decolonization is not a metaphor. Decolonization, 1(1), 1–40.

Wilson, S. (2020). Research is ceremony: Indigenous research methods. Fernwood publishing.

Zuberi, T., & Bonilla-Silva, E. (Eds.). (2008). White logic, white methods: Racism and methodology. Rowman & Littlefield.

▸ Diversity in Academia 23 / 23

Diversity is the presence of difference within a specific environment, e.g. racial diversity, gender diversity, social-economic diversity, neurodiversity, etc.

APA. (2017, July). Women & Socioeconomic Status. American Psychological Association. https://www.apa.org/pi/ses/resources/publications/women

APA (2010). Disability & Socioeconomic Status. American Psychological Association. https://www.apa.org/pi/ses/resources/publications/disability

APA (2010). Sexual Orientation, Gender identity & Socioeconomic Status. American Psychological Association. https://www.apa.org/pi/ses/resources/publications/lgbt

APA (2017, July). Education and Socioeconomic Status. American Psychological Association. https://www.apa.org/pi/ses/resources/publications/education

APA (2017, July). Ethnic and Racial Minorities & Socioeconomic Status American Psychological Association. https://www.apa.org/pi/ses/resources/publications/minorities

critique Paper

Open Science Isn't Always Open to All Scientists

Bossu, C. & Vladimirschi, V. (2020). Diversity, equity and inclusion in Latin America in the context of an open education initiative, OE Global Connect. https://connect.oeglobal.org/t/diversity-equity-and-inclusion-in-latin-america-in-the-context-of-an-open-education-initiative/387

critique Paper

Ableism in academia: where are the disabled and ill academics?

This resource critiques the current marketized and performance-driven culture of higher education, highlighting how these structures create barriers for disabled and chronically ill academics. It draws attention to the rising rates of burnout and mental health issues caused by the intensification of workloads and excellence exercises.

Brown, N., & Leigh, J. (2020). Ableism in Academia: Theorising Experiences of Disabilities and Chronic Illnesses in Higher Education. UCL Press.

evidence Paper

Bias against research on gender bias

This resource investigates the meta-scientific phenomenon of bias directed specifically toward research that explores gender bias. Through a bibliometric investigation, it demonstrates how research on gender bias is undervalued within academic institutions, identifying this as a distinct barrier to scientific progress and equity.

evidence Paper

Women’s career confidence in a fixed, sexist STEM environment

This longitudinal study tracks doctoral students to identify the psychological mechanisms that influence women's career confidence and persistence within sexist STEM environments. It contributes empirical data on how social identity threats at the graduate level impact the long-term retention of diverse talent.

advocacy Preprint

Bridging Neurodiversity and Open Scholarship: How Shared Values Can Guide Best Practices for Research Integrity, Social Justice, and Principled Education

This position statement articulates the synergy between neurodiversity and open scholarship, arguing that shared values of transparency and inclusivity can improve research integrity and social justice. It offers a roadmap for academic reform authored by researchers with lived experience, focusing on institutional changes that better support neurodivergent scholars.

Flaherty, C. (2020, August, 20). Something's Got to Give. Inside Higher Ed. https://www.insidehighered.com/news/2020/08/20/womens-journal-submission-rates-continue-fall

practice/tools Paper

How we investigated the diversity of our undergraduate curriculum

This resource provides a practical framework and case study for auditing the diversity of an undergraduate curriculum. It outlines a methodology for assessing representation in reading lists and course materials, offering a replicable process for institutions seeking to evaluate pedagogical inclusivity.

evidence Paper

The Pandemic and Gender Inequality in Academia

This resource investigates how the COVID-19 pandemic exacerbated existing gender disparities within academic research and productivity. It provides empirical insights into the disproportionate impact of the global health crisis on the career trajectories and labor patterns of women in academia.

advocacy Preprint

Open Science as Confused: Contradictory and Conflicting Discourses in Open Science Guidance to Researchers

evidence Paper

Bibliometrics: Global gender disparities in science

This bibliometric study quantifies global gender gaps in scientific authorship, citations, and collaboration across millions of papers. It provides large-scale empirical evidence of the systemic disadvantages women face in the scientific enterprise, highlighting the need for structural changes in academic evaluation.

critique Paper

Open Science and Epistemic Diversity: Friends or Foes?

evidence Paper

Unequal effects of the COVID-19 pandemic on scientists

This study provides empirical evidence on how the COVID-19 pandemic disproportionately affected the research productivity of different groups of scientists. It highlights significant disparities based on gender and caregiving responsibilities, quantifying the unequal impact of the crisis on the scientific workforce.

evidence Paper

Underrepresented and in/visible: A Hispanic first-generation student’s narratives of college.

This article uses a case study approach to provide narrative-based insights into the unique challenges and experiences of Hispanic first-generation college students. It highlights the systemic barriers to persistence and graduation while emphasizing the intersection of identity and institutional support structures.

Quagliata, T. (2008). Is there a positive correlation between socioeconomic status and academic achievement?. Paper: Education masters (p. 78). https://fisherpub.sjfc.edu/cgi/viewcontent.cgi?article=1077&context=education_ETD_masters

advocacy Journal Article

Why and how to incorporate issues of race/ethnicity and gender in research integrity education

This resource critiques the persistent authorship discrepancies in collaborative research between the Global North and Global South, arguing that current voluntary guidelines are insufficient. It advocates for the implementation of mandatory authorship for local researchers as a necessary policy intervention to ensure equitable credit and power dynamics in global health partnerships.

advocacy Paper

On supporting early-career Black scholars

This resource advocates for institutional and individual commitment to better support early-career Black scholars within the academic system. It emphasizes the importance of active mentorship and structural changes to address the unique barriers these researchers face while pursuing scientific careers.

▸ Diversity sample and generalizability 4 / 4

Data within Psychology has been primarily from a Western, Educated, Industrious, Rich and Democratic population and generalised the findings to participants across the globe. However, researchers have rarely included Global South or discussed sample diversity in terms of race, ethnicity, gender diversity among other areas. Even when these samples are included, the findings are found to be limited to that specific region.

advocacy Paper

Psychology should generalize from — not just to — Africa

This article challenges the field of psychology to treat African contexts as sources of foundational theory rather than merely sites for testing Western-centric models. It argues for a paradigm shift that centers African perspectives to improve the global generalizability and validity of psychological research.

critique Paper

It’s time to reimagine sample diversity and retire the WEIRD dichotomy

This resource critiques the scientific community's reliance on the WEIRD dichotomy as an oversimplified proxy for sample diversity. It calls for researchers to adopt more sophisticated and nuanced frameworks for conceptualizing and reporting human diversity across the behavioral sciences.

critique Preprint

Big-team science does not guarantee generalizability

This resource challenges the assumption that large-scale collaborative research projects inherently provide generalizable global findings. It specifically analyzes a study on temporal discounting to show how large-scale data collection can still fail to achieve true globalizability in psychological research.

practice/tools Paper

Constraints on Generality (COG): A Proposed Addition to All Empirical Papers

The authors propose the "Constraints on Generality" (COG) statement as a standardized addition to the discussion section of all empirical research articles. This practice requires researchers to explicitly define their target populations and justify why their specific samples allow for the inferences being made.

▸ Equity 26 / 26

Equity is that everyone has access to the same opportunities and that we all have privileges and barriers, thus we do not all start from the same starting position.

APA. (2017, July). Women & Socioeconomic Status. American Psychological Association. https://www.apa.org/pi/ses/resources/publications/women

APA (2010). Disability & Socioeconomic Status. American Psychological Association. https://www.apa.org/pi/ses/resources/publications/disability

APA (2010). Sexual Orientation, Gender identity & Socioeconomic Status. American Psychological Association. https://www.apa.org/pi/ses/resources/publications/lgbt

APA (2017, July). Education and Socioeconomic Status. American Psychological Association. https://www.apa.org/pi/ses/resources/publications/education

APA (2017, July). Ethnic and Racial Minorities & Socioeconomic Status American Psychological Association. https://www.apa.org/pi/ses/resources/publications/minorities

critique Paper

Open Science Isn't Always Open to All Scientists

evidence Paper

‘$100 Is Not Much To You’: Open Science and neglected accessibilities for scientific research in Africa

Based on empirical fieldwork in African laboratories, this paper highlights how the universalist goals of open science often overlook the specific infrastructural and financial barriers faced by researchers in the Global South. It demonstrates how costs and "neglected accessibilities" can turn open science practices into new mechanisms of exclusion.

critique Paper

Ableism in academia: where are the disabled and ill academics?

critique Paper

Replication and Reproduction: Crises in Psychology and Academic Labour

This resource analyzes the replication crisis in psychology through the lens of academic labor conditions and socio-economic shifts like the adjunctification of research. It argues that the crisis of "reproduction" in science cannot be separated from the precarious material conditions of the researchers performing the work.

evidence Paper

Bias against research on gender bias

advocacy Preprint

Bridging Neurodiversity and Open Scholarship: How Shared Values Can Guide Best Practices for Research Integrity, Social Justice, and Principled Education

Flaherty, C. (2020, August, 20). Something's Got to Give. Inside Higher Ed. https://www.insidehighered.com/news/2020/08/20/womens-journal-submission-rates-continue-fall

overview Paper

Equity, diversity, and inclusion in open education: A systematic literature review

This systematic literature review identifies the current state of equity, diversity, and inclusion (EDI) within open educational contexts and summarizes key implementation strategies. It provides a synthesized framework for involving institutional stakeholders to improve EDI outcomes across global educational systems.

evidence Paper

The Pandemic and Gender Inequality in Academia

evidence Journal Article

Care for the soul of science: Equity and virtue in reform and reformation

This paper provides a detailed case study of the 'KEMRI Community Representatives' network in coastal Kenya, an alternative model for community engagement in research ethics. It contributes empirical insights into how community members perceive their roles and the selection processes used to ensure representative voices in international research settings.

advocacy Preprint

Open Science as Confused: Contradictory and Conflicting Discourses in Open Science Guidance to Researchers

evidence Paper

Bibliometrics: Global gender disparities in science

evidence Paper

Unequal effects of the COVID-19 pandemic on scientists

evidence Review Article

Global health collaborative research: beyond mandatory collaboration to mandatory authorship

This study presents empirical findings from community consultations in Kenya regarding what constitutes fair benefits and payments for participants in international health research. It challenges traditional concerns about undue inducement by highlighting the ethical necessity of addressing the social realities and structural unfairness faced by impoverished participants.

advocacy Journal Article

Why and how to incorporate issues of race/ethnicity and gender in research integrity education

advocacy Paper

On supporting early-career Black scholars

critique Letter

Open science, done wrong, will compound inequities

This work warns that the global adoption of open science practices may exacerbate existing inequalities if the diverse socio-economic contexts of researchers are not considered. It serves as a call to action for the research-reform movement to prioritize equity and avoid creating new barriers for scholars in resource-constrained settings.

Sven Ulpts. (2024). Responsible assessment of what research? Beware of epistemic diversity! Meta-Psychology, 8. https://doi.org/10.15626/MP.2023.3797

▸ Feminist Thought 13 / 13

It aims to understand the nature of gender inequality. Themes explored include discrimination, objectification, oppression, patriarchy, stereotyping, and aesthetics. It examines women's and men's social roles, experiences, interests, chores, and feminist politics in a variety of fields.

Anon. (2022). OpenSexism Archives on Open Science. OpenSexism. https://opensexism.wordpress.com/tag/open-science/

critique Paper

Open Science and Feminist Ethics: Promises and Challenges of Open Access

overview Paper

Feminist Theory, Feminist Psychology: A Bibliography of Epistemology, Critical Analysis, and Applications

This resource provides a curated bibliography of feminist scholarship across psychology, social sciences, and philosophy, organized into thematic sections such as epistemology and critical concept analysis. It serves as a navigational tool for researchers seeking to understand the intersection of feminist theory and psychological practice post-1980.

Davis, A. Y. (1983). Women, race & class. Vintage.

D’Ignazio & Klein (2020). Data Feminism.

evidence Paper

Feminism and psychology: Analysis of a half-century of research on women and gender.

This article presents a longitudinal empirical analysis of 50 years of psychological research on women and gender, utilizing PsycINFO data to track the emergence and impact of feminist scholarship. It documents the historical growth of the field and evaluates how feminist activism influenced the volume and focus of psychological inquiry.

overview Paper

Doing reflexivity in psychological research: What’s the point? What’s the practice?

overview Paper

Celebrating 30 years of Feminism & Psychology

This editorial retrospective reflects on three decades of the journal Feminism & Psychology, highlighting its historical mission to critique and reconstruct mainstream psychological theory and practice. It provides an overview of the journal's evolution, celebrating its scholarly contributions while identifying ongoing challenges in the field.

overview Paper

Invited Reflection

This reflection provides an expert commentary on the historical development and theoretical priorities of feminist psychology. It synthesizes professional insights to articulate the importance of feminist perspectives in challenging traditional psychological paradigms.

advocacy Paper

Bridging Feminist Psychology and Open Science: Feminist Tools and Shared Values Inform Best Practices for Science Reform

This resource argues for the integration of feminist theory into the open science movement, highlighting how feminist scholarship can address systemic issues in psychology. It proposes a methodological synergy that leverages feminist tools to improve scientific practices and dismantle traditional power structures within the discipline.

Mohanty, C. T. (2003). Feminism Without Borders.

advocacy Paper

Can Open Science be a Tool to Dismantle Claims of Hardwired Brain Sex Differences? Opportunities and Challenges for Feminist Researchers

This resource evaluates the utility of open science practices, such as pre-registration and data sharing, as tools to challenge biased claims regarding hardwired brain sex differences. It specifically explores how these strategies can improve the reliability of neuroscience research and counter the misuse of findings to support gender stereotypes.

advocacy Paper

Navigating Open Science as Early Career Feminist Researchers

This article identifies the specific tensions and barriers feminist early career researchers encounter when engaging with the open science movement in psychology. It advocates for the integration of feminist perspectives to improve the inclusivity and rigor of open research while proposing ways to overcome systemic hurdles.

▸ Hidden curriculum 6 / 6

The ‘hidden curriculum’ of higher education refers to certain unspoken ‘rules of the game’ about the norms, processes, and language of higher education that students are implicitly assumed to have but are not explicitly taught or explained (Hubbard et al, 2020; Semper and Blasco, 2018). The existence of the ‘hidden curriculum’ means that some students are not equipped to navigate the unfamiliar territory of higher education, which can have consequences for their wellbeing, sense of belonging, and success.

Birtill, P., Harris, R., & Pownall, M. (2022). Student Guide to the Hidden Curriculum: Expanded Edition. https://warwick.ac.uk/students/together/welcome/internationalstudents/student-guide-to-the-hidden-curriculum_1.pdf

Gable, R. (2021). The hidden curriculum: First generation students at legacy universities. Princeton University Press.

evidence Paper

A Scoping Review on the Hidden Curriculum in Education

This scoping review analyzes 23 articles to empirically categorize the components of the hidden curriculum in education, such as reproduced social norms and power dynamics. It maps out how these implicit lessons are transmitted through teaching materials and routines, as well as their resulting effects on students and society.

overview Book

The Hidden Curriculum in Higher Education

This resource provides a comprehensive survey of how the hidden curriculum operates across various facets of higher education, including mentoring, dissertation advising, and specific professional tracks like engineering and business. It explores how institutional structures implicitly reproduce social stratification, gender roles, and capitalist values.

teaching/training Paper

Supporting students during the transition to university in COVID-19: Five key considerations and recommendations for educators

This paper utilizes the 'Five Senses of Student Success' model to offer educators evidence-based recommendations for supporting undergraduate students transitioning to university during the COVID-19 pandemic. It focuses on helping students reacclimatize to academic demands and rebuilding a sense of community and competence in a disrupted educational context.

overview Paper

Revealing the Hidden Curriculum in Higher Education

This resource provides an analysis of the 'hidden curriculum' in higher education, focusing on the implicit values and unspoken norms that influence student success. It aims to reveal how these informal structures can create barriers for students and offers a perspective on how institutions can recognize and address these underlying pedagogical factors.

▸ Inclusion 22 / 22

Inclusion is that individuals with different representations, identities and feelings being respected, influenced, and welcomed in a specific environment.

critique Paper

Open Science Isn't Always Open to All Scientists

critique Paper

Ableism in academia: where are the disabled and ill academics?

evidence Paper

Stereotypes About Gender and Science

This empirical study investigates the mismatch between gender stereotypes and the perceived characteristics of successful scientists, showing that scientific roles are strongly associated with agentic traits typically attributed to men. The findings provide evidence of how these deeply ingrained stereotypes can discourage women from pursuing scientific careers and contribute to ongoing gender imbalances within STEM fields.

evidence Paper

Bias against research on gender bias

evidence Paper

Scientific Eminence

This publication evaluates the gender gap in scientific eminence within the field of psychology by comparing publication metrics like the h-index and citation rates between male and female professors. It analyzes how historical trends and current productivity metrics contribute to the continued underrepresentation of women among the discipline's most honored figures.

advocacy Preprint

Bridging Neurodiversity and Open Scholarship: How Shared Values Can Guide Best Practices for Research Integrity, Social Justice, and Principled Education

Flaherty, C. (2020, August, 20). Something's Got to Give. Inside Higher Ed. Retrieved from https://www.insidehighered.com/news/2020/08/20/womens-journal-submission-rates-continue-fall.

practice/tools Paper

Inclusive Practices for Neurodevelopmental Research

critique Paper

Most people are not WEIRD

This landmark paper critiques the pervasive reliance on 'WEIRD' (Western, Educated, Industrialized, Rich, and Democratic) populations in behavioral research, highlighting the limitations this creates for the generalizability of scientific findings. It challenges researchers to broaden their sampling strategies and reconsider the universality of human behavior as currently documented in mainstream psychology and social science.

advocacy Preprint

Open Science as Confused: Contradictory and Conflicting Discourses in Open Science Guidance to Researchers

evidence Paper

Bibliometrics: Global gender disparities in science

evidence Paper

Why is concealment associated with health and wellbeing? An investigation of potential mechanisms

This empirical research investigates the psychological pathways linking the concealment of stigmatized identities to poor health and well-being outcomes. Within the context of open science, it contributes evidence regarding how social exclusion and lack of transparency about identity can negatively impact researcher welfare.

Liu, M. (2023). Whose open science are we talking about? From open science in psychology to open science in applied linguistics. Language Teaching, 56(4), 443–450. https://doi.org/10.1017/S0261444823000307

teaching/training Paper

Surviving (thriving) in academia: feminist support networks and women ECRs

This paper reflects on the role of informal feminist reading groups as vital support networks for doctoral and early career researchers in political science. It highlights how these peer-led pedagogical spaces provide emotional and professional sustenance for scholars navigating institutional challenges and academic austerity.

evidence Paper

Unequal effects of the COVID-19 pandemic on scientists

practice/tools Paper

Shedding the cloak of neutrality: A guide for reflexive practices to make the sciences more inclusive and just

advocacy Paper

Navigating Open Science as Early Career Feminist Researchers

evidence Paper

Leveraging a collaborative consortium model of mentee/mentor training to foster career progression of underrepresented postdoctoral researchers and promote institutional diversity and inclusion

This publication presents evidence for the effectiveness of a collaborative, multi-institutional training model designed to enhance mentoring for underrepresented postdoctoral researchers in the biomedical sciences. It demonstrates how implementing evidence-based mentoring practices can drive cultural change and improve career outcomes for minority scholars.

advocacy Paper

On supporting early-career Black scholars

evidence Paper

The Gender Gap: Who Is (and Is Not) Included on Graduate-Level Syllabi in Social/Personality Psychology

This study provides empirical evidence of significant gender disparities in graduate-level social and personality psychology syllabi, where female first authors are consistently underrepresented. The findings reveal that this gap has not improved significantly since the 1980s and cannot be explained away by a preference for older classic literature.

practice/tools Paper

Ten simple rules for socially responsible science

▸ Multimodal barriers to communication 8 / 8

The way a person communicates can influence how they are perceived and their access to opportunities in academia, including not being a nonnative English speaker, sign-language, or accents.

Amano, T. (2023, July 18). Non-native English speaking scientists work much harder just to keep up, global research reveals. The Conversation. https://theconversation.com/non-native-english-speaking-scientists-work-much-harder-just-to-keep-up-global-research-reveals-208750

evidence Paper

Languages Are Still a Major Barrier to Global Science

This empirical study quantifies the prevalence of non-English scientific literature in the field of biodiversity conservation, showing that over a third of relevant documents are not in English. It demonstrates how language barriers create geographic and taxonomic biases in scientific knowledge, potentially hindering global conservation efforts.

practice/tools Paper

Ten tips for overcoming language barriers in science

This resource provides practical tips for researchers and institutions to navigate and reduce the barriers created by the dominance of the English language in science. It offers guidance on how to make scientific knowledge more accessible across linguistic boundaries and how to better support non-native English speakers in the scientific community.

advocacy Paper

Don’t focus on English at the expense of your science

This piece argues against prioritizing linguistic perfection in scientific publishing, calling for a shift in focus toward the quality of research rather than the mastery of English. It identifies language barriers as a systemic hurdle that restricts global participation in science and urges the community to reduce the burden on non-native speakers.

practice/tools Paper

Pointing Forward: Typing for Academic Access

This resource provides actionable principles and practices for including non-verbal students who use augmentative typing for communication in academic environments. It specifically champions the "presumption of competence" and universal design as practical frameworks to foster meaningful social and academic participation.

advocacy Paper

Academic Migration, Linguistic Justice, and Epistemic Injustice*

This work explores how the dominance of the English language in academia creates epistemic injustice, particularly for migrant scholars. It theorizes the systemic barriers that prevent non-native speakers from participating equitably in knowledge production and advocates for a more just linguistic framework in research.

advocacy Dataset

Reviewers, don't be rude to nonnative English speakers

This article provides a direct call to action for peer reviewers to eliminate linguistic bias and rudeness when evaluating the work of non-native English speakers. It highlights how insensitive feedback on language acts as an exclusionary gatekeeping mechanism that can unfairly penalize quality research.

evidence Paper

Is Your Accent Right for the Job? A Meta-Analysis on Accent Bias in Hiring Decisions

This meta-analysis provides empirical evidence for accent bias in hiring, quantifying the extent to which standard-accented candidates are favored over non-standard-accented peers. It evaluates competing psychological explanations, finding that linguistic "otherness" and prejudice are significant drivers of devaluation in professional evaluations.

▸ Neurodiversity 19 / 19

Neurodiversity refers to non-pathological variation in the human brain regarding movement, sociability, learning, attention, mood, and other mental functions at a group level (Singer, 2017).

practice/tools Paper

Doing it differently: emancipatory autism studies within a neurodiverse academic space

This resource provides a methodological reflection on how to implement emancipatory and co-produced research practices within autism studies. It specifically demonstrates how a neurodiverse academic community can challenge traditional knowledge production by involving autistic and autism communities directly in the research process.

overview Paper

Come as You Are: Examining Autistic Identity Development and the Neurodiversity Movement through an Intersectional Lens

This resource integrates intersectionality into neurodiversity research by reviewing literature on the development of autistic identity alongside other marginalized social identities. It highlights how cultural traditions and autistic culture can mitigate minority stress, providing a theoretical framework for more inclusive research and practice.

Brixius-Anderko, S. (2023). Nothing wrong with me. ELife, 12. CLOCKSS. https://doi.org/10.7554/eLife.93330

critique Paper

Ableism in academia: where are the disabled and ill academics?

critique Paper

Neurodiversity and the Social Ecology of Mental Functions

This paper critiques the traditional medical model of psychiatry that pathologizes neurocognitive differences by comparing them to narrow functional norms. It proposes a theoretical alternative that reframes neurodiversity as a natural manifestation of biodiversity within a social-relational framework.

overview Paper

What Can Stutterers Learn from the Neurodiversity Movement?

This resource explores the application of neurodiversity principles to the experience of stuttering, contrasting the traditional medical model of pathology with social and relational models of disability. It provides a conceptual framework for individuals who stutter to understand their traits as natural variations rather than impairments.

overview Paper

The Neurodiversity Approach(es): What Are They and What Do They Mean for Researchers?

This paper provides a conceptual clarification of the neurodiversity movement and its various theoretical approaches to help researchers navigate conflicting definitions and debates. It specifically addresses common misconceptions, such as the tension between individual characteristics and societal barriers, to facilitate more nuanced research on disability.

advocacy Preprint

Bridging Neurodiversity and Open Scholarship: How Shared Values Can Guide Best Practices for Research Integrity, Social Justice, and Principled Education

practice/tools Paper

Inclusive Practices for Neurodevelopmental Research

advocacy Paper

Opening up understanding of neurodiversity: A call for applying participatory and open scholarship practices

teaching/training Paper

Compassionate pedagogy for neurodiversity in higher education: A conceptual analysis

advocacy Letter

From Puzzle to Progress: How Engaging With Neurodiversity Can Improve Cognitive Science

This paper advocates for the integration of the neurodiversity paradigm into cognitive science to improve the field's research quality and ethical standards. It argues that recognizing neurodivergence as natural variation, rather than a deficit, leads to more robust and representative scientific conclusions.

critique Paper

On the ontological status of autism: the ‘double empathy problem’

This piece critiques traditional psychological discourses that treat autism as an inherent social deficit, proposing the "double empathy problem" as an alternative framework. It argues that social communication breakdowns are a bidirectional issue occurring between different neurotypes rather than a pathology located within the individual.

advocacy Paper

Academia’s ableist culture laid bare

This article examines the pervasive culture of ableism in academia and the structural barriers it presents to disabled and neurodivergent researchers. It serves as a call to action for institutional change, advocating for a shift in academic norms to foster genuine inclusion and accessibility.

advocacy Paper

The conference challenges faced by scientists who stutter

This resource highlights the specific communication barriers and exclusionary practices faced by scientists who stutter within professional conference settings. It calls for conference organizers to implement more inclusive norms and accommodations to ensure equitable participation for all researchers.

evidence Paper

Only Human: Mental-Health Difficulties Among Clinical, Counseling, and School Psychology Faculty and Trainees

This study provides empirical data on the high prevalence of mental health difficulties among faculty and trainees in applied psychology fields, highlighting that such experiences are common despite professional taboos. It contributes to the discussion on neurodiversity by documenting the gap between the lived experiences of practitioners and the field's silence on the topic.

advocacy Paper

Five things I wish academia understood about my social anxiety

This personal narrative highlights the specific barriers that traditional academic structures and social norms pose for researchers with social anxiety. It advocates for institutional shifts toward more inclusive environments that better accommodate and value the contributions of neurodivergent scholars.

advocacy Paper

Disabled in academia: to be or not to be, that is the question

This resource explores the complex personal and professional dilemmas regarding disability disclosure within the academic workforce. It contributes to the dialogue on inclusivity by examining how systemic barriers influence whether disabled scholars choose to remain visible or hidden in their roles.

To add: https://www.scienceopen.com/hosted-document?doi=10.13169/intljofdissocjus.5.3.0002

▸ Objectivity in Research 4 / 4

Objectivity in scientific research refers to a truth or independent reality exists outside of any observation such that personal beliefs, interests, judgements, bias, bias or opinions should not affect the independent reality or truth. Here we discuss that readers understand that being objective in research is a myth.

Cooke, L. (2022). Bitch: A Revolutionary Guide to Sex, Evolution and the Female Animal. Doubleday UK.

advocacy Preprint

Subjectivity is a Feature, not a Flaw: A Call to Unsilence the Human Element in Science

Harding, S. (2015). Objectivity and Diversity.

Melanie, F. (2023, April 17). The Myth of Objective Data. The MIT Press Reader. https://thereader.mitpress.mit.edu/the-myth-of-objective-data/?s=03

▸ Racism in science 32 / 32

Addresses the legacy and continuing impact of racism in research contexts, from biased study designs to the underrepresentation and marginalization of researchers of color today. Encourages critical examination of how racial biases can affect who does science, what topics are studied, and how results are interpreted.

advocacy Paper

Changing the culture of peer review for a more inclusive and equitable psychological science.

This article examines how current peer-review practices in psychology and neuroscience can perpetuate systemic inequities and exclusion. It provides a framework for reform, advocating for reviewer training and institutional changes to ensure feedback is culturally sensitive and professionally supportive of a diverse scientific community.

teaching/training Paper

Tracking Privilege‐Preserving Epistemic Pushback in Feminist and Critical Race Philosophy Classes

This resource analyzes 'privilege-preserving epistemic pushback' as a form of willful ignorance encountered when teaching social justice topics in philosophy and critical race studies. It provides a conceptual framework for understanding how students from dominant groups resist learning about structural injustices, helping educators identify and navigate these classroom dynamics.

practice/tools Paper

Upending racism in psychological science: Strategies to change how science is conducted, reported, reviewed, and disseminated.

This article provides a roadmap for deconstructing systemic racism within psychological science by offering specific strategies for the conduct, review, and dissemination of research. It identifies how traditional scientific norms maintain white supremacy and proposes actionable changes to reform the research lifecycle to be more equitable for BIPOC scholars and populations.

advocacy Paper

Intersectionality Research in Psychological Science: Resisting the Tendency to Disconnect, Dilute, and Depoliticize

This work critiques the way psychological research often dilutes or depoliticizes intersectionality, stripping it of its focus on power and systemic oppression. It advocates for researchers to return to the concept's critical roots to ensure that studies of identity are both methodologically rigorous and socially transformative.

advocacy Paper

Rigour versus the need for evidential diversity

This paper defends the necessity of 'evidential diversity,' arguing that a narrow focus on traditional scientific rigor can overlook the variety of evidence required to understand complex causal processes. It offers a template for categorizing different types of evidence, advocating for a mixed-methods approach to better establish causal claims in specific, real-world settings.

overview Paper

Racial Inequality in Academia: Systemic Origins, Modern Challenges, and Policy Recommendations

This resource synthesizes research on the systemic origins and psychological challenges of racial inequality within the broader academic landscape. It provides a comprehensive set of policy recommendations designed to address the structural barriers that hinder the recruitment, retention, and scientific impact of scholars of color.

advocacy Paper

Psychological Science Is Not Race Neutral

This resource identifies specific psychological phenomena—racial ignorance, threats to belonging, and racial-progress narratives—that sustain racial inequality within the field of psychology. It moves beyond identifying the lack of diversity to explaining the structural drivers of this inequality and offers actionable recommendations for journals to mitigate these issues.

critique Paper

Making the Invisible Visible: Acts of Commission and Omission

This paper critiques traditional social psychological theories of prejudice for focusing almost exclusively on active "commissions" while ignoring the harmful effects of "omissions" or the invisibility of Native Americans in public consciousness. It highlights how the intentional exclusion of specific groups from research and representation constitutes a distinct and damaging form of discrimination.

critique Paper

Epistemic Oppression, Construct Validity, and Scientific Rigor: Commentary on Woo et al. (2022)

This commentary critiques a specific psychometric study on the GRE for failing to include relevant bodies of research and neglecting construct validity, framing these omissions as forms of epistemic oppression. It emphasizes the necessity of maintaining scientific rigor and respect when addressing topics related to systemic violence and standardized testing.

practice/tools Editorial

Responses to 10 common criticisms of anti-racism action in STEMM

This resource provides a practical guide for addressing and rebutting ten common arguments used to resist anti-racism initiatives within STEMM fields. It equips scientists with evidence-based counter-arguments to foster more productive dialogue and promote institutional change regarding racial bias.

critique Paper

Pushing Back Against the Microaggression Pushback in Academic Psychology: Reflections on a Concept-Creep Paradox

This paper analyzes the academic resistance to the concept of microaggressions within psychology, documenting how concept-creep concerns and political skepticism have stifled empirical research on the topic. It critiques the field's tendency to publish theoretical objections to microaggression theory while simultaneously failing to produce the empirical data necessary to evaluate its validity.

critique Letter

Fighting over who dictates the nature of prejudice

This resource critiques current psychological trends that shift the operationalization of prejudice away from the lived experiences of marginalized groups toward right-leaning priorities. It argues for a research approach that prioritizes macro-level societal contexts and aligns with real-world findings rather than ideological abstractions.

advocacy Paper

Scholars of color explore bias in academe: Calling in allies and sharing affirmations for us by us

This article centers the voices of scholars of color to share affirmations and strategies for navigating systemic bias within the academic landscape. It serves as an advocacy piece that calls for active allyship while providing communal support for those facing institutional racism.

evidence Paper

Derogating the Victim: The Interpersonal Consequences of Blaming Events on Discrimination

This experimental study provides empirical evidence of the 'troublemaker' stigma faced by African Americans who attribute negative outcomes to racial discrimination. It demonstrates that victims incur significant interpersonal costs for reporting bias, even when the discrimination they encountered was blatant and undeniable.

critique Paper

On the limits of antiracism: how antiracist opposition is connected to racism denial in Germany

This work analyzes the limitations of antiracist efforts in Germany, specifically how opposition to these movements is intertwined with systemic racism denial. It examines the socio-political barriers that prevent antiracist discourse from effectively addressing structural inequality in this specific national context.

evidence Paper

US universities are not succeeding in diversifying faculty

This resource presents data-driven findings on the failure of US universities to meaningfully diversify their faculty populations over time. It highlights the disconnect between institutional diversity pledges and actual demographic shifts, providing a critique of the efficacy of current academic hiring practices.

McIntosh, P. (1989, July–August). White privilege: Unpacking the invisible knapsack. Peace and Freedom

advocacy Paper

Addressing racism in editorial practices

This resource highlights the pervasive nature of racism within academic publishing, specifically during the editorial stages of screening, reviewing, and editing. It serves as a call to action for journal editors to recognize their own biases and implement structural changes to promote racial equity in scientific dissemination.

critique Preprint

Dealing with Diversity in Psychology: Science and Ideology

This resource provides a critical perspective on the ideological and systemic barriers that marginalize minority scholars within psychology. By framing the discussion as a personal and professional testimony, it challenges the discipline to confront unsettling truths about how scholarly discourse is shaped by exclusion and oppression.

Roberts, S. O., Bareket-Shavit, C., Dollins, F. A., Goldie, P. D., & Mortenson, E. (2020). Racial inequality in psychological research: Trends of the past and recommendations for the future. Perspectives on psychological science, 15(6), 1295-1309. https://journals.sagepub.com/doi/full/10.1177/1745691620927709

evidence Paper

Challenging the White = Neutral Framework in Psychology

This resource identifies and provides evidence for the "White = neutral" framework in psychology, where White samples are often treated as the universal standard for human behavior. It contributes a clear analysis of how these research practices undermine the field’s generalizability and ignore the racialized experiences of diverse populations.

overview Paper

The psychology of American racism.

This resource synthesizes a wide range of interdisciplinary research to categorize seven distinct psychological and social factors that contribute to the persistence of American racism. It provides a structured conceptual framework for understanding how cognitive biases, group dynamics, and institutional power interact to sustain racial inequality.

critique Paper

Overcoming racism in the twin spheres of conservation science and practice

This resource analyzes how institutional structures within conservation science and practice work together to maintain systemic racism and marginalize BIPOC communities. It specifically traces how educational curricula and professional advancement opportunities perpetuate limited and exclusionary perspectives on conservation history and goals.

Saini, A. (2019). Superior: the return of race science. Beacon Press.

overview Paper

Toward a Critical Race Psychology

This resource articulates a framework for Critical Race Psychology by integrating core tenets of Critical Race Theory into the psychological domain. It shifts the analytical focus from individual bias to systemic forces and explores how neoliberal ideologies of individualism often reproduce racial domination.

overview Paper

Racism in the Structure of Everyday Worlds: A Cultural-Psychological Perspective

This review provides a cultural-psychological perspective on racism, arguing that it must be studied as a systemic reality embedded in cultural artifacts and institutional structures rather than just individual prejudice. It highlights specific examples of how historically derived cultural patterns continue to maintain contemporary racial inequalities.

Thompson, A. (1997). For: Anti-Racist Education. Curriculum Inquiry, 27(1), 7–44. http://www.jstor.org/stable/1180053

practice/tools Preprint

Racial Equity in Social Psychological Science: A Guide for Scholars, Institutions, and the Field

This article serves as a practical guide for scholars and institutions to implement anti-racist practices and achieve racial equity within social psychology. It provides specific, actionable recommendations to challenge objectivity norms that reify Whiteness and to address the inequitable allocation of resources.

West, C. (2001). A genealogy of modern racism. In P. Essed & D. T. Goldberg (Eds.), Race critical theories: Text and context (pp. 90–112). Wiley-Blackwell

evidence Paper

It's only discrimination when they do it to us: When White men use ingroup‐serving double standards in definitional boundaries of discrimination

Through three experiments, this study provides empirical evidence that White men employ ingroup-serving double standards when defining discrimination. The findings demonstrate that this group tends to define racism and sexism narrowly when committed by their own group but broadly when they are the targets of such actions.

evidence Preprint

Antiracism and its Discontents: Opposition to Antiracism is a Widespread and Politically Influential Racial Ideology among White Americans

This research identifies and empirically validates "anti-antiracism" as a distinct, stable, and coherent racial ideology among White Americans. Across five studies, the authors demonstrate how this ideology functions as a primary driver of opposition to antiracist claims and contemporary social justice movements.

https://en.wikipedia.org/wiki/Elsevier (go to "Scientific Racism” section)

▸ Religion and culture 7 / 7

One’s degree of privilege/marginalisation based on religion or culture will vary depending on the country or region that people reside in.

evidence Paper

Islamophobia in education: perceptions on the use of veil/niqab in higher education

This qualitative study provides empirical evidence regarding the motivations and experiences of female students wearing the niqab in Indonesian higher education. It highlights the specific social and ideological obstacles these students face, offering insight into how religious identity intersects with perceptions of extremism in academic settings.

teaching/training Paper

Reflections on addressing antisemitism in a Canadian faculty of medicine

This article offers pedagogical and institutional reflections on addressing antisemitism within a Canadian medical faculty. It contributes to the discussion by sharing specific experiences and strategies for creating more inclusive and culturally sensitive training environments in health professions education.

evidence Paper

Outsider status, and racialised habitus: the experiences of Gypsy, Roma, and Traveller students in higher education

Using qualitative interviews and Bourdieusian theory, this research explores the systemic exclusion of Gypsy, Roma, and Traveller students in UK higher education. The study reveals how university environments operate as 'white habitus' spaces that devalue the cultural capital of these minority groups.

advocacy Letter

Preserve cultural diversity in author names

This resource argues for the importance of respecting and preserving cultural diversity in the representation of author names across scholarly publishing. It highlights the need for publishing systems and databases to accommodate diverse naming conventions rather than forcing them into a Western-centric format.

advocacy Paper

More Than a Checklist: Meaningful Indigenous Inclusion in Higher Education

This work makes the case for moving beyond superficial 'checklists' toward meaningful institutional change for Indigenous inclusion in higher education. It identifies the systemic barriers that maintain educational disparities and provides a framework for creating academic spaces that truly value Aboriginal participation and success.

evidence Paper

Palestinian students in an Israeli-Hebrew University: obstacles and challenges

This resource examines the specific socio-political and institutional obstacles encountered by Palestinian students within the context of Israeli-Hebrew universities. It details how the intersection of cultural identity and the institutional environment creates unique challenges for integration and academic success.

advocacy Paper

Early career Latinas in STEM: Challenges and solutions

This publication identifies the unique institutional and cultural barriers faced by Latina scientists in STEM and offers actionable solutions to improve their representation and career progression. It emphasizes the need for intersectional institutional reforms to address the specific challenges that early-career women of color encounter in scientific fields.

▸ Research assessment, proposals, and reforms 10 / 10

Examines how researchers and institutions are evaluated and efforts to reform these systems to better align with open and responsible science. INSERT DESCRIPTION

advocacy Paper

An Agenda for Open Science in Communication

evidence Paper

Ungendered writing: Writing styles are unlikely to account for gender differences in funding rates in the natural and technical sciences

This study provides empirical evidence evaluating whether gendered writing styles account for disparities in grant funding rates within the natural and technical sciences. By analyzing a large dataset of applications, it demonstrates that linguistic differences are unlikely to be the primary cause of funding gaps, focusing attention instead on other institutional biases.

critique Paper

Messing with Merton: The intersection between open science practices and Mertonian values

This resource explores the friction between modern open science initiatives and classic Mertonian scientific values, highlighting the practical obstacles that arise during implementation. It specifically examines how reliance on third-party technology and entrenched disciplinary cultures can hinder the realization of universalism and communalism in research.

evidence Journal Article

Care for the soul of science: Equity and virtue in reform and reformation

advocacy Letter

Gender, diversity, and the responsible assessment of researchers

This book advocates for a fundamental shift in the philosophy of science to incorporate feminist perspectives and greater social engagement. It contributes a framework for a socially responsible science that is explicitly designed to address societal needs and ethical concerns rather than maintaining a stance of detached objectivity.

Ross-Hellauer, Tony & Aubert Bonn, Noémie & Horbach, Serge P. J. M., (2023). Understanding the social and political dimensions of research(er) assessment: Interpretative flexibility and hidden criteria. SocArXiv https://ideas.repec.org/p/osf/socarx/e5uyv_v1.html

evidence Journal Article

Understanding the social and political dimensions of research(er) assessment: evaluative flexibility and hidden criteria in promotion processes at research institutes

This research investigates the "hidden criteria" and social dimensions that influence researcher assessment beyond formal policies and quantitative metrics. It provides empirical insights into how evaluative flexibility allows for subjective factors to persist in promotion processes despite institutional calls for more transparent and diverse assessment sets.

evidence Paper

Putting open science into practice: A social dilemma?

Drawing on interviews with researchers from diverse academic backgrounds, this study identifies the individual and structural barriers that cause a discrepancy between open science ideals and actual research practices. It frames the challenges of putting open science into practice as a social dilemma, offering a systematic analysis of the obstacles faced by researchers across disciplines.

critique Preprint

The social replication of replication: Moving replication through epistemic communities

practice/tools Paper

Opening science to society: how to progress societal engagement into (open) science policies

This resource provides actionable guidance for integrating societal engagement and public participation into national open science policies, addressing a critical implementation gap in the 2021 UNESCO Recommendation. It identifies specific policy instruments that can facilitate a shift toward inclusive knowledge systems and active engagement with diverse societal actors.

▸ Sexuality & Gender 6 / 6

Sexuality refers to the various aspects of an individual’s being related to their sexual feelings, thoughts, attractions and behaviour. We use “LGBTQ+” as an inclusive term to refer to all sexual identities and orientations which are not heterosexual. This includes but is not limited to lesbian, gay, bisexual, pansexual, asexual, queer, and questioning[ap].

evidence Paper

Systemic inequalities for LGBTQ professionals in STEM

This paper presents empirical documentation of five dimensions of systemic inequality faced by LGBTQ professionals in STEM using a large-scale representative survey. It demonstrates that LGBTQ individuals experience significantly higher levels of professional devaluation and workplace harassment compared to their non-LGBTQ colleagues, even when controlling for other demographic and job factors.

critique Paper

Sexism, racism, prejudice, and bias: a literature review and synthesis of research surrounding student evaluations of courses and teaching

This literature review synthesizes research on student evaluations of teaching to demonstrate that these metrics are systemically biased and influenced by racist, sexist, and homophobic prejudices. It critiques the continued use of these flawed assessments in higher education, highlighting how they unfairly penalize marginalized academics.

overview Dataset

How scientific conferences are responding to abortion bans and anti-LGBTQ+ laws

This article explores how scientific organizations are adjusting their conference planning in response to abortion bans and anti-LGBTQ+ legislation in various jurisdictions. It describes the range of institutional responses, from boycott strategies to increased safety measures, aimed at protecting participants and upholding professional values of inclusivity.

advocacy Preprint

Open Science as Confused: Contradictory and Conflicting Discourses in Open Science Guidance to Researchers

evidence Paper

Identity Concealment May Discourage Health-Seeking Behaviors: Evidence From Sexual-Minority Men During the 2022 Global Mpox Outbreak

This research provides empirical evidence that concealing a stigmatized identity can act as a barrier to accessing healthcare, specifically during public health emergencies. By surveying sexual-minority men during the 2022 mpox outbreak, the study demonstrates that concealment correlates with lower rates of vaccination and health-seeking behaviors.

advocacy Paper

Inclusive LGBTQ+ fieldwork: Advancing spaces of belonging and safety

This resource highlights the unique risks and exclusionary practices faced by LGBTQ+ researchers conducting fieldwork within the discipline of geography. It advocates for a systematic shift toward creating safer, more inclusive environments that ensure the physical and emotional well-being of marginalized researchers.

▸ Socially Responsible Research 1 / 1

Research agendas need to be balanced with societal needs and ethical imperatives.

practice/tools Paper

Ten simple rules for socially responsible science

▸ Societal acceptance of appearance 4 / 4

How we appear to others can contribute to privilege. Several appearances include facial features, hair texture, skin conditions and body size.

evidence Paper

The Natural Hair Bias in Job Recruitment

This publication offers empirical evidence of systemic bias in recruitment processes, specifically demonstrating how Black women with natural hairstyles are perceived as less professional and competent compared to those with straightened hair. Through multiple experimental studies, it quantifies the impact of hairstyle-based prejudice on competency ratings and hiring recommendations.

Rahbari, L. (2019). Beauty or the Beast? University Academics’ Perceptions of Women’s Physical Appearance and Academic Achievements. Journal of International Women's Studies, 20(2), 309-323. https://vc.bridgew.edu/jiws/vol20/iss2/21

Shahani-Denning, C. (2003). Physical attractiveness bias in hiring: What is beautiful is good. Hofstra Horizon, 14-17.

Shahani-Denning, C., Andreoli, N., Snyder, J., Tevet, R., & Fox, S. (2011). The effects of physical attractiveness and gender on selection decisions: An experimental study. International Journal of Management, 28(4), 16-23.

▸ Structures and incentives in academia 69 / 69

The academic system (promotion criteria, funding structures, competition for limited jobs) can incentivize quantity of publications, impact factor chasing, and other behaviors that conflict with openness and rigor.

Andersen, J. P., & Horbach, S. P. J. M. (2024). If AI is an accelerator, will research need speed limits? Research Professional News (Research Europe). https://www.researchprofessionalnews.com/rr-news-europe-views-of-europe-2024-12-if-ai-is-an-accelerator-will-research-need-speed-limits/

advocacy Letter

Towards a culture of open scholarship: the role of pedagogical communities

practice/tools Paper

How quality control could save your science

evidence Paper

Robust research: Institutions must do their part for reproducibility

This resource presents findings from a large-scale survey of researchers regarding their engagement with integrity policies and their perceptions of institutional research culture. It identifies a disconnect between formal integrity guidelines and the daily realities of research practice, emphasizing that cultural commitment is more critical to research integrity than administrative compliance.

evidence Paper

The Matthew effect in science funding

critique Paper

Games academics play and their consequences: how authorship, h -index and journal impact factors are shaping the future of academia

This paper critiques the current academic incentive structure, focusing on the negative consequences of relying on metrics like the h-index and journal impact factors for evaluation. It highlights how these pressures encourage "gaming" the system and can ultimately damage scientific integrity and the careers of early-career researchers.

advocacy Preprint

Why a Focus on Eminence is Misguided: A Call to Return to Basic Scientific Values

This resource advocates for a shift away from prestige-based evaluations in science toward a focus on transparency, universalism, and methodological rigor. It argues that scientific authority should derive from open and reproducible methods rather than the reputation of the individual researcher.

critique Preprint

Responsible Research Assessment Should Prioritize Theory Development and Testing Over Ticking Open Science Boxes

This paper cautions against an over-reliance on open science checklists in research assessment, arguing that such a narrow focus can marginalize the critical roles of theory development and formal modeling. It advocates for a more holistic approach to evaluating scientific quality that prioritizes a candidate's ability to contribute to theoretical knowledge alongside methodological transparency.

advocacy Paper

Promoting inclusive metrics of success and impact to dismantle a discriminatory reward system in science

This resource argues for the replacement of traditional bibliometric indicators with success metrics that incorporate principles of justice, equity, diversity, and inclusion. It specifically highlights the need for a paradigm shift that values multidimensional mentorship and mentee well-being as core components of scientific impact.

critique Paper

Scientific reform, citation politics and the bureaucracy of oblivion

advocacy Paper

Improving Departments of Psychology

This paper proposes structural reforms for psychology departments, specifically recommending the implementation of faculty development programs and specialized career tracks for research, education, and outreach. It argues that moving beyond simple hiring practices toward ongoing professional support and strategic faculty alignment will enhance departmental excellence.

advocacy Paper

The credibility crisis and democratic governance: how to reform university governance to be compatible with the nature of science

This resource proposes a model for university reform that aligns institutional governance with democratic principles and scientific logic to address the systemic causes of the credibility crisis. It outlines how restructuring institutional culture to be more transparent and participatory can foster a more reliable and flourishing research environment.

practice/tools Letter

Finding the best fit for improving reproducibility: reflections from the QUEST Center for Responsible Research

This resource shares practical lessons and reflections from five years of implementing reproducibility initiatives at a major biomedical research center. It identifies institutional challenges and provides actionable insights for engaging stakeholders across different levels to foster a culture of responsible research.

evidence Paper

Scientists’ Reputations Are Based on Getting It Right, Not Being Right

This study provides empirical evidence that a scientist’s reputation is more strongly influenced by their commitment to rigorous methodology and transparent responses to replication than by the initial correctness of their findings. By surveying diverse populations, it demonstrates that both the academic community and the public value the process of integrity over the pressure to produce positive results.

critique Paper

How Open Science organizations generate epistemic oppression

This paper provides a critical examination of how the organizational structures and standardized practices of the Open Science movement can lead to the marginalization of diverse knowledge systems. It highlights the risk of epistemic oppression when universal norms are applied without considering the power dynamics and varying contexts of researchers globally.

evidence Paper

Negative results are disappearing from most disciplines and countries

This meta-research study documents the increasing prevalence of positive-outcome bias across nearly all scientific disciplines and geographic regions over time. It provides quantitative evidence of the "disappearing" negative result, arguing that systemic pressures are distorting the scientific record and discouraging high-risk, rigorous research.

advocacy Paper

Intrinsic and Extrinsic Science

This article proposes a conceptual framework that balances "intrinsic" research driven by curiosity with "extrinsic" research driven by professional incentives and metrics. It argues that scientific excellence and transformative impact are best achieved through a synthesis of these two approaches, offering a new perspective on how researchers can navigate academic structures.

critique Paper

Open Science for private Interests? How the Logic of Open Science Contributes to the Commercialization of Research

critique Preprint

Fernanda Ferreira -- Fame: I'm Skeptical (2017)

This resource challenges the use of personal fame and name recognition as proxies for scientific quality in the field of psychology. It highlights how these biases perpetuate stereotypes, limit diversity, and contribute to systemic issues like the replication crisis by incentivizing prestige over rigorous work.

advocacy Paper

Faculty promotion must assess reproducibility

This piece advocates for a fundamental shift in academic evaluation by making research reproducibility a core criterion for faculty promotion and tenure. It argues that institutionalizing these standards is necessary to align professional incentives with the goals of scientific reliability and integrity.

overview Paper

Eminence and Omniscience

This article provides a conceptual framework for evaluating scientific merit by distinguishing between 'deep' and 'surface' eminence. It surveys existing suggestions for merit evaluation and offers a perspective on how decision-makers can better identify true scientific contributions amidst the fallibility of human judgment.

advocacy Paper

Rewarding Research Transparency

This resource argues for the importance of implementing formal rewards and incentives for research transparency within the academic structure. It emphasizes that shifting institutional values to favor open practices is essential for fostering a more transparent and credible research environment.

critique Paper

Perceived publication pressure in Amsterdam: Survey of all disciplinary fields and academic ranks

This article interrogates the popular "cargo cult science" metaphor to reveal its limitations in explaining the complexities of modern scientific failure. It encourages researchers to look beyond the surface-level imitation of scientific rituals and to critically evaluate the deeper methodological and statistical logic underpinning their work.

evidence Paper

Personally perceived publication pressure: revising the Publication Pressure Questionnaire (PPQ) by using work stress models

This study provides empirical data on the level of perceived publication pressure across different academic ranks and disciplines within a major metropolitan research hub. It highlights how the 'publish or perish' culture manifests differently depending on a researcher's field and seniority, offering a benchmark for institutional self-assessment.

practice/tools Paper

Researchers’ Perceptions of a Responsible Research Climate: A Multi Focus Group Study

This resource introduces and validates the Revised Publication Pressure Questionnaire (PPQr), a psychometrically sound instrument for measuring publication-related stress. It provides a standardized tool that institutions and researchers can use to quantitatively assess the impact of publication pressure on mental health and research integrity.

evidence Paper

Perceptions of research integrity climate differ between academic ranks and disciplinary fields: Results from a survey among academic researchers in Amsterdam

This research establishes a statistical link between perceived research climate and the reported prevalence of research misbehavior among academics. It demonstrates that local departmental norms are significant predictors of researcher conduct, suggesting that institutional interventions should target local culture rather than just individual behavior.

practice/tools Paper

An index to quantify an individual’s scientific research output that takes into account the effect of multiple coauthorship

This publication proposes a specific quantitative index designed to measure a researcher's individual scientific impact while mathematically adjusting for the influence of multiple coauthorship. It provides a standardized tool for academic evaluation that attempts to refine traditional citation metrics to better reflect personal contribution in collaborative research environments.

evidence Paper

On the Willingness to Report and the Consequences of Reporting Research Misconduct: The Role of Power Relations

This publication uses a comparative case study approach to model how universities respond to allegations of scientific misconduct through processes of sensemaking, sensegiving, and sensehiding. It addresses a gap in the literature by focusing on the organizational perspective rather than the individual researcher, illustrating how institutional reputations influence the handling of misconduct cases.

overview Letter

Hundreds of journals’ editorial practices captured in database

advocacy Paper

Partial lottery can make grant allocation more fair, more efficient, and more diverse

This paper advocates for the implementation of partial randomization in grant allocation processes as a solution to the shortcomings of traditional peer review. It argues that incorporating lotteries can mitigate systemic biases, increase diversity in funded projects, and improve the overall efficiency and fairness of research funding distribution.

advocacy Paper

Research funders should be more transparent: a plea for open applications

This resource makes a case for the adoption of 'open applications' in the research funding process, proposing that grant proposals, reviewer reports, and funding justifications should be made publicly accessible. It provides a call to action for funding agencies to increase their transparency and align more closely with the foundational principles of Open Science.

overview Paper

Disentangling the local context—imagined communities and researchers’ sense of belonging

This resource maps the conceptual landscape of research integrity, distinguishing between "minimal" definitions focused on misconduct and "maximal" definitions that encompass broader science ethics. It highlights how different stakeholders use language to frame integrity in varied ways, revealing subtle differences in a debate often treated as a single, universal discussion.

evidence Journal Article

Thou Shalt Not! – How the institutional afterlife of research misconduct scandals shapes research integrity training

This research evaluates the implementation of Moral Case Deliberation (MCD) as a tool for facilitating structured discussions about research integrity within diverse academic groups. It provides empirical evidence that this method, adapted from clinical ethics, can help researchers collectively navigate ethical dilemmas and foster a more open, reflective research culture.

critique Paper

Open Research Reforms and the Capitalist University: Areas of Opposition and Alignment

This paper provides a theoretical analysis of how open research reforms interact with the existing structures of academic capitalism, such as metricized performance and administrative compliance systems. It highlights potential areas of friction where reforms might inadvertently exacerbate competitive pressures rather than resolving them.

Hostler, T. (2024). Research assessment using a narrow definition of “research quality” is an act of gatekeeping: A comment on Gärtner et al. (2022). Meta-Psychology, 8. https://doi.org/10.15626/MP.2023.3764

advocacy Preprint

The Focus on Fame distorts Science

The author argues that the academic obsession with individual eminence and fame distorts scientific progress and discourages meaningful inquiry. It advocates for a fundamental shift in scientific culture toward valuing collaborative groups and the evolution of ideas rather than individual metrics.

overview Paper

A user’s guide to inflated and manipulated impact factors

This resource provides an overview of the various strategies used to artificially inflate and manipulate journal impact factors. It serves to help researchers and evaluators recognize the flaws and vulnerabilities of relying on bibliometric markers to assess scientific merit.

advocacy Preprint

Open Science as Confused: Contradictory and Conflicting Discourses in Open Science Guidance to Researchers

evidence Paper

Early coauthorship with top scientists predicts success in academic careers

Through a matched pair analysis of junior researchers across four disciplines, this study provides empirical evidence that early-career coauthorship with top scientists confers a persistent competitive advantage. It demonstrates that these collaborations significantly increase the probability of a junior researcher becoming a highly-cited scientist themselves, revealing structural biases in academic career progression.

evidence Paper

Communism, Universalism and Disinterestedness: Re-examining Contemporary Support among Academics for Merton’s Scientific Norms

This research empirically re-evaluates contemporary academic support for Merton’s classic scientific norms, such as universalism and disinterestedness. It contributes to the sociology of science by identifying how modern academic pressures and structures have shifted researcher attitudes toward these traditional ethical foundations.

advocacy Editorial

Negativity towards negative results: a discussion of the disconnect between scientific worth and scientific culture

This editorial makes a persuasive case for the importance of negative results, arguing that the current scientific culture's obsession with positive findings leads to significant publication bias. It calls for a fundamental shift in how null findings are valued and disseminated to ensure the integrity and progress of the scientific record.

critique Preprint

Adversarial reanalysis and the challenge of open data in regulatory science

overview Paper

The Matthew Effect in Science

overview Paper

The Matthew Effect in Science, II: Cumulative Advantage and the Symbolism of Intellectual Property

evidence Paper

Prestige drives epistemic inequality in the diffusion of scientific ideas

This empirical study investigates how institutional prestige and faculty hiring networks in computer science dictate the diffusion of scientific ideas. It demonstrates that the prestige of the institution where an idea originates significantly influences its spread, independent of the idea's intrinsic quality.

Naudet, F., Ioannidis, J., Miedema, F., Cristea, I. A., Goodman, S. N., & Moher, D. (2018). Six principles for assessing scientists for hiring, promotion, and tenure. Impact of Social Sciences Blog. http://eprints.lse.ac.uk/90753/

advocacy Paper

Let's PublishFewerPapers

This piece argues for a fundamental change in academic incentives by prioritizing the quality and rigor of research over the quantity of publications. It calls for the scientific community to publish fewer papers to alleviate the strain on the peer-review system and improve the overall reliability of the literature.

critique Paper

Process and Bureaucracy: Scientific Reform as Civilisation

This paper offers a critical analysis of the scientific reform movement, suggesting that its focus on standardized procedures like preregistration risks creating a restrictive bureaucracy. It argues that this drive for procedural uniformity may inadvertently suppress scientific plurality and diversity of research approaches.

practice/tools Paper

Shedding the cloak of neutrality: A guide for reflexive practices to make the sciences more inclusive and just

advocacy Preprint

Let's Look at the Big Picture: A System-Level Approach to Assessing Scholarly Merit

This commentary advocates for a paradigm shift in scholarly assessment, moving from an individualistic focus on career success metrics toward a system-level approach to merit. It argues that current incentives prioritize personal productivity over collective scientific health and proposes valuing contributions that benefit the broader research ecosystem.

advocacy Paper

Quality research needs good working conditions

policies Paper

Navigating the Science System: Research Integrity and Academic Survival Strategies

This resource introduces the Transparency and Openness Promotion (TOP) Guidelines, providing a modular framework for journals to adopt standards for transparency and reproducibility. It specifically outlines eight categories of standards with varying levels of rigor, allowing journals to progressively implement policies that mandate data sharing, preregistration, and open materials.

Roediger III, H. L. (2016). Varieties of fame in psychology. Perspectives on Psychological Science, 11(6), 882-887. https://psycnet.apa.org/doi/10.1177/1745691616662457

evidence Journal Article

Understanding the social and political dimensions of research(er) assessment: evaluative flexibility and hidden criteria in promotion processes at research institutes

advocacy Paper

Taking Advantage of Citation Measures of Scholarly Impact

This resource argues for the utility of the h-index as a robust and reliable measure of scholarly impact, suggesting it provides healthy incentives for researcher productivity and influence. It highlights the index's accessibility to non-experts and its resilience against messy or incomplete citation data.

Scheliga, K., & Friesike, S. (2014). Putting open science into practice: A social dilemma? First Monday. https://firstmonday.org/ojs/index.php/fm/article/view/5381/4110

critique Paper

Stop blaming external factors: A historical-sociological argument

This commentary critiques the view that external metric evaluations are the sole cause of failures in scientific knowledge production, arguing instead for a more complex historical-sociological understanding. It posits that while external evaluations may amplify existing problems, internal disciplinary factors are also fundamental to the current crisis situation.

critique Preprint

PPS submission "Am I Famous Yet" - Shiota

This publication examines how visibility heuristics and systemic biases create a "snowball effect" that skews metrics of merit and fame in research. It specifically addresses how these processes can disproportionately disadvantage women and minority scholars within the academic hierarchy.

overview Paper

Giving Credit Where Credit’s Due

This resource provides a review of empirical research concerning the assessment and recognition of lifetime career achievements specifically within the field of psychology. It evaluates the predictive validity and reliability of productivity and citation indicators while offering precautions for their application in professional evaluations.

advocacy Paper

“Am I Famous Yet?” Judging Scholarly Merit in Psychological Science

This paper advocates for the adoption of new criteria to judge scholarly and creative merit within psychological science research. It reviews current institutional evaluation methods and proposes an alternative standard intended to improve how individuals and institutions assess professional quality.

evidence Paper

High Impact = High Statistical Standards? Not Necessarily So

This study empirically evaluates whether high impact factor journals maintain superior statistical standards by analyzing reporting practices in top-tier medical and psychological publications. It specifically compares these prestigious journals against those with explicit editorial policies designed to mitigate the limitations of null hypothesis significance testing, finding that prestige is not a reliable proxy for statistical rigor.

evidence Journal Article

The misalignment of incentives in academic publishing and implications for journal reform

This research provides empirical data on the prevalence of scientific misconduct among biomedical scientists and its direct correlation with perceived publication pressure. It specifically quantifies how the emphasis on productivity in medical research environments contributes to unethical practices like data manipulation and falsification.

overview Website

Epistemic replicability: A primer for psychological science and beyond

evidence Letter

Publication metrics and success on the academic job market

This research provides empirical data on the relationship between quantitative publication metrics and successful outcomes in the academic hiring process. It contributes to the discussion on institutional incentives by identifying which specific markers of productivity and impact are most strongly associated with job market success.

critique Preprint

Against Eminence

This piece critiques the culture of eminence-seeking in science, arguing that the pursuit of professional status and 'sexy' findings directly undermines transparency and replicability. It calls for a fundamental shift in scientific values, prioritizing methodological openness and rigor over the individual prestige of authors or journals.

evidence Journal Article

Listing quality: Chinese journal lists in incoherent valuation regimes

This ethnographic study provides an empirical account of 'researcher discretion' by observing how decisions are made during the actual execution of research plans in specific healthcare settings. It highlights the practical complexities of researcher degrees of freedom, showing how these choices are often necessary responses to fieldwork realities rather than simple methodological lapses.

evidence Journal Article

The Journal Attention Cycle: Indicators as Assets in the Chinese Scientific Publishing Economy

This explorative study investigates how Chinese researchers navigate institutional 'blacklists' and 'whitelists' when selecting publication outlets. It identifies three interacting logics—administrative, professional, and market—that shape how scientists value journals within a complex and sometimes incoherent valuation regime.

practice/tools Paper

Opening science to society: how to progress societal engagement into (open) science policies

10 Qualitative Research

10 sub-clusters · 194 references

Invite review of: Qualitative Open Science OS Booklet.pdf Description Qualitative research refers to research conducted with non-numeric data, such as interviews, focus groups, ethnographies, photovoice, and others. There are 10 sub-clusters that can help readers understand open science from a qualitative perspective:

▸ FAIR data and materials: Choosing to share data 30 / 30

Data sharing may be a particularly useful way for researchers to increase the impact of their research. Readings in this section will highlight what data sharing can look like and why researchers and communities might benefit from these practices.

practice/tools Paper

Qualitative Data Management and Analysis within a Data Repository

This resource describes the practical application of a data repository for managing and analyzing complex qualitative data, such as photos and audio recordings, within a large nursing study. It offers a case-study perspective on how digital infrastructure can be adapted to support the unique security and collaboration needs of qualitative research teams.

overview Paper

Revisiting Qualitative Data Reuse

Bishop, L. (2005). Protecting Respondents and Enabling Data Sharing: Reply to Parry and Mauthner. Sociology, 39(2), 333–336.

overview Paper

Ethical Sharing and Reuse of Qualitative Data

This paper provides a conceptual framework for the ethical reuse of qualitative data by shifting the debate from narrow participant rights to broader philosophical ethical frameworks. It summarizes the current ethical landscape to help researchers navigate the impasse of ethical objections toward archiving and sharing qualitative materials.

practice/tools Paper

A context-consent meta-framework for designing open (qualitative) data studies

Braukmann, R. (2025). Archiving & publishing qualitative data: Repository perspective. [Workshop]. Qualitative Open Science: Challenges, Opportunities, Tensions, and Synergies (Symposium). Community of Practice for Naturally Occurring Data. https://zenodo.org/records/15095032

practice/tools Paper

Open-Science Guidance for Qualitative Research: An Empirically Validated Approach for De-Identifying Sensitive Narrative Data

This resource offers an empirically validated systematic approach for de-identifying sensitive narrative data, addressing a major barrier to data sharing in qualitative psychology. It provides specific guidance on maintaining the integrity of qualitative findings while protecting the identities of participants from marginalized or traumatized populations.

overview Paper

Open Data in Qualitative Research

practice/tools Paper

Introducing the Qualitative Data Repository's Curation Handbook

Dienlin, T., Johannes, N., Bowman, N. D., Masur, P. K., Engesser, S., Kümpel, A. S., ... & De Vreese, C. (2021). An agenda for open science in communication. Journal of Communication, 71(1), 1-26.

overview Paper

Is it time to share qualitative research data?

This paper examines the scientific, ethical, and policy implications of sharing qualitative research data, providing a comprehensive analysis of why these practices lag behind quantitative data sharing. It identifies specific advantages such as research verification and pedagogical support while addressing the practical barriers and institutional requirements involved.

Elman, C., & Kapiszewski, D. (2014). Data Access and Research Transparency in the Qualitative Tradition. PS: Political Science & Politics, 47(1), 43–47. doi:10.1017/S1049096513001777

Jarman, B. (2020). Open Data and sensitive interviews: Reflecting on ethics, consent, and reproducibility. Apollo - University of Cambridge Repository. https://doi.org/10.17863/CAM.62157

Kapiszewski, D., Karcher S., SSRC & Qualitative Data Repository. (n.d.). Managing qualitative social science data [Online course]. Social Science Research Council. https://managing-qualitative-data.org

advocacy Paper

How Data Curation Enables Epistemically Responsible Reuse of Qualitative Data

overview Paper

Rethinking Data Sharing and Human Participant Protection in Social Science Research: Applications from the Qualitative Realm

Korkiakangas, T. (2014). Challenges in archiving and sharing video data: Considering moral, pragmatic, and substantial arguments. Journal of Research Practice, 10(1), Article M3. https://discovery.ucl.ac.uk/id/eprint/10019201/1/Challenges_in_Video_Data_PRE_PRINT.pdf

Lester, J. (2017, December 11). Episode 3: The Qualitative Data Repository & Dr. Sebastian Karcher [Audio podcast episode]. In Qualitative Conversations. AERA Qualitative Special Interest Group. https://podcasts.apple.com/us/podcast/episode-3-qualitative-data-repository-dr-sebastian/id1324213425?i=1000413594733

Mannheimer, S., Pienta, A., Kirilova, D., Elman, C., & Wutich, A. (2019). Qualitative data sharing: Data repositories and academic libraries as key partners in addressing challenges. American Behavioral Scientist, 63(5), 643-664. https://pmc.ncbi.nlm.nih.gov/articles/PMC6830543

advocacy Paper

Data sharing in qualitative research: opportunities and concerns

The article advocates for the adoption of data sharing within qualitative research paradigms by highlighting its benefits for cumulative science and maximizing participant contributions. It addresses common concerns while articulating how sharing discourages fraud and enables critical scrutiny of qualitative findings.

evidence Paper

Research Participant Views regarding Qualitative Data Sharing

Piñeiro, R., & Rosenblatt, F. (2016). Pre-analysis plans for qualitative research. Revista de Ciencia Política, 36(3), 785–796. https://doi.org/10.4067/S0718-090X2016000300785

critique Paper

When open data closes the door: A critical examination of the past, present and the potential future for open data guidelines in journals

Qualitative Data Repository (n.d.). Guidance and resources. Qualitative Data Repository. https://qdr.syr.edu/guidance

Roulston, K. (2019, May 9). Archiving qualitative data [Blog post]. QualPage: Examining the world through qualitative inquiry. https://qualpage.com/2019/05/09/archiving-qualitative-data

Tamminen, K. A., Bohaker, H., Bundon, A., Gastaldo, D., Gladstone, B. M., Krmpotich, C., McDonough, M., & Smith, B. (2020). Building and supporting the use of digital research infrastructure among qualitative researchers. International Society of Qualitative Research in Sport and Exercise. https://alliancecan.ca/sites/default/files/2022-03/tamminen-et-al-ndrio-white-paper-qualitative-research.pdf

overview Paper

Promises and pitfalls of data sharing in qualitative research

evidence Paper

Qualitative Data Sharing: Participant Understanding, Motivation, and Consent

practice/tools Report

Making Qualitative Data Reusable - A Short Guidebook For Researchers And Data Stewards Working With Qualitative Data

This guidebook offers actionable guidance for researchers and data stewards on making qualitative data reusable throughout the entire research data life cycle. It includes a specialized decision tree to help researchers evaluate and select the most appropriate strategies for preserving and sharing qualitative materials.

Yardley SJ, Watts KM, Pearson J, Richardson JC. Ethical Issues in the Reuse of Qualitative Data: Perspectives From Literature, Practice, and Participants. Qualitative Health Research. 2013;24(1):102-113. doi:10.1177/1049732313518373

▸ FAIR data and materials: Ethical and legal challenges 35 / 35

Sharing qualitative data comes with its own challenges. Depending on the nature of the data, it may not be ethical to share data so that anyone can access it. This section outlines some of the common issues in sharing qualitative data and how researchers might respond to these challenges.

practice/tools Paper

Qualitative Data Management and Analysis within a Data Repository

Bishop, L. (2005). Protecting Respondents and Enabling Data Sharing: Reply to Parry and Mauthner. Sociology, 39(2), 333–336.

overview Paper

Ethical Sharing and Reuse of Qualitative Data

teaching/training Paper

Using archived qualitative data for teaching: practical and ethical considerations

This resource provides practical and ethical guidance for using archived qualitative datasets specifically as pedagogical tools in the classroom. It details necessary infrastructure, the procurement of customized teaching materials, and the symmetrical ethical responsibilities shared by researchers and educators when reusing data.

Bishop, L. (2014). Re-using qualitative data: A little evidence, on-going issues and modest reflections. Studia Socjologiczne. 3. 167-176.

Bochynska, A., et al. (2025). Supporting the adoption of open science practices by qualitative researchers [Plenary slides]. In Qualitative Open Science: Challenges, Opportunities, Tensions, and Synergies (Symposium). Community of Practice for Naturally Occurring Data. https://osf.io/xdw3m

Bochynska, A., et al. (2025). Supporting the adoption of open science practices by qualitative researchers [Video]. In Qualitative Open Science: Challenges, Opportunities, Tensions, and Synergies (Symposium). Community of Practice for Naturally Occurring Data. https://osf.io/tv8g6

Bosch, S., (2025). Supporting qualitative open science through institutional policy and infrastructure [Plenary slides]. In Qualitative Open Science: Challenges, Opportunities, Tensions, and Synergies (Symposium). Community of Practice for Naturally Occurring Data. https://osf.io/ezutr

Bosch, S., (2025). Supporting qualitative open science through institutional policy and infrastructure [Video]. In Qualitative Open Science: Challenges, Opportunities, Tensions, and Synergies (Symposium). Community of Practice for Naturally Occurring Data. https://osf.io/ajdfb

Branney, P. (2025). Open science and qualitative research: FAIR archiving, safeguarding, and stewarding of potentially identifiable qualitative data [Workshop]. Qualitative Open Science: Challenges, Opportunities, Tensions, and Synergies (Symposium). Community of Practice for Naturally Occurring Data. https://osf.io/qv8f6

overview Preprint

Reproducibility and replicability of qualitative research: an integrative review of concepts, barriers and enablers

This integrative review synthesizes diverse literature to clarify how the concepts of reproducibility and replicability are conceptualized within the specific context of qualitative research. It maps out the unique methodological barriers and enablers relevant to qualitative practitioners, providing a foundation for adapting open science principles to non-quantitative paradigms.

evidence Paper

Impact of Open Data Policies on Consent to Participate in Human Subjects Research: Discrepancies between Participant Action and Reported Concerns

This study provides empirical evidence on whether open data mandates impact participant consent rates and dropout behavior in human subjects research. The results suggest a discrepancy between participants' expressed concerns and their actual actions, indicating that such policies do not necessarily hinder recruitment.

critique Preprint

Open With Care! Consent, Context, and Co-production in Open Qualitative Research

evidence Paper

Participation in patient support forums may put rare disease patient data at risk of re-identification

Jarman, B. (2020). Open Data and sensitive interviews: Reflecting on ethics, consent, and reproducibility. Apollo - University of Cambridge Repository. https://doi.org/10.17863/CAM.62157

Joyce, J. B., & Douglass, T. (2025). Practicalities of qualitative data sharing [Workshop]. Qualitative Open Science: Challenges, Opportunities, Tensions, and Synergies (Symposium). Community of Practice for Naturally Occurring Data. https://osf.io/nrv9w

critique Paper

A dataset without a code book: ethnography and open science

critique Paper

The challenges of open data sharing for qualitative researchers

practice/tools Paper

Qualitative Data Sharing: Data Repositories and Academic Libraries as Key Partners in Addressing Challenges

advocacy Paper

Data sharing in qualitative research: opportunities and concerns

evidence Paper

Barriers and facilitators to qualitative data sharing in the United States: A survey of qualitative researchers

evidence Paper

Research Participant Views regarding Qualitative Data Sharing

evidence Paper

Research Participant Views regarding Qualitative Data Sharing

Owoyele, B. A., Schilling, M., Sawahn, R., Kaemer, N., Zherebenkov, P., Verma, B., Pouw, W., & de Melo, G. (2025, March). Masking tactics and strategies for deidentifying audio-visual data in qualitative research [Workshop]. Qualitative Open Science: Challenges, Opportunities, Tensions, and Synergies (Symposium). Community of Practice for Naturally Occurring Data. https://osf.io/apg9w

critique Paper

Reflection over compliance: Critiquing mandatory data sharing policies for qualitative research

evidence Paper

When open data closes the door: A critical examination of the past, present and the potential future for open data guidelines in journals

Through a content analysis of 261 journals, this paper provides empirical evidence of the gap between existing open data guidelines and the specific needs of qualitative social psychology. It establishes the current state of journal policies and advocates for more nuanced standards that incorporate long-standing debates regarding the ethics and practicalities of sharing qualitative data.

evidence Paper

Putting open science into practice: A social dilemma?

overview Paper

Promises and pitfalls of data sharing in qualitative research

van Ravenzwaaij, D., de Jong, M., Hoekstra, R., Scheibe, S., Span, M. M., Wessel, I., & Heininga, V. E. (2025). De-Identification When Making Data Sets Findable, Accessible, Interoperable, and Reusable (FAIR): Two Worked Examples From the Behavioral and Social Sciences. Advances in Methods and Practices in Psychological Science, 8(2), 25152459251336130.

Whylly, K. E., Karcher, S., & Renbarger, R. (2023, January 25). Data sharing for qualitative research: Webinar and panel. Center for Open Science. https://youtu.be/eWZvmSIfhQY

▸ Philosophical underpinnings of qualitative research 15 / 15

Qualitative researchers tend to approach research from a non-positivist perspective, which affects the kinds of questions qualitative researchers ask, the methodology they use, and the types of conclusions they want to draw. Thus, qualitative researchers have their own way of interacting (or not) with open science practices. These resources will help readers understand how qualitative researchers approach research, and by extension, open science.

Braun, V. & Clarke, V. (2014). Successful qualitative research: A practical guide for beginners. Sage Publications.

practice/tools Paper

Using thematic analysis in psychology

This foundational paper introduces a clear, six-phase framework for conducting thematic analysis, providing researchers with a structured approach to identifying and reporting patterns in qualitative data. It offers actionable guidance on coding and theme development, ensuring methodological rigor and theoretical consistency within psychological research and beyond.

Creswell, J. W., & Poth, C. N. (2017). Qualitative inquiry and research design: Choosing among five approaches. Sage Publications.

Mackenzie, N., & Knipe, S. (2006). Research dilemmas: Paradigms, methods and methodology. Issues in Educational Research, 16(2), 193–205. http://www.iier.org.au/iier16/mackenzie.html

overview Paper

Ready for a paradigm shift? Part 1: Introducing the philosophy of qualitative research

This resource provides an introductory overview of the philosophical foundations of qualitative research, specifically explaining ontology and epistemology to help researchers navigate the paradigm shift from positivism. It serves as a foundational guide for understanding the theoretical underpinnings that differentiate qualitative inquiry from traditional quantitative approaches.

advocacy Paper

Towards a social constructionist, criticalist, Foucauldian-informed qualitative research approach: Opportunities and challenges

This paper proposes a specialized qualitative framework that integrates social constructionism, criticalism, and Foucauldian analysis to capture the nuances of human behavior. It contributes a specific methodological stance designed to help researchers navigate the political and socio-cultural complexities inherent in studying social actions.

Pownall, M., Talbot, C. V., Henschel, A., Lautarescu, A., Lloyd, K. E., Hartmann, H., ... & Siegel, J. A. (2021). Navigating open science as early career feminist researchers. Psychology of Women Quarterly, 45(4), 526-539.

overview Book

The Oxford Handbook of Qualitative Research

This handbook offers a comprehensive survey of qualitative research, blending historical context and ethical foundations with a wide array of methodological practices. It is distinctive for its inclusion of both established techniques and newer approaches, such as arts-based and internet research, making it a versatile reference for researchers across various experience levels.

Kapiszewski, D., & Wood, E. J. (2021). Ethics, Epistemology, and Openness in Research with Human Participants. Perspectives on Politics, 20(3), 948–964. https://doi.org/10.1017/S1537592720004703

Cole, N. L., (2025). Why the Open Science movement needs qualitative researchers. [Video]. In Qualitative Open Science: Challenges, Opportunities, Tensions, and Synergies (Plenary session). Community of Practice for Naturally Occurring Data. https://osf.io/sgbvj

Cole, N. L., (2025). Why the Open Science movement needs qualitative researchers. [Plenary slides]. In Qualitative Open Science: Challenges, Opportunities, Tensions, and Synergies (Plenary session). Community of Practice for Naturally Occurring Data. https://osf.io/sgrhk

advocacy Preprint

Subjectivity is a Feature, not a Flaw: A Call to Unsilence the Human Element in Science

critique Preprint

Open With Care! Consent, Context, and Co-production in Open Qualitative Research

▸ Preregistration and Registered reports 11 / 11

Preregistration and registered reports may be useful for qualitative researchers who are hoping to confirm hypotheses. Preregistration may also be a helpful tool for reflexivity for some qualitative researchers.

Hartman, A., Kern, F. G., & Mellor, D. T. (2018). Preregistration for Qualitative Research Template. Open Science Framework. https://osf.io/j7ghv/

Haven, T., Rosenblatt, F., Pineiro, R., & Kern, F. G. (2020). Qualitative preregistration. [Blog] Center for Open Science. https://www.cos.io/blog/qualitative-preregistration

overview Paper

Preregistering qualitative research

evidence Paper

Preregistering Qualitative Research: A Delphi Study

Haven, T. L. (2020). OSCG workshop “Preregistration of Qualitative Research” [Video]. YouTube. https://www.youtube.com/watch?v=BNf9totTA_4

Center for Open Science. (n.d.). Registered Reports: Peer review before results are known to align scientific values and practices. https://www.cos.io/initiatives/registered-reports

advocacy Paper

Registered reports for qualitative research

Kern, Florian & Gleditsch, Kristian. (2017). Exploring Pre-registration and Pre-analysis Plans for Qualitative Inference. 10.13140/RG.2.2.14428.69769.

Silverstein, P., Pennington, C., Branney, P., O’Connor, D., Lawlor, E., O’Brien, E., & Lynott, D. (2024). A registered report survey of open research practices in psychology departments in the UK and Ireland. British Journal of Psychology, 115: 497-534

https://bpspsychub.onlinelibrary.wiley.com/doi/pdf/10.1111/ bjop.12700

critique Paper

Rethinking Transparency and Rigor from a Qualitative Open Science Perspective

▸ Qualitative approaches to open science 30 / 30

This section contains articles that outline what open science can look like from a qualitative approach. These articles also outline several places where there may be tensions between mainstream, quantitatively-focused open science perspectives and those often held by qualitative researchers.

critique Paper

Open Science From a Qualitative, Feminist Perspective: Epistemological Dogmas and a Call for Critical Examination

Bosch, S., (2025) Supporting qualitative open science through institutional policy and infrastructure [Plenary slides]. In Qualitative Open Science: Challenges, Opportunities, Tensions, and Synergies (Symposium). Community of Practice for Naturally Occurring Data. https://osf.io/ezutr

practice/tools Paper

Towards Open Science for the Qualitative Researcher: From a Positivist to an Open Interpretation

overview Paper

Opportunities From the Digital Revolution

This article surveys the evolution of online qualitative research, exploring the opportunities and challenges posed by the digital revolution for data collection and publishing. It specifically examines the methodological implications and emergent requirements of applying open science principles to data captured through digital sources.

advocacy Paper

An Agenda for Open Science in Communication

critique Paper

Qualitative Research Using Open Tools

This article critiques the imposition of quantitative concepts of rigor onto qualitative social research, arguing that these measures fail to meet the specific needs of qualitative methodology. It proposes an alternative framework for rigor based on ethical and relational values, including attentiveness, empathy, and reflexivity.

advocacy Preprint

Qualitative Open Science – Pain Points and Perspectives

This article addresses common ethical and practical concerns regarding open qualitative data, arguing that these challenges are manageable rather than prohibitive. It provides a framework for researchers to evaluate the cost-benefit balance of sharing qualitative data to encourage broader adoption of open science practices.

critique Preprint

Open With Care! Consent, Context, and Co-production in Open Qualitative Research

García-Álvarez, E., & López Sintas, J. (2012). Open science, e-science and the new technologies: Challenges and old problems in qualitative research in the social sciences. Intangible Capital, 8(3), 497-519. http://hdl.handle.net/2099/12934

Hocker, J., Schindler, C., & Rittberger, M. (2020). Participatory design for ontologies: a case study of an open science ontology for qualitative coding schemas. Aslib Journal of Information Management, 72(4), 671–685. https://doi.org/10.1108/AJIM-11-2019-0320

Huang, R. (2016). RQDA: R-based qualitative data analysis (Version 0.2-8) [Computer software]. R Project. http://rqda.r-forge.r-project.org/

Humphreys, L., Lewis Jr, N. A., Sender, K., & Won, A. S. (2021). Integrating qualitative methods and open science: Five principles for more trustworthy research. Journal of Communication, 71(5), 855-874.

critique Paper

Open Science and Epistemic Diversity: Friends or Foes?

overview Book

Philosophy of Open Science

This resource provides an epistemological overview of the open science movement and its impact on contemporary research. It specifically examines the tension between resource-sharing mandates and epistemic diversity, highlighting how policies can inadvertently disadvantage researchers in resource-poor environments.

advocacy Letter

Response to Sakaluk (2020): Let’s Get Serious About Including Qualitative Researchers in the Open Science Conversation

This response advocates for the active and intentional inclusion of qualitative researchers in the development of open science standards to prevent the marginalization of non-quantitative approaches. It emphasizes that meaningful engagement requires the open science movement to recognize and respect the distinct epistemological and ethical requirements of qualitative inquiry.

overview Paper

Bridging qualitative methods and open research

This piece serves as an introductory guide for researchers looking to reconcile qualitative methodologies with the core principles of the open research movement. It explores how open science concepts can be translated into qualitative contexts while addressing the specific methodological and ethical challenges inherent to the field.

Pownall, M. (2025). Incorporating open science into qualitative methods teaching [Workshop]. Qualitative Open Science: Challenges, Opportunities, Tensions, and Synergies (Symposium). Community of Practice for Naturally Occurring Data. https://osf.io/rfyju

critique Editorial

Editorial Essay: The Tumult over Transparency: Decoupling Transparency from Replication in Establishing Trustworthy Qualitative Research

practice/tools Paper

Quantity Over Quality? Reproducible Psychological Science from a Mixed Methods Perspective

This resource provides actionable guidance on implementing reproducibility standards within mixed methods research, a sub-field often overlooked by quantitative-centric open science guidelines. It draws on the authors' experience to clarify how researchers can maintain transparency and rigor when integrating qualitative and quantitative data in psychology.

Schindler, C., Veja, C., Hocker, J., Kminek, H., & Meier, M. (2020). Collaborative open analysis in a qualitative research environment. Education for Information, 36(3), 247–261. https://doi.org/10.3233/EFI-190261

Smith, B., & McGannon, K. R. (2017). Developing rigor in qualitative research: problems and opportunities within sport and exercise psychology. International Review of Sport and Exercise Psychology, 11(1), 101–121. https://doi.org/10.1080/1750984X.2017.1317357

Steinhardt, I. (2020). Learning Open Science by doing Open Science. A reflection of a qualitative research project-based seminar. Education for Information, 36(3), 263–279. https://doi.org/10.3233/EFI-190308

critique Paper

Rethinking Transparency and Rigor from a Qualitative Open Science Perspective

overview Paper

Open Science in Qualitative Evaluation: Considerations and Opportunities

This resource explores the intersection between open science principles and the specific methodology of qualitative evaluation, identifying shared goals of transparency and rigor. It outlines how evaluators can adapt practices like data and process sharing to be compatible with qualitative epistemologies.

Tamminen, A. Bundon, B. Smith, M. H. McDonough, Z. A. Poucher & M. Atkinson (2021) Considerations for making informed choices about engaging in open qualitative research, Qualitative Research in Sport, Exercise and Health, 13:5, 864-886, DOI: 10.1080/2159676X.2021.1901138

overview Paper

Open science in sport and exercise psychology: Review of current approaches and considerations for qualitative inquiry

This paper reviews the current state of open science practices within the field of sport and exercise psychology, specifically focusing on how these mandates impact qualitative inquiry. It highlights the challenges qualitative researchers face when navigating quantitative-centric requirements like preregistration and data sharing.

van Burgsteden, L. (2025). Building qualitative open science communities: Challenges and opportunities [Workshop]. Qualitative Open Science: Challenges, Opportunities, Tensions, and Synergies (Symposium). Community of Practice for Naturally Occurring Data https://osf.io/s7p9m

▸ Reflexivity and positionality 31 / 31

Reflexivity is an important practice within qualitative methods, and the critical examination of one’s position within a research study can lend itself to increased contextualization of and transparency in reporting.

critique Paper

Experimenter as automaton; experimenter as human: exploring the position of the researcher in scientific research

advocacy Preprint

Subjectivity is a Feature, not a Flaw: A Call to Unsilence the Human Element in Science

critique Preprint

Open With Care! Consent, Context, and Co-production in Open Qualitative Research

teaching/training Paper

Challenges of Critical Reflection: ‘Nothing Ventured, Nothing Gained’

evidence Paper

From understanding to insight: using reflexivity to promote students’ learning of qualitative research

overview Paper

Ethics, Reflexivity, and “Ethically Important Moments” in Research

practice/tools Paper

In search of a critical stance: Applying qualitative research practices for critical quantitative research in psychology

practice/tools Paper

Reflexivity in quantitative research: A rationale and beginner's guide

overview Paper

Doing reflexivity in psychological research: What’s the point? What’s the practice?

teaching/training Paper

Embracing the Spiral

practice/tools Paper

Shedding the cloak of neutrality: A guide for reflexive practices to make the sciences more inclusive and just

practice/tools Paper

Ethical dilemmas and reflexivity in qualitative research

critique Paper

Experimenter as automaton; experimenter as human: exploring the position of the researcher in scientific research

advocacy Preprint

Subjectivity is a Feature, not a Flaw: A Call to Unsilence the Human Element in Science

critique Preprint

Open With Care! Consent, Context, and Co-production in Open Qualitative Research

teaching/training Paper

Challenges of Critical Reflection: ‘Nothing Ventured, Nothing Gained’

evidence Paper

From understanding to insight: using reflexivity to promote students’ learning of qualitative research

overview Paper

Ethics, Reflexivity, and “Ethically Important Moments” in Research

practice/tools Paper

In search of a critical stance: Applying qualitative research practices for critical quantitative research in psychology

practice/tools Paper

Social Identity Map: A Reflexivity Tool for Practicing Explicit Positionality in Critical Qualitative Research

practice/tools Paper

Reflexivity in quantitative research: A rationale and beginner's guide

Kapiszewski, D., & Wood, E. J. (2021). Ethics, Epistemology, and Openness in Research with Human Participants. Perspectives on Politics, 20(3), 948–964. https://doi.org/10.1017/S1537592720004703

overview Paper

Doing reflexivity in psychological research: What’s the point? What’s the practice?

teaching/training Paper

Embracing the Spiral

policies Paper

POSITIONALITY STATEMENTS ARE JUST THE TIP OF THE ICEBERG: MOVING TOWARDS A REFLEXIVE PROCESS

practice/tools Paper

Ethical dilemmas and reflexivity in qualitative research

critique Paper

Rethinking Transparency and Rigor from a Qualitative Open Science Perspective

▸ Replication research 7 / 7

Replication is a somewhat controversial topic within qualitative circles. Some researchers (e.g., Makel et al., 2022) argue that replication is useful for supporting transparency and intentionality, examining transferability of findings, and evaluating connections between reflexivity and research findings. Other researchers (e.g., Pownall, 2022) argue that before someone engages in replication of qualitative studies, they should critically examine how, why, and when it would make sense to do so, given differences in epistemologies and ontologies among qualitative researchers.

evidence Paper

The (Non)Academic Community Forming around Replications: Mapping the International Open Science space via its Replication Initiatives

overview Preprint

Reproducibility and replicability of qualitative research: an integrative review of concepts, barriers and enablers

advocacy Paper

Replication is relevant to qualitative research

critique Preprint

Is replication possible for qualitative research?

This article challenges the push for replication in qualitative research by identifying fundamental philosophical and methodological incompatibilities. It argues that applying replication standards to qualitative inquiry may overlook essential differences in how knowledge is constructed in those fields.

critique Paper

Is replication possible in qualitative research? A response to Makel et al. (2022)

critique Editorial

Editorial Essay: The Tumult over Transparency: Decoupling Transparency from Replication in Establishing Trustworthy Qualitative Research

critique Paper

Rethinking Transparency and Rigor from a Qualitative Open Science Perspective

▸ Secondary data analysis 5 / 5

Secondary data analysis of quantitative methods is now prevalent and encouraged across disciplines in order to reduce costs of data collection, whereas the practice for qualitative data has been fraught with controversy that leads to concerns regarding methodological and ethical dilemmas (e.g. identity of the individual). However, depending on the positionality of the individual, it leads to more nuanced meanings that can ensure researchers can learn from one another. This section outlines some of the issues in secondary data analysis and recommendations to address these challenges.

teaching/training Paper

Using archived qualitative data for teaching: practical and ethical considerations

Bishop, L. (2014). Re-using qualitative data: A little evidence, on-going issues and modest reflections. Studia Socjologiczne. 3. 167-176.

overview Paper

Getting the most from archived qualitative data: epistemological, practical and professional obstacles

practice/tools Paper

Secondary analysis of qualitative data: a valuable method for exploring sensitive issues with an elusive population?

evidence Paper

Conducting secondary analysis of qualitative data: Should we, can we, and how?

▸ Standards and guidelines for rigorous qualitative research 20 / 20

Learn about different standards for reporting qualitative research processes and results. It may be helpful to first read Clarke’s (2022) piece on whether generic reporting standards are helpful in the first place.

practice/tools Paper

One size fits all? What counts as quality practice in (reflexive) thematic analysis?

This resource identifies and critiques common misconceptions and misapplications of reflexive thematic analysis in published literature, specifically challenging the use of universal 'coding reliability' metrics. It offers clear guidance on what constitutes high-quality, paradigmatically consistent practice within this specific qualitative framework.

practice/tools Paper

critique Paper

Determining Validity in Qualitative Inquiry

practice/tools Paper

Qualitative Research and the Question of Rigor

critique Paper

Qualitative Research Using Open Tools

practice/tools Preprint

A Transparency Checklist for Qualitative Research

Huang, R. (2016). RQDA: R-based qualitative data analysis (Version 0.2-8) [Computer software]. R Project. http://rqda.r-forge.r-project.org/

overview Paper

A Review of the Quality Indicators of Rigor in Qualitative Research

overview Paper

Journal article reporting standards for qualitative primary, qualitative meta-analytic, and mixed methods research in psychology: The APA Publications and Communications Board task force report.

Levitt, H. M. (2020). Reporting qualitative research in psychology: How to meet APA style journal article reporting standards. American Psychological Association.

overview Paper

But is it rigorous? Trustworthiness and authenticity in naturalistic evaluation

Malterud, K. (2001). Qualitative research: standards, challenges, and guidelines. The Lancet, 358(9280), 483–488. https://doi.org/10.1016/S0140-6736(01)05627-6

overview Paper

Establishing methodological rigour in international qualitative nursing research: a case study from Ghana

teaching/training Editorial

Enhancing the quality and transparency of qualitative research methods in health psychology

Groot-Sluijsmans, B., & van Acht, F. (2025). Qualitative participatory research and citizen science: A workshop on quality and ethics of community-based collaborations in research practice [Workshop]. Qualitative Open Science: Challenges, Opportunities, Tensions, and Synergies (Symposium). Community of Practice for Naturally Occurring Data. https://osf.io/xbqm3

practice/tools Paper

Valid replications require valid methods: Recommendations for best methodological practices with lab experiments.

▸ Transparency in qualitative research 10 / 10

Transparency in qualitative research tends to focus not only on transparency towards other researchers and/or funders, but also the communities with which they work. Rather than focusing on transparency insofar as it leads to reproducibility, qualitative researchers tend to focus on transparency insofar as it allows readers to understand the context under which research was done and allow them to come to their own conclusions about the extent to which research findings are logical, reliable, and generalizable.

practice/tools Paper

Integrating Qualitative Methods and Open Science: Five Principles for More Trustworthy Research*

Kapiszewski, D., & Karcher, S. (2020). Transparency in Practice in Qualitative Research. PS: Political Science & Politics, 54(2), 285–291. https://doi.org/10.1017/S1049096520000955

Kapiszewski, D., & Wood, E. J. (2021). Ethics, Epistemology, and Openness in Research with Human Participants. Perspectives on Politics, 20(3), 948–964. https://doi.org/10.1017/S1537592720004703

overview Book

Denzin, Norman

Prosser, A. M., Brown, O., Augustine, G., & Ellis, D. (2024). It’s time to join the conversation: Visions of the future for qualitative transparency and openness in management and organisation studies. SocArXiv. https://osf.io/preprints/socarxiv/ntf73

teaching/training Editorial

Enhancing the quality and transparency of qualitative research methods in health psychology

critique Paper

Rethinking Transparency and Rigor from a Qualitative Open Science Perspective

11 Research Integrity

7 sub-clusters · 159 references

Research Integrity (RI) encompasses the moral and professional standards that ensure research is trustworthy, transparent, and ethical from inception to publication. Traditionally, RI efforts have centered on preventing misconduct—the blatant fabrication, falsification, or plagiarism (FFP) that betrays the core of science. Modern perspectives, however, advocate a holistic vision: cultivating a culture of responsible, equitable, and open research practices that goes beyond avoiding misconduct to actively promoting excellence and fairness. Supervision, mentorship, and everyday collegial relations have an important part in cultivating a culture of research integrity and shaping what “good practice” looks like in labs and teams to new researchers. This includes how power dynamics, authorship, and credit are negotiated. Institutions matter too, policies, incentives, workload, and leadership either enable or erode integrity. RI is intrinsically linked with the Open Science movement. Both strive to make research more transparent and accountable, thereby strengthening credibility and public trust. Open Science initiatives (e.g. data sharing, preregistration, open access) can make misconduct easier to detect and discourage, while fostering norms of honesty and rigor. Conversely, RI provides the ethical foundation for openness – emphasizing values like honesty, accountability, respect, and fairness that guide how openness is pursued. By making research integrity “possible, easy, normative, and rewarding” (Haven et al., 2022), institutions and communities create an environment where ethical, inclusive, and rigorous research thrives. Ultimately, RI is about more than rule-following; it is about embedding integrity as a fundamental ethos of research design, conduct, and dissemination – ensuring science advances knowledge and the public good in tandem.[at]

▸ Principles and Frameworks of Research Integrity 20 / 20

This sub-cluster covers the core principles, codes, and global frameworks that define research integrity. It introduces the fundamental values (e.g. honesty, rigor, transparency, accountability, respect) that underpin responsible science. Key international statements and guidelines – such as the Singapore Statement (2010) and ALLEA’s European Code of Conduct (2017) – articulate universal norms and responsibilities for researchers. They emphasize that integrity spans all stages of research, from study design and data collection to authorship and peer review. Foundational documents (e.g. the U.S. National Academies 2017 report) situate RI in a broader context, calling for supportive research environments and institutional policies that foster ethical behavior. By studying these frameworks, one gains insight into how core values (e.g., honesty, transparency, accountability, respect) are operationalized into norms and good research practice across the research cycle. Explicitly articulated in the Netherlands Code of Conduct and agreed upon by the global research community “doing the right thing” means not only avoiding misconduct but proactively promoting openness, accountability, and social responsibility in science with clear links between specific practices and potential breaches or allegations.

ALLEA – All European Academies. (2017). The European Code of Conduct for Research Integrity (Revised Edition). – European guidance emphasizing honesty, reliability, respect, and accountability in all research practices.

advocacy Book

Handbook of Academic Integrity

This resource argues that academic institutions must take active responsibility for research reproducibility rather than leaving it solely to individual researchers. It calls for structural reforms in hiring, promotion, and training to create incentives that prioritize robust and transparent scientific practices over research volume.

overview Letter

In Defense of the Netherlands Code of Conduct for Research Integrity: Response to Radder

This volume serves as a comprehensive foundational reference for understanding the principles and practical applications of academic integrity across global research contexts. It synthesizes diverse perspectives on ethical scholarship to provide a framework for maintaining institutional and individual research standards.

critique Preprint

Open With Care! Consent, Context, and Co-production in Open Qualitative Research

critique Paper

Working with Research Integrity—Guidance for Research Performing Organisations: The Bonn PRINTEGER Statement

This publication critiques the structural "perverse incentives" and hypercompetition that characterize modern academic research. It provides a systemic analysis of how current funding models and the business-oriented evolution of higher education undermine scientific integrity and the reliability of findings.

practice/tools Paper

Guidance on research integrity provided by pan-European discipline-specific learned societies: A scoping review

This resource provides a consensus-based framework for research-performing organizations to strengthen integrity through institutional measures. It translates high-level principles into actionable organizational guidance tailored to the daily challenges faced by researchers.

evidence Paper

Designing and implementing a research integrity promotion plan: Recommendations for research funders

This study provides empirical evidence on how researchers define their "local context," showing that they identify with both their immediate institutional surroundings and their broader epistemic communities. The findings suggest that research integrity policies must account for these dual loyalties rather than focusing solely on institutional environments to be successfully implemented.

practice/tools Report

Nederlandse gedragscode wetenschappelijke integriteit

This resource offers a set of actionable recommendations for research funders to design and implement institutional Research Integrity Promotion Plans (RIPPs). It bridges the gap between high-level integrity principles and the specific administrative levers available to funding bodies.

policies Paper

Promoting Research Integrity in <scp>A</scp>frica: An African Voice of Concern on Research Misconduct and the Way Forward

advocacy Book

Fostering Integrity in Research

This publication addresses the geographical bias in research integrity literature by highlighting the lack of documented prevalence data and credible integrity systems within the African research context. It advocates for the development of context-sensitive policies and frameworks to address scientific misconduct in regions traditionally overlooked by Western-centric research integrity discussions.

overview Paper

Value pluralism in research integrity

This report provides a comprehensive examination of the modern research environment, identifying systemic influences that can either support or compromise research integrity. It updates foundational concepts of scientific misconduct and offers high-level institutional and policy recommendations to foster an ethical research culture across all scientific disciplines.

overview Paper

The Ethical Challenges of Socially Responsible Science

This resource provides a philosophical analysis of the value pluralism inherent in research codes of conduct, distinguishing between metaphysical and axiological dimensions. It categorizes the diverse epistemic, moral, professional, social, and legal norms that researchers must navigate, explaining why these standards often appear incommensurable.

policies Paper

Navigating the Science System: Research Integrity and Academic Survival Strategies

Singapore Statement on Research Integrity (2010). Singapore: 2nd World Conference on Research Integrity. – Landmark statement outlining 4 principles and 14 responsibilities for ethical research conduct.

overview Paper

Fostering integrity in research: Definitions, current knowledge, and future directions

This resource synthesizes current knowledge on research integrity, providing essential definitions and a conceptual framework for fostering ethical conduct. It identifies existing gaps in integrity research and offers strategic directions for future academic and policy developments in the field.

overview Paper

Enhancing the Taxonomies Relating to Academic Integrity and Misconduct

This resource provides a comprehensive conceptual framework for understanding research integrity by defining core terminology and synthesizing the current state of knowledge in the field. It serves as a foundational guide for navigating the complexities of ethical research conduct while identifying critical gaps for future investigation.

practice/tools Paper

Towards a Research Agenda for Promoting Responsible Research Practices

This publication presents a standardized taxonomy and glossary of terms related to academic integrity developed by the European Network for Academic Integrity. It provides a rigorous, multi-component framework designed to harmonize disparate definitions and interpretations of misconduct across different geographical and academic contexts.

UNESCO. (2017). Recommendation on Science and Scientific Researchers. Paris: UNESCO. https://unesdoc.unesco.org/ark:/48223/pf0000263618 – United Nations guidelines stressing researchers’ responsibilities to society, the need for training, and the importance of an inclusive, ethical research system globally.

UNESCO. (2023). UNESCO Recommendation on Open Science — About open science. https://www.unesco.org/en/open-science/about – Both OS and RI principles aim to make research transparent and accountable. Open Science helps deter and detect misconduct and normalizes rigor. Research Integrity provides the ethical base, guiding how openness is pursued;

World Conference on Research Integrity. (2013). Montreal Statement on Research Integrity in Cross-Boundary Research Collaborations. – Guidelines to ensure integrity, fairness, and equity in international and interdisciplinary research partnerships.

▸ Research Misconduct: Fabrication, Falsification, Plagiarism 8 / 8

This sub-cluster focuses on the “cardinal sins” of research – intentional acts that grossly violate integrity. Research misconduct is formally defined (in many policies) as fabrication (making up data or results), falsification (manipulating research processes or data), and plagiarism (using others’ ideas or words without credit). These actions undermine the very foundation of science by injecting falsehoods and eroding trust. Students explore famous misconduct cases and their repercussions, as well as studies on how frequently scientists admit to misbehavior. Surveys suggest blatant misconduct is rare but not vanishingly so (on the order of 2%–4% of researchers, depending on field, have admitted to Fabrication, Fallisfication and Plagiarism (FFP) (Bouter, 2024), with higher percentages witnessing or suspecting it in others (Bouter, 2024)). Beyond statistics, readings discuss the causes and risk factors for misconduct – e.g. extreme publication pressure or inadequate oversight – and the systems in place to detect and deter FFP (institutional investigations, whistleblower protections, and sanctions like retractions). This sub-cluster sets a cautionary foundation: understanding what not to do in research, why such behavior occurs, and how the scientific community responds when the worst breaches of integrity come to light.

advocacy Paper

Why research integrity matters and how it can be improved

This resource makes a persuasive case for the adoption of open science practices as the primary means to ensure research accountability and maintain public trust. It connects the prevalence of questionable research practices to the necessity of transparency, positioning open science as a systemic solution to issues of research misconduct.

advocacy Paper

How Many Scientists Fabricate and Falsify Research? A Systematic Review and Meta-Analysis of Survey Data

evidence Paper

Explanations of Research Misconduct, and How They Hang Together

practice/tools Paper

Organisational responses to alleged scientific misconduct: Sensemaking, sensegiving, and sensehiding

This resource offers a structured framework called the Research Integrity Promotion Plan (RIPP) specifically designed for research funders to foster ethical research practices. It provides actionable recommendations and real-world examples of how funding organizations can integrate integrity requirements into their institutional mandates and support systems.

evidence Paper

Scientists behaving badly

This publication provides empirical evidence on how universities handle allegations of scientific misconduct through a comparative analysis of cases in the Netherlands and Norway. It introduces a theoretical model highlighting the organizational processes of sensemaking, sensegiving, and sensehiding that dictate how institutions manage and communicate about integrity breaches.

evidence Paper

Scientific Misconduct and the Myth of Self-Correction in Science

evidence Paper

Repairing research integrity

This resource evaluates the efficacy of science’s self-correction mechanisms by analyzing high-profile fraud cases, particularly within social psychology. It challenges the assumption that standard peer review and replication processes are sufficient for detecting fabrication, suggesting that discovery often depends on whistleblowers rather than systemic safeguards.

evidence Paper

Prevalence of Research Misconduct and Questionable Research Practices: A Systematic Review and Meta-Analysis

▸ Questionable Research Practices and Responsible Research Practices 14 / 14

Not all integrity problems are as black-and-white as FFP. This sub-cluster examines the gray zone of Questionable Research Practices (QRPs) – behaviors that don’t blatantly falsify data, yet still deviate from good scientific practice and can undermine credibility. Examples include p-hacking (tuning analyses until results are significant), HARKing (Hypothesizing After Results are Known), selective reporting of only positive outcomes, incomplete methods reporting, inadequate data management, or sidestepping human subjects protocols. Studies have revealed alarmingly high rates of QRPs: for instance, a survey in psychology found over half of researchers admitted to at least one QRP such as selectively reporting studies or unexpected analyses - (Bouter, 2024). QRPs collectively contribute to the reproducibility crisis by inflating false-positive findings and distorting the literature - (Bouter, 2024). To address this, the research community has begun promoting Responsible Research Practices (RRPs) – methodological reforms and Open Science techniques designed to curb QRPs. These include preregistration of study plans (to prevent HARKing), sharing data and code (to increase transparency), publishing replication studies and null results, and using reporting guidelines. Embracing RRPs can make honest, thorough research the path of least resistance. In sum, this sub-cluster highlights the continuum between outright misconduct and ideal practices, stressing that everyday decisions in analysis and reporting are central to research integrity.

advocacy Book

Handbook of Academic Integrity

overview Preprint

Prevalence of questionable research practices: A survey among academic researchers in Cypriot and Greek institutions, Registered Report Stage 1

This comprehensive handbook serves as a primary reference work for the field of academic integrity, covering theoretical frameworks, historical developments, and current policy debates. It brings together diverse perspectives to define the standards of ethical conduct and the systemic factors influencing integrity in research and higher education.

evidence Paper

Prevalence of questionable research practices, research misconduct and their potential explanatory factors: A survey among academic researchers in The Netherlands

This study provides empirical data on the prevalence of research misconduct and questionable research practices specifically within the academic communities of Cyprus and Greece. It identifies regional trends in researcher behavior and explores how systemic pressures, such as publication requirements, correlate with ethical lapses.

evidence Preprint

Prevalence of responsible research practices among academics in The Netherlands

This large-scale national survey investigates the frequency of research misbehaviors among Dutch researchers across all academic ranks and disciplines. It distinguishes itself by employing the randomized response method to improve the accuracy of reporting on sensitive topics like fraud and by identifying specific explanatory factors behind these practices.

evidence Letter

Promoting trust in research and researchers: How open science and research integrity are intertwined

Shifting the empirical focus from misconduct to positive behaviors, this research quantifies the adoption of responsible research practices like open data and open code among Dutch academics. It provides valuable baseline data on the prevalence of open science behaviors and analyzes the institutional factors that facilitate or hinder their implementation.

evidence Paper

Explaining variance in perceived research misbehavior: results from a survey among academic researchers in Amsterdam

This study uses focus group methodology to identify the specific characteristics, barriers, and facilitators associated with a responsible research climate. Its contribution lies in providing a ground-up definition of research integrity culture based on the lived experiences and perceptions of researchers across various career stages.

evidence Paper

The extent and causes of academic text recycling or ‘self-plagiarism’

overview Paper

Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling

evidence Paper

With Low Power Comes Low Credibility? Toward a Principled Critique of Results From Underpowered Tests

critique Paper

Promoting an open research culture

This article provides a theoretical evaluation of the 'low-power/low-credibility' critique, arguing that it often misapplies frequentist and Bayesian principles. It challenges the common assumption that a significant result from an underpowered study is inherently less likely to be true, emphasizing that power is a property of the test design rather than a direct measure of an individual result's credibility.

policies Paper

Navigating the Science System: Research Integrity and Academic Survival Strategies

SOPs4RI. (n.d.). Dealing with breaches of research integrity. https://sops4ri.eu/tool_category/breaches/ - A curated SOPs4RI toolbox category offering practical policies and templates for handling breaches of research integrity.

evidence Paper

An Excess of Positive Results: Comparing the Standard Psychology Literature With Registered Reports

This study provides empirical evidence of publication bias by comparing the proportion of positive results in standard psychology journals versus Registered Reports. It demonstrates that Registered Reports significantly reduce the prevalence of positive results, suggesting that the format effectively mitigates selective reporting practices.

evidence Paper

False-Positive Psychology

▸ Research Culture, Incentives, and Institutional Responsibilities 37 / 37

Research integrity is not upheld by individual virtue alone – the culture and incentive structures of academia play a decisive role. This sub-cluster examines how funding, publishing, and career advancement pressures can either encourage integrity or inadvertently foster misconduct/QRPs. A “publish or perish” climate, hypercompetition for grants, and evaluation systems focused on quantity over quality (e.g. rewarding scientists for high-impact publications, citations, and impact factors) can create perverse incentives (Bouter, 2024). Such pressures may tempt researchers toward sloppy or dishonest practices to secure positive findings and prestige (Bouter, 2024). Crucially, this sub-cluster highlights reforms aimed at aligning incentives with integrity: initiatives like the San Francisco Declaration on Research Assessment (DORA) and Leiden Manifesto call for valuing rigorous methods and openness over simplistic metrics. The Hong Kong Principles (2019) advocate evaluating researchers on transparency, collaboration, and reliability rather than just output count. Institutions and funders are beginning to adopt policies (e.g. random audits, mentoring programs, or even grant lotteries) to promote a healthier research climate. By understanding these dynamics, students appreciate that sustaining RI requires top-down support: universities, journals, and funders must cultivate an environment where ethical, careful science is the most rewarded and celebrated science.

evidence Preprint

Researchers on research integrity: a survey of European and American researchers

This foundational paper uses computer simulations and experimental data to show how 'researcher degrees of freedom' in data analysis can inflate false-positive rates far beyond the standard five percent threshold. It offers a practical demonstration of how common but undisclosed flexibility in data collection and reporting makes it easy to find significant evidence for even impossible hypotheses.

evidence Paper

Robust research: Institutions must do their part for reproducibility

advocacy Book

Handbook of Academic Integrity

overview Paper

Academic Research in the 21st Century: Maintaining Scientific Integrity in a Climate of Perverse Incentives and Hypercompetition

critique Paper

Working with Research Integrity—Guidance for Research Performing Organisations: The Bonn PRINTEGER Statement

policies Paper

Interrogating the “cargo cult science” metaphor

critique Paper

Perceived publication pressure in Amsterdam: Survey of all disciplinary fields and academic ranks

evidence Paper

Personally perceived publication pressure: revising the Publication Pressure Questionnaire (PPQ) by using work stress models

practice/tools Paper

Researchers’ Perceptions of a Responsible Research Climate: A Multi Focus Group Study

evidence Paper

Explaining variance in perceived research misbehavior: results from a survey among academic researchers in Amsterdam

evidence Paper

Perceptions of research integrity climate differ between academic ranks and disciplinary fields: Results from a survey among academic researchers in Amsterdam

evidence Website

Bibliometrics: The Leiden Manifesto for research metrics

This paper reports on how perceptions of research integrity climate vary significantly across different academic disciplines and hierarchy levels. By identifying these disparities, the study provides evidence that integrity-building efforts must be tailored to the specific needs and environmental pressures of different academic subgroups.

overview Paper

Promoting Virtue or Punishing Fraud: Mapping Contrasts in the Language of ‘Scientific Integrity’

overview Paper

Disentangling the local context—imagined communities and researchers’ sense of belonging

evidence Paper

Designing and implementing a research integrity promotion plan: Recommendations for research funders

practice/tools Paper

Organisational responses to alleged scientific misconduct: Sensemaking, sensegiving, and sensehiding

evidence Paper

On the Willingness to Report and the Consequences of Reporting Research Misconduct: The Role of Power Relations

evidence Journal Article

Thou Shalt Not! – How the institutional afterlife of research misconduct scandals shapes research integrity training

evidence Journal Article

Care for the soul of science: Equity and virtue in reform and reformation

advocacy Preprint

Open Science as Confused: Contradictory and Conflicting Discourses in Open Science Guidance to Researchers

practice/tools Review Article

Important Topics for Fostering Research Integrity by Research Performing and Research Funding Organizations: A Delphi Consensus Study

This resource provides a practical guide on using co-creation methods to develop research integrity guidelines that incorporate multiple stakeholder perspectives. It outlines a step-by-step approach—addressing the 'how, what, why, and when'—to help practitioners create more inclusive, relevant, and implementable institutional policies.

evidence Journal Article

How to combine rules and commitment in fostering research integrity?

This resource details a Delphi consensus study that identifies and prioritizes the most important topics to be included in institutional research integrity policies. The resulting list provides an evidence-based foundation for research organizations and funders to develop comprehensive frameworks for fostering ethical research practices.

advocacy Letter

Improving the reproducibility and integrity of research: what can different stakeholders contribute?

This paper explores institutional governance models for research integrity, specifically addressing how to balance top-down bureaucratic rules with bottom-up researcher commitment. It advocates for the use of "network processes" and communicative action to move beyond mere compliance toward a culture of shared ethical responsibility.

overview Journal Article

Transparency in conducting and reporting research: A survey of authors, reviewers, and editors across scholarly disciplines

This resource outlines the distinct roles that various stakeholders in the research ecosystem can play to address failures in reproducibility and integrity. It specifically distinguishes between research integrity (focused on study design and robustness) and researcher integrity (focused on individual conduct) to help stakeholders target their interventions effectively.

practice/tools Journal Article

Research integrity: nine ways to move from talk to walk

This resource presents evidence-based guidelines for developing comprehensive research integrity education programs within research institutions. It provides tailored recommendations for various target groups and highlights the importance of integrating formal training with informal learning approaches to foster a culture of integrity across the organization.

practice/tools Journal Article

The Hong Kong Principles for assessing researchers: Fostering research integrity

This resource provides a set of nine actionable recommendations designed to help research institutions transition from theoretical commitments to practical implementation of research integrity. It offers a structured roadmap for organizations to foster a culture of integrity through concrete policy changes and support mechanisms.

policies Editorial

Research integrity is much more than misconduct

This resource presents the Hong Kong Principles, a formal framework designed to reform researcher assessment by rewarding behaviors that strengthen research integrity. It provides specific criteria for institutions and funders to prioritize transparency, rigor, and open science practices in their evaluation and promotion processes.

advocacy Journal Article

The San Francisco Declaration on Research Assessment

This piece argues for a paradigm shift in how research integrity is defined, moving beyond a narrow focus on blatant misconduct like fraud and plagiarism. It advocates for addressing the more common and cumulative impact of questionable research practices and suboptimal study designs that undermine the reliability of the scientific record.

evidence Journal Article

Understanding the social and political dimensions of research(er) assessment: evaluative flexibility and hidden criteria in promotion processes at research institutes

evidence Review Article

Practices for Research Integrity Promotion in Research Performing Organisations and Research Funding Organisations: A Scoping Review

This study investigates the discrepancy between formal assessment criteria and the informal, "hidden" criteria used in academic promotion processes. It provides empirical evidence on how evaluative flexibility allows social and political factors to influence researcher assessment, highlighting the complexities of implementing policy reform.

overview Journal Article

Publication Pressure and Scientific Misconduct in Medical Scientists

evidence Journal Article

The misalignment of incentives in academic publishing and implications for journal reform

critique Journal Article

Decisions, Decisions, Decisions: An Ethnographic Study of Researcher Discretion in Practice

This paper analyzes the systemic tension between the goals of knowledge dissemination and career advancement within current academic publishing structures. It critiques how commercial publishers exploit these misaligned incentives and proposes specific directions for reform to prioritize scientific integrity over publication metrics.

evidence Journal Article

Listing quality: Chinese journal lists in incoherent valuation regimes

evidence Journal Article

The Journal Attention Cycle: Indicators as Assets in the Chinese Scientific Publishing Economy

▸ Publication, Peer Review, and Research Integrity 33 / 33

This sub-cluster examines integrity issues in the dissemination phase of science – covering authorship ethics, peer review, and publication practices. It tackles questions like: Who deserves authorship and in what order? How to avoid ghost authorship (uncredited contributors) or gift/guest authorship (undeserved credit)? Students learn about authorship guidelines (e.g., International Committee of Medical Journal Editor ((ICMJE)) criteria) designed to ensure fair credit and accountability. We also discuss the role of peer review as a quality safeguard and the integrity challenges it faces – from biases in reviewer selection to cases of peer review fraud. The rise of predatory journals (which subvert quality standards for profit) is a contemporary integrity threat, potentially flooding literature with unvetted findings. Additionally, this sub-cluster emphasizes the importance of corrections and retractions as part of the self-correction mechanism of science. We explore whether increasing retraction rates signify improving vigilance or persistent problems (Bouter, 2024). Key themes include the responsibilities of journals (via ethics committees like the Committee on Publication Ethics ((COPE)) and editors in handling misconduct or honest errors, and emerging innovations like open peer review to improve transparency. By engaging with these topics, future researchers learn to navigate the publication process with integrity – ensuring proper attribution, objective review, and willingness to correct the record. Key Readings:

evidence Journal Article

Reproducibility: A tragedy of errors

This article analyzes the 'journal attention cycle' to demonstrate how bibliometric indicators are assetized and converted into financial and reputational capital. Focusing on the Chinese scientific publishing economy, it illustrates how different business models leverage measured attention to secure readership, submissions, and state support.

critique Journal Article

Generative Artificial Intelligence (GenAI) in the research process – A survey of researchers’ practices and perceptions

This resource critiques the scientific community's systemic failure to correct errors in the published record, characterizing the process as a 'tragedy of errors.' It highlights the practical barriers and institutional resistance that researchers face when attempting to rectify invalid or reproducible findings in the literature.

evidence Editorial

Predatory Conferences: Not the Meeting You Expected

This study provides large-scale survey evidence regarding the adoption and ethical perception of generative AI tools across all phases of the research process. By analyzing responses from over 2,500 researchers, it maps specific AI use cases and identifies which practices the academic community currently considers to be consistent with research integrity.

overview Report

Guidelines for retracting articles

This resource provides an introductory guide to identifying predatory conferences, detailing the common warning signs of fraudulent academic meetings. It highlights the professional and financial risks associated with these deceptive events to help researchers protect their reputations and funding.

policies Journal Article

Penetrating the Omerta of Predatory Publishing: The Romanian Connection

This document outlines standardized COPE guidelines for the retraction of scholarly articles, establishing ethical criteria for when and how papers should be removed from the record. It provides specific procedural advice for journal editors to ensure the integrity and reliability of the scientific literature.

evidence Journal Article

Why Growing Retractions Are (Mostly) a Good Sign

This resource investigates the specific networks and cultural factors contributing to predatory publishing within the Romanian academic context. It contributes empirical insights into how regional silence or lack of transparency allows deceptive publishing practices to proliferate in certain geographic areas.

advocacy Journal Article

Predatory journals: no definition, no defence

This essay argues that the rising number of research retractions is primarily an indicator of improved scientific self-correction and stronger integrity oversight rather than a simple decline in research quality. It reframes the discussion around retractions by positioning them as a positive sign of a healthy and transparent scholarly ecosystem.

policies Book Chapter

Designing journal peer review: diverse expectations, procedures and concerns

overview Review Article

Preclinical efficacy in investigator's brochures: Stakeholders' views on measures to improve completeness and robustness

This resource maps the diverse landscape of contemporary peer review models and the evolving expectations placed upon the system by different stakeholders. It examines the tensions between various quality-assurance goals and explores various innovations aimed at addressing systemic flaws in scholarly publishing.

evidence Journal Article

Open Science at the generative AI turn: An exploratory analysis of challenges and opportunities

This study identifies and analyzes the perspectives of various stakeholders on how to improve the reporting of preclinical efficacy data in investigator’s brochures. It highlights specific barriers to completeness and robustness in these documents, proposing measures to enhance the translatability of preclinical research into clinical trials.

overview Journal Article

The ghosts of HeLa: How cell line misidentification contaminates the scientific literature

evidence Review Article

The changing forms and expectations of peer review

overview Journal Article

The ability of different peer review procedures to flag problematic publications

evidence Editorial

Journal Peer Review and Editorial Evaluation: Cautious Innovator or Sleepy Giant?

evidence Journal Article

Changing peer review practices: transforming roles and future challenges

overview Letter

Hundreds of journals’ editorial practices captured in database

practice/tools Editorial

The Platform for Responsible Editorial Policies: An initiative to foster editorial transparency in scholarly publishing

practice/tools Journal Article

Pandemic publishing: Medical journals strongly speed up their publication process for COVID-19

This article introduces the Platform for Responsible Editorial Policies (PREP), a tool designed to foster transparency and responsible management in scholarly publishing. The platform serves as a practical resource for editors and researchers to document, share, and analyze editorial procedures openly.

evidence Journal Article

No time for that now! Qualitative changes in manuscript peer review during the Covid-19 pandemic

Horbach, S. P. J. M., Kalpazidou Schmidt, E., Fishberg, R., & Sørensen, M. P. (2024). Writing assistant, workhorse, or accelerator? How academics are using GenAI. Impact of Social Sciences (LSE). https://blogs.lse.ac.uk/impactofsocialsciences/2024/11/12/writing-assistant-workhorse-or-accelerator-how-academics-are-using-genai/

practice/tools Journal Article

Fighting reviewer fatigue or amplifying bias? Considerations and recommendations for use of ChatGPT and other large language models in scholarly peer review

This article examines the implications of using large language models in scholarly peer review, balancing the potential for efficiency against risks like bias and data insecurity. It offers actionable recommendations for reviewers and editors, focusing on the necessity of disclosure, transparency, and maintaining human accountability in the evaluation process.

practice/tools Review Article

A Systematic Review of Research on the Meaning, Ethics and Practices of Authorship across Scholarly Disciplines

This resource outlines ethical considerations and practical recommendations for the integration of Large Language Models (LLMs) into the scholarly peer review process. It specifically addresses concerns regarding data confidentiality and algorithmic bias while suggesting a framework for transparent disclosure when these tools are employed by reviewers and editors.

evidence Review Article

Honorary authorship is highly prevalent in health sciences: systematic review and meta-analysis of surveys

This systematic review synthesizes existing research on authorship ethics and practices to highlight the high prevalence of authorship-related issues across various academic fields. It identifies methodological gaps in the current literature and underscores the need for more rigorous, cross-disciplinary studies to ensure the integrity of publication credit allocation.

evidence Review Article

Global health collaborative research: beyond mandatory collaboration to mandatory authorship

advocacy Journal Article

Retractions are increasing, but not enough

This resource addresses authorship inequities in research collaborations between the Global North and Global South, specifically within the context of global health. It proposes moving beyond voluntary collaboration guidelines toward mandatory authorship for local researchers to ensure equitable recognition and power dynamics in collaborative projects.

advocacy Review Article

Additional experiments required: A scoping review of recent evidence on key aspects of Open Peer Review

evidence Journal Article

Open peer review urgently requires evidence: A call to action

advocacy Review Article

What is open peer review? A systematic review

overview Journal Article

Peer review: a flawed process at the heart of science and journals

This systematic review clarifies the ambiguous and often contradictory terminology surrounding Open Peer Review by identifying and categorizing its core traits and definitions. It provides a comprehensive taxonomy that serves as a conceptual framework for researchers, editors, and publishers to communicate more clearly about peer review models.

Sørensen, M. P., Horbach, S. P. J. M., Dorofeeva, O., & Schäfer Bak, M. (2024). Using generative artificial intelligence (GenAI) across different research phases: Cases, potential and risks [Report]. Danish Centre for Studies in Research and Research Policy (CFA), Aarhus University. https://tidsskrift.dk/cfasr/article/view/157222

overview Journal Article

What is it like to attend a predatory conference?

This resource provides a descriptive account of the experience of attending a predatory conference, detailing the deceptive tactics and lack of academic rigour associated with these events. It serves as an informative explainer to help researchers recognize warning signs and distinguish legitimate scholarly gatherings from fraudulent ones.

▸ Education and Training in Research Integrity 28 / 28

Fostering a culture of integrity requires education. This sub-cluster looks at how researchers are taught (and learn) responsible conduct. Many jurisdictions and funders mandate Responsible Conduct of Research (RCR) training for students and staff – covering topics from data management and mentoring to publication ethics and conflicts of interest. We evaluate the impact of these educational programs: what works, what doesn’t, and how to innovate. Early meta-analyses showed that standard ethics training has modest positive effects on knowledge and attitudes (Antes et al., 2009), especially when using interactive, case-based approaches rather than dry lectures (Antes et al., 2009). Contemporary efforts aim to go beyond simply knowing the rules – to shape researchers’ ethical decision-making skills and “virtue ethics” (instilling values like honesty and care). We discuss novel training interventions, such as engaging researchers in moral case deliberation on real dilemmas or training faculty to model and transmit RI principles (e.g. “train-the-trainer” programs for PhD supervisors (Bouter, 2024). Policy initiatives like the Cape Town Statement (2022) emphasize that RI education should be continuous, assessed, and supported by institutions, not a one-off workshop. Overall, this sub-cluster reinforces that integrity is a skill set and mindset that can be nurtured. By empowering researchers through education, the community can proactively prevent misconduct and normalize ethical best practices as the default mode of work.

overview Journal Article

A Meta-Analysis of Ethics Instruction Effectiveness in the Sciences

This article offers a first-hand account of the experience and mechanics of attending a predatory conference, illustrating how these events operate under the guise of academic legitimacy. It serves as an informative guide for researchers to help them recognize the red flags and deceptive practices associated with such fraudulent academic gatherings.

evidence Journal Article

Research rigor and reproducibility in research education: A CTSA institutional survey

This meta-analysis evaluates the effectiveness of ethics instruction programs, revealing that while overall impact is modest, certain delivery methods and content types are more effective than others. It identifies specific instructional design factors that contribute to successful ethics training, providing empirical data to help improve future research integrity curriculum.

evidence Review Article

Understanding Research Misconduct: A Comparative Analysis of 120 Cases of Professional Wrongdoing

This resource presents survey data on the implementation of rigor and reproducibility training within institutions funded by the National Center for Advancing Translational Sciences. It identifies a significant gap between the prevalence of basic training and the lack of monitoring or incentives, highlighting the need for more formal integration of these principles into translational science education.

Haven, T. L., Abunijela, S., & Hildebrand, N. (2023). Biomedical supervisors’ role modeling of open science practices. ELife, 12. CLOCKSS. https://doi.org/10.7554/eLife.83484

evidence Journal Article

Superb supervision: A pilot study on training supervisors to convey responsible research practices onto their PhD candidates

This study examines the relationship between the open science behaviors of PhD supervisors and their candidates, focusing on open access publishing and data sharing in medical research. It provides empirical evidence suggesting that supervisors act as role models who socialize early-career researchers into responsible research practices through their own professional conduct.

evidence Editorial

Editorial: Evaluating supervision and research leadership in promoting responsible research

This pilot study evaluates a three-day training program for PhD supervisors that integrates technical instruction on responsible research practices with the development of interpersonal communication skills. It provides empirical evidence on how such training can influence supervisors' perceptions and improve their ability to mentor candidates toward research integrity.

overview Journal Article

It takes two flints to start a fire: A focus group study into PhD supervision for responsible research

This editorial synthesizes recent research on measuring and implementing responsible leadership and supervision within academic environments. It clarifies various conceptual frameworks—ranging from psychometric assessments to practical roadmaps—that help clarify how responsible leadership promotes research integrity at an institutional level.

evidence Journal Article

Can moral case deliberation in research groups help to navigate research integrity dilemmas? A pilot study

This focus group study investigates how the interpersonal relationship between supervisors and PhD candidates influences the adoption of responsible research practices. By linking qualitative insights to established leadership theories, it highlights the social and relational dynamics necessary to foster an ethical research culture.

evidence Journal Article

Thou Shalt Not! – How the institutional afterlife of research misconduct scandals shapes research integrity training

evidence Journal Article

The Contribution of Moral Case Deliberation to Teaching RCR to PhD Students

This resource investigates how the collective memory and institutional legacies of research misconduct scandals influence the design and delivery of integrity training. It provides empirical insights into how institutions use past failures to shape pedagogical narratives and normalize specific ethical standards.

Kalichman, M. (2013). A brief history of RCR education. Accountability in research, 20(5-6), 380-394. doi.org/10.1080/08989621.2013.822260 – Traces the development of research ethics education from the 1980s to present. Reviews U.S. and international efforts to require training and the evolution of content, highlighting persistent challenges in assessing effectiveness and engagement.

evidence Journal Article

Effective Strategies for Research Integrity Training—a Meta-analysis

This meta-analysis evaluates the effectiveness of Responsible Conduct of Research (RCR) training by testing eleven specific hypotheses across three decades of educational studies. It identifies key moderator variables that influence learning outcomes, demonstrating that different instructional goals require specifically tailored pedagogical strategies to be effective.

practice/tools Journal Article

Rethinking the Responsible Conduct of Research (RCR) Course

This resource outlines a practical framework consisting of eleven strategies aimed at institutionalizing open science and reproducibility training within research organizations. It provides actionable guidance for stakeholders to move these practices from individual efforts to systemic, standard requirements across academia.

teaching/training Journal Article

A roadmap to good practice for training supervisors and leadership: a European perspective

This resource provides a pedagogical reflection on the limitations of current research integrity curricula and proposes new ways to structure Responsible Conduct of Research (RCR) courses. It focuses on how to redesign training to be more impactful and better aligned with the practical needs of modern researchers.

practice/tools Journal Article

Using co-creation methods for research integrity guideline development – how, what, why and when?

This publication offers a strategic roadmap for European research institutions to implement effective research integrity training for supervisors and leadership. It addresses the specific organizational challenges of these roles and provides actionable guidance on establishing good practices and assessing their effectiveness to improve institutional culture.

practice/tools Review Article

Important Topics for Fostering Research Integrity by Research Performing and Research Funding Organizations: A Delphi Consensus Study

evidence Journal Article

Co-creating Research Integrity Education Guidelines for Research Institutions

This Delphi consensus study identifies the most critical topics for fostering research integrity that should be addressed in institutional and funder policies. By synthesizing expert opinions into a prioritized list of recommendations, it provides an empirical foundation for organizations to focus their policy efforts on the factors most likely to influence researcher behavior.

practice/tools Journal Article

Research integrity: nine ways to move from talk to walk

advocacy Journal Article

Stakeholders' perspectives on research integrity training practices: a qualitative study

This article presents an influential call to action for research performing organizations to move from abstract integrity principles to concrete institutional implementation. It outlines nine specific areas for organizational change, providing a framework for institutions to 'walk the walk' by developing standard operating procedures and supportive research environments.

evidence Journal Article

How can research institutions support responsible supervision and leadership?

This qualitative study provides empirical insights into how stakeholders perceive the structure and delivery of research integrity education. It identifies specific needs regarding training objectives, organizational requirements, and pedagogical approaches to help standardize research integrity curricula.

advocacy Journal Article

Why and how to incorporate issues of race/ethnicity and gender in research integrity education

advocacy Journal Article

Teaching research integrity: a manual of good practices: an outline

The author argues for the essential inclusion of race, ethnicity, and gender issues within research integrity and responsible conduct of research (RCR) curricula. The paper makes a persuasive case for why these social dimensions are critical to modern research ethics and offers a rationale for expanding the traditional scope of training.

teaching/training Journal Article

Global Research Integrity Training

This article provides a practical manual of good practices for educators tasked with teaching research integrity. It specifically advocates for and outlines an ethics-inclusive pedagogical approach that encourages students to look beyond regulatory compliance toward the deeper ethical dimensions of scientific work.

advocacy Journal Article

A Taxonomy for Research Integrity Training: Design, Conduct, and Improvements in Research Integrity Courses

This resource calls for the globalization of research integrity training, emphasizing that such efforts must be grounded in shared professional standards. It promotes the idea that responsible conduct of research should be taught through a unified, international lens to match the global nature of contemporary science.

teaching/training Journal Article

Evaluating empowerment towards responsible conduct of research in a small private online course

This resource introduces a taxonomy designed to help educators navigate the complexities of designing and improving research integrity courses. It provides a framework for matching specific teaching activities with desired learning outcomes and target audiences to support more effective pedagogical decision-making.

van den Hoven, M., & van Loon, M. (2025). Responsible conduct of research [Online course]. Utrecht University. https://www.coursera.org/learn/research-integrity

Watts, Logan & Medeiros, Kelsey & Mulhearn, Tyler & Steele, Logan & Connelly, Shane & Mumford, Michael. (2017). Are Ethics Training Programs Improving? A Meta-Analytic Review of Past and Present Ethics Instruction in the Sciences. Ethics & Behavior. 27. 351-384. 10.1080/10508422.2016.1182025.

▸ Research Integrity, Social Responsibility, and Equity 19 / 19

This sub-cluster broadens the scope of research integrity to include the social and justice dimensions of research practice. It asks: What obligations do researchers have to society and to the communities affected by their work? Traditional RI focuses on truthfulness and accuracy, but integrity also entails conducting research responsibly with regard to its societal impact. Topics here include inclusive citation and recognition practices (avoiding exclusion or bias in credit), ensuring research agendas are not harmful or exploitative, and global equity in collaborations. For example, the Montreal Statement (2013) provides guidance for equitable partnerships, stressing respect, clarity, and fairness when researchers from high-income and low-income settings collaborate. We also explore contemporary issues like “ethics dumping” – exporting unethical research to regions with lax oversight – and the global efforts to counter it with codes of conduct (Zhaksylyk et al., 2023). Open Science intersects with equity by pushing for accessibility of knowledge and participation from diverse stakeholders (e.g. citizen science, indigenous knowledge considerations). Readings encourage reflection on how striving for integrity means striving for a research enterprise that is not only reliable but also just. This includes acknowledging and addressing structural biases (racism, sexism, colonialism) in research contexts – because an equitable, inclusive research culture is integral to truly responsible science.

ALL European Academies (ALLEA). (2021). Truth, Trust and Expertise – The Ethics of Science and Public Engagement. Report found at the bottom of page alongside six other reports – A report examining the relationship between scientific integrity and public trust. It covers ethical science communication, avoiding hype, and the duty of researchers to engage honestly with society. Highlights that maintaining public trust is an aspect of research integrity, requiring transparency and humility on the part of experts.

practice/tools Journal Article

Strengthening the Informed Consent Process in International Health Research through Community Engagement: The KEMRI-Wellcome Trust Research Programme Experience

Chatfield, Kate & Schroeder, Doris & Singh, Michelle & Chennells, Roger & herissone-kelly, peter. (2019). Equitable Research Partnerships A Global Code of Conduct to Counter Ethics Dumping. 32–34. ISBN: 978-3-030-15745-6 – Presents a Global Code of Conduct aimed at preventing researchers from exploiting vulnerable populations or regions with weaker oversight. Offers four pillars (fairness, respect, care, honesty) to guide projects in low-resource settings, aligning integrity with social justice.

advocacy Preprint

Fostering Research Integrity through the promotion of fairness, equity and diversity in research collaborations and contexts: Towards a Cape Town Statement (pre-conference discussion paper)

This discussion paper advocates for a shift in the global research integrity framework to explicitly include principles of fairness, equity, and diversity. It specifically addresses power imbalances in international collaborations and proposes a new standard for conducting research ethically in an unequal world.

advocacy Journal Article

The Cape Town Statement on fairness, equity and diversity in research

This pre-conference discussion paper makes the case for expanding the definition of research integrity to include fairness, equity, and diversity, particularly when conducting research in an unequal world. It outlines the rationale and proposed framework for the Cape Town Statement, advocating for a more socially responsible approach to global research collaborations.

policies Journal Article

Involving Research Stakeholders in Developing Policy on Sharing Public Health Research Data in Kenya

evidence Journal Article

Research Stakeholders’ Views on Benefits and Challenges for Public Health Research Data Sharing in Kenya: The Importance of Trust and Social Relations

evidence Journal Article

Engaging Communities to Strengthen Research Ethics in Low‐Income Settings: Selection and Perceptions of Members of a Network of Representatives in Coastal <scp>K</scp>enya

evidence Journal Article

Care for the soul of science: Equity and virtue in reform and reformation

advocacy Book

Philosophy of Science after Feminism

Using a historical analogy to the Christian Reformation, this article argues that the scientific reform movement must look beyond purely epistemic fixes to address its underlying moral and sociopolitical programs. It advocates for a value-prioritization that emphasizes equity and virtue to ensure the long-term legitimacy and social responsibility of science.

advocacy Letter

Gender, diversity, and the responsible assessment of researchers

advocacy Preprint

Open Science as Confused: Contradictory and Conflicting Discourses in Open Science Guidance to Researchers

evidence Journal Article

Research Integrity and Research Fairness: Harmonious or in Conflict?

Through a scoping review and critical discourse analysis of 69 international guidance documents, this study provides empirical evidence of the conflicting ways open science is defined globally. It highlights how the lack of a unified conceptualization creates contradictory instructions for researchers attempting to navigate open science requirements.

overview Journal Article

Consulting communities on feedback of genetic findings in international health research: sharing sickle cell disease and carrier information in coastal Kenya

Montreal Statement on Research Integrity in Cross-Boundary Research Collaborations (2013). – Articulates 20 principles to ensure ethical and equitable conduct in international collaborations (e.g. fair division of labor and credit, respect for local norms, benefit-sharing, transparency between partners). Emphasizes that integrity requires fairness when research transcends borders and cultures.

evidence Journal Article

What Are Fair Study Benefits in International Health Research? Consulting Community Members in Kenya

evidence Review Article

Global health collaborative research: beyond mandatory collaboration to mandatory authorship

advocacy Journal Article

Why and how to incorporate issues of race/ethnicity and gender in research integrity education

Shamoo, A. E., & Resnik, D. B. (2009). Responsible conduct of research. Oxford University Press. – Discusses challenges of maintaining RI in a global context: varying cultural norms, differing regulations, and issues like intellectual property, benefit sharing, and authorship across borders. Affirms that core RI principles are universal and must underpin international scientific collaborations and technology transfer.

FORRT's Clusters

1 Replication Crisis and Credibility Revolution

2 Conceptual and Statistical Knowledge

3 Ways of Working

4 Pre-analysis Planning

5 Transparency and reproducibility in computation and analysis

6 FAIR data and materials

7 Publication Sharing

8 Replication and meta-research

9 Academic Structures and Institutions

10 Qualitative Research

11 Research Integrity