Research Transparency and Reproducibility: A Complementary Module for All

Abstract

The Berkeley Initiative for Transparency in the Social Sciences awarded me a Catalyst grant for a project aimed at advancing transparent, reproducible, and ethical research in early 2022. The purpose of my project was to create a set of teaching materials aimed at first year graduate students undertaking coursework in research methodology who have no previous experience with the subject. The aim of this module was to complement existing curricula in an existing course on research methods, in order to reach a broad and interdisciplinary audience. The hallmarks of this work involve intriguing case studies, lively storytelling, and various engagement points, with the goal of making research transparency and reproducibility interesting and accessible for students who are not familiar with the subject.

The module was developed over the course of a few years while I was a PhD student at George Washington University. It was refined and piloted among 11 students between January and April 2022 who were pursuing a Master of Public Health degree online, in a course introducing them to research methods in the social and behavioral sciences. The module comprises five slide decks with various engagement points throughout. Additional materials are provided related to a standalone group activity, an example of descriptive coding using SPSS syntax, a questionnaire used to assess various psychosocial outcomes, R script files for cleaning and analyzing the data, and an RMarkdown file for reporting the results.

The results of this informal pilot project are presented in the “Results.pdf”. The questionnaire was assessed prior to any module content being delivered, and and after all module content was delivered, approximately three months later. The results show that the module resulted in significant increases in all but one questionnaire items, across all the domains assessed - knowledge, attitudes, normative perceptions, and attitudes related to research transparency and reproducibility. Short-answer questions indicated greater engagement with research transparency and reproducibility concepts. Despite the limitations of this informal study design and assessment, the module content appears to have succeeded in generating greater interest and engagement with the concepts at post-test.

Some content from this module was used to record a segment on research transparency and reproducibility for a new set of recordings for the online course in which this module was developed. This material will launch in the Fall of 2022, and should benefit future students in this course. It is my hope that this content will also leave an indelible mark on the curriculum for this course, so that future recordings will update this material. I intend to continue iterating on this work and using it with students in my future institutions. I recommend all those interested in providing instruction in research transparency and reproducibility concepts who are unsure of where to begin look at the content from this module, and to tailor it to suit their students’ needs. I am more than happy to respond to queries on this content, and can be reached at shaonlahiri@gmail.com. I am also very curious to hear examples of other ideas and activities that you may have to bring this content to life for those that are unfamiliar with the concepts.

Please consult the README file for this repository to see a description of all the files.