Wishlists and Workflows

Integrating Research Transparency into Editorial and Publishing Processes

Workshop Overview and Agenda

Wednesday, August 28, 2019, Washington, DC

Agenda

Overview

There has been substantial movement towards greater research transparency across the social science disciplines. Expectations are increasing that, where possible and ethical, published research claims will be buttressed by access to the data and materials that underpin authors’ results. While the timing, nature and extent of what is to be included differs across research traditions, there is no question that in almost all traditions there has been a movement to providing more.

While a variety of institutional stakeholders have been involved in this movement, journals have been crucial facilitators for their authors to provide research transparency. There is a risk, however, that achieving greater openness might place an additional burden on journal editors. Research transparency might involve additional tasks, extend the publishing timeline, and give editors more risks to manage. Data repositories can assist journals with several of the functions that openness requires, and integrating those functions into publishing processes can lessen the burden of facilitating openness. The focus of this workshop will be how repositories can help journals as they optimize their workflows to achieve transparency.

The workshop is the fourth held at APSA in a series on "Developing and Implementing Data Policies: Conversations Between Journals and Data Repositories." The series is designed to promote discussion among social science journal editors, personnel from data repositories, data librarians, and other relevant constituencies about current approaches to data citation, management, and archiving.

As with previous events in the series, the workshop is being organized and led by various members of the Data Preservation Alliance for the Social Sciences (Data-PASS, www.data-pass.org), a consortium of social science data repositories: the Institute for Quantitative Social Science (IQSS) at Harvard University; the Howard W. Odum Institute for Research in Social Science at the University of North Carolina‐Chapel Hill; the Inter‐university Consortium for Political and Social Research (ICPSR) at the University of Michigan; Databrary, based at New York University and at Pennsylvania State University; and the Qualitative Data Repository (QDR) at Syracuse University.

Lunch
Introductions (1:00 – 1:30pm)

Slides by Colin Elman, QDR

Introductions, workshop outcomes, initial questions.

Journal Data Workflows with ICPSR (1:45 - 2:30pm)

Presentation by Jared Lyle, The Inter-university Consortium for Political and Social Research (ICPSR) – Slides

This presentation will describe ICPSR’s data workflows for journals. Authors publishing data receive an immediate, persistent citation, and can associate other related publications to the data collection. Journal editors have administrative access to view and edit submissions. Sensitive data can be shared in a restricted environment.

Video as Data and Documentaiton (2:30 - 3:15pm)

Presentation by Rick Gilmore, Databrary – Slides

Databrary.org is a data library specialized for storing and sharing video. Video has unique strengths as a form of temporally dense data about human behavior and untapped potential as a means of documenting research procedures. It also poses some unique challenges. This presentation will discuss how Databrary has met some of these challenges and discuss our ideas for addressing others.

Streamlining Data Curation and Verification Workflows (3:15 – 4:00pm)

Presentation by Thu-Mai Christian, The Howard W. Odum Institute for Research in Social Science – Slides

For over four years, the Odum Institute has executed pre-publication verification workflows for the American Journal of Political Science and the State Politics and Policy Quarterly to confirm the computational reproducibility of results published in those journals. Having performed data curation and verification processes for over 275 manuscripts, we have identified technological and educational opportunities to further streamline these workflows. This presentation will describe these opportunities and how they have been informing Odum Institute efforts to reduce the amount of time and effort required for verification.

Coffee Break (4:00 - 4:15pm)

New Reproducibility Workflows with Dataverse (4:15 – 5:00pm)

Presentation by Mercè Crosas, Harvard Dataverse – Slides

This presentation first reviews the adoption of data policies in social science journals and recent recommendations on reproducibility. It then provides a summary of new reproducibility support currently being implemented for the Dataverse repository software platform. In particular, integration between Harvard Dataverse and Code Ocean, a computational and reproducibility tool, is discussed in more detail.

Qualitative and Multi-Method Verification and Badging (5:00 – 5:45pm)

Presentation by Colin Elman and Sebastian Karcher, Qualitative Data Repository (QDR) – Slides

This presentation will provide an account of the practice of issuing open science badges, and explain the peer-reviewed and protected access variants of the badges. There will also be a discussion of pre-publication verification of the qualitative component of articles. Drawing on recent experience providing pre-publication reviews for the American Journal of Political Science (AJPS), the presentation will explain the steps that are followed in the review process, and some of the challenges that qualitative verification pose.