This article is published in the August 2020 issue.

CCC Releases Evolving Methods for Evaluating and Disseminating Computing Research White Paper

By: CCC StaffIn: /

Evolving Methods for Evaluating and Disseminating Computing Research coverThe Computing Community Consortium (CCC) has released a white paper titled Evolving Methods for Evaluating and Disseminating Computing Research. This white paper was written by Future of the Research Enterprise (FRE) task force members Ben Zorn (Microsoft Research), Tom Conte (Georgia Tech), Keith Marzullo (University of Maryland), and Suresh Venkatasubramanian (University of Utah). 

The FRE task force was assembled in 2019 to study the evolution of the computing research ecosystem, including topics such as the impact of academia-industry relations, the peer review process, and the future of open source projects. Based on interviews with members of the computing research community, the task force wrote the Evolving Methods for Evaluating and Disseminating Computing Research white paper, which presents the trends the task force observed, discusses the impacts of changing review and dissemination processes, and suggests methods to reduce the negative impacts of these trends. 

Overall the task force found that “Trends impacting computing research are largely positive and have increased the participation, scope, accessibility, and speed of the research process,” however “Challenges remain in securing the integrity of the process, including addressing ways to scale the review process, avoiding attempts to misinform or confuse the dissemination of results, and ensuring fairness and broad participation in the process itself.” (p. 1). 

In response to these trends and challenges, the task force recommends: 

  • “Regularly polling members of the computing research community, including program and general conference chairs, journal editors, authors, reviewers, etc., to identify specific challenges they face to better understand these issues.
  • An influential body, such as the Computing Research Association (CRA), regularly issues a “State of the Computing Research Enterprise” report to update the community on trends, both positive and negative, impacting the computing research enterprise.
  • A deeper investigation, specifically to better understand the influence that social media and preprint archives have on computing research, is conducted.
  • Initiate an investigation of the impact of COVID-19 on the broader computing research enterprise, including the impact on evaluation and dissemination.” (pp. 1-2).

Learn more about the evolving trends in computing research and read the recommendations of the FRE task force in the white paper, which can be accessed here. The white paper is also available on arXiv. For citation use: Zorn B., Conte T., Marzullo K., & Venkatasubramanian S. (2020) Evolving Methods for Evaluating and Disseminating Computing Research https://cra.org/ccc/resources/ccc-led-whitepapers/ or arXiv identifier: arXiv:2007.01242 [cs.CY].