CRA Best Practice Memos
The Computing Research Association (CRA) produces resources that standardize and establish best practices in the field. Most recently, last spring the CRA Board of Directors released its latest Best Practices Memo, “Incentivizing Quality and Impact: Evaluating Scholarship in Hiring, Tenure, and Promotion.” Distinguishing between quality and quantity is key to promoting the future growth of the computing and information field. The memo advocates adjustments to hiring, promotion, and tenure practices as well as to the publication culture. Below is a summary of the reports main points. Click here to download the full memo.
The recommendations in the report were developed over an 18-month period by the CRA Committee on Best Practices for Hiring, Promotion, and Scholarship, led by Fred B. Schneider (Chair) and Batya Friedman (Co-Chair). The committee conducted interviews in autumn 2013 with more than 75 academic and industry computing and information unit heads to understand the issues and gain insights from practice. Preliminary recommendations were vetted with department chairs and CRA Deans at the CRA Conference at Snowbird in July 2014.
“Above all, quality and impact need to be incentivized over quantity…What ultimately should matter when it comes to hiring, tenure, and promotion is the quality of the research.”
The memo provides recommendations for both hiring and tenure and promotion cases.
Hiring Recommendation. Evaluate candidates on the basis of the contributions in their top one or two publications, in concert with the research statement and the other standard material (e.g., letters of recommendation, full CV, teaching statement) generally read by hiring committees in determining whom to invite to campus for an interview and, ultimately, whom to hire. Candidates should identify publications where they have played a significant role.
Tenure and Promotion Recommendation. Evaluate candidates for tenure and promotion on the basis of the contributions in their most important three to five publications (where systems and other artifacts may be included).¹ Tenure and promotion committees should invite external reviewers to comment on impact, depth, and scholarship of these publications or artifacts as well as the standard material (e.g., full CV, research statement, teaching statement). Some institutions might ask a candidate to suggest which publications or artifacts be considered, other institutions might leave that determination to the external reviewers. Per standard practice, tenure and promotion committees should read the external letters and the standard material in determining tenure and promotion decisions.
Implementing these recommendations will require attention to the transition for young researchers. Annual or reappointment reviews (which often occur after three years of hiring) should reflect the emphasis on quality—not quantity—and should recognize that high caliber research activities may take two or three years to come to fruition (e.g., publication or artifact deployment) and even longer for the impact to become apparent. A corollary follows: Evaluation of senior faculty similarly should emphasize quality over quantity, with incentives for pursuing greater risk-taking in research activities.
This memo was endorsed by Informatics Europe and highlighted by Moshe Vardi in Communications of the ACM.
In addition to this memo, CRA has also produced a Best Practices Guide for Postdocs, providing guidance to graduate students, postdocs, advisors and mentors, and institutions on how to develop positive postdoctoral experiences within computer science and engineering.
Take a minute to check out all of our best practice memos found here: https://cra.org/resources/best-practice-memos/.
[1] For some disciplines represented in Information Schools (e.g., philosophy), the publication outcome is a book, with the expectation that one book would be in press or published at the time of evaluation for tenure.
