CRA Statement on US News and World Report Rankings of Computer Science Universities
To the Computing Research Community:
The latest US News and World Report (USN&WR) ranking of Computer Science (CS) at global universities does a grave disservice to USN&WR readers and to CS departments all over the world. Last week, we respectfully asked the ranking be withdrawn. Unfortunately USN&WR declined.
The methodology used — rankings based on journal publications collected by Web of Science — ignores conference publications and as a consequence does not accurately reflect how research is disseminated in the CS community or how faculty receive recognition or have impact. Furthermore, the list of venues is not public. So while some may debate the soundness of any bibliometric-based rankings, there will be no debate about the flaws in the rankings USN&WR has published; the methodology makes inferences from the wrong data without transparency and, consequently, it arrives at an absurd ranking.
Another important factor in USN&WR rankings is reputation. However, reputational rankings in a large, mostly disconnected community, arguably are problematic. It is unreasonable to expect that departments half-way around the world will have anything close to an accurate assessment of each other, given that they are speaking a different language, have different value systems, different levels of resources, and different goals.
Anyone with knowledge of CS research will see these rankings for what they are — nonsense — and ignore them. But others may be seriously misled.
CRA hosted a discussion which included Robert Morse, USN&WR lead on rankings, at Snowbird 2016, our biennial meeting. Among the many items discussed, there was a robust discussion of rankings using bibliometrics and the various issues involved. In particular, we outlined the problems with using a data source that does not index the conferences in which the most impactful and highly cited peer reviewed research publications appears. CRA offered to partner to create a ranking scheme that would be credible to the computing research community, but he declined to work with either CRA or the broader community. On November 1st, after the global rankings were published, we contacted Morse pointing out the problems with the methodology used and asking that these Global Rankings based on flawed source data be withdrawn, but he did not respond.
Sincerely,
Susan Davidson, Chair CRA Board of Directors, Weiss Professor, University of Pennsylvania
Andrew P. Bernat, CRA Executive Director
Carla Brodley, Dean, Northeastern University
Laura Haas, Dean, University of Massachusetts Amherst
H.V. Jagadish, Bernard A. Galler Collegiate Professor, University of Michigan
Kathryn S. McKinley, Google
Mario Nascimento, Chair and Professor, University of Alberta
Fred Schneider, Chair and Samuel B. Eckert Professor, Cornell University
