Publishing Quarks: Considering Our Culture

Musings from the Chair

Over the past thirty years, I have accumulated the common artifact of an academic research career—bookshelves overflowing with research journals and conference proceedings. Each time I pull an old and yellowing volume from my shelves, it is simultaneously nostalgic and thought-provoking to read a few randomly selected articles. Not only does this stroll down memory lane illuminate how far we have come, both technologically and theoretically; it also shows how profoundly the publication culture of our field has changed.

Conference Hegemony

Not that many years ago, CRA published a “best practices” memorandum entitled, “Evaluating Computer Scientists and Engineers for Promotion and Tenure” (available at https://cra.org/resources/best-practice-memos/evaluating-computer-scientists-and-engineers-for-promotion-and-tenure/). At a time when many departments were struggling to make the case to their science and engineering colleagues that conference publications mattered, this memorandum demonstrated that computing conference publications were of a quality comparable to those in archival journals.

The perception battle won, is all right with our publication world? Perhaps, but I suspect not. Our prestigious conferences have become the moral equivalent of highly selective journals in other fields. The computing conference review process is rigorous and highly selective, and polished results are required for publication. In many of our sub-disciplines, the conference paper is the final result. There is no expectation that the preliminary results will be expanded, augmented and published in a journal. Consequently, many—arguably most—of our journals have receded in significance. I believe this is a regrettable and worrisome development.

First, it has truncated the continuum of publication options. In most disciplines, conferences are the venue where late-breaking results, thought-providing theories and controversial ideas are aired and debated. Many of these later are proven incorrect or validated and expanded with additional data, but the free exchange of ideas stimulates research and innovation. At the risk of sounding like an “old geezer,” I encourage you to read some old conference proceedings. It is illuminating to see how many of our conferences have evolved from idea exchanges to publication venues.

Our emphasis on the conference cycle has also encouraged and rewarded production of publishing “quarks”—units of intellectual endeavor that can be generated, summarized and reviewed in a calendar year. We now see new faculty and research staff candidates with more publications than were once common in promotion and tenure dossiers.

Do not misunderstand. I am not suggesting that our current conference-centric culture is all bad, merely that we should be more thoughtful regarding the time scales and range of our publication options. I would also humbly suggest that we consider how this approach shapes the types and kinds of research conducted. We all know that quality trumps quantity, and that research results have a wide range of natural sizes and time scales.

Journal Revival

What then becomes of our often languishing journals? Are they a hidebound and archaic notion, doomed to irrelevance by ubiquitous electronic access? To be sure, the nature of publication is in flux in both popular and professional culture, with the physical artifacts likely to disappear. However, the notions of scholarly review and archival recording of research are independent of these artifacts.

I believe we need to restore journals to their rightful place as the lasting archives of scientific knowledge. This will require a cultural shift, making our conferences the harbingers of extended, rigorous publication in journals. Equally importantly, it will require us to review those journal submissions thoughtfully and with alacrity.

As anyone who has ever been the editor of a computing journal knows, obtaining timely reviews is challenging. Even with gentle (and sometimes not so gentle) nagging, the weeks can stretch to months; the months sometimes turn to years. Contrast this with other technical disciplines where submissions can be reviewed and published in weeks or months. Is it any wonder that paper authors in our field eschew journals for conferences with known publication dates?

As a discipline, we benefit from the entire continuum of venues for communicating research ideas and results, from informal workshops and conferences to research surveys and expanded publication in archival journals. Let’s recognize and embrace the distinct and important roles that each plays in the free and fruitful exchange of research ideas.

Dan Reed, CRA’s Board Chair, is Microsoft’s Scalable and Multicore Computing Strategist. Contact him at Daniel.Reed @ microsoft.com or his blog at www.hpcdan.org