Publication is a central means by which scholars establish their reputation. In consequence, how publications are vetted has significant impact on academic careers.
The Center for Studies in Higher Education at UC Berkeley has extensively surveyed the imperatives of scholarly publication in several disciplines. You could see a current overview at their web site.
Three of their reports from which to start are:
- Assessing the Future Landscape of Scholarly Communication: An Exploration of Faculty Values and Needs in Seven Disciplines by Diane Harley, Sophia Krzys Acord, Sarah Earl-Novell, Shanon Lawrence, and C. Judson King, Jan 1, 2010.
- Peer Review in Academic Promotion and Publishing: Its Meaning, Locus, and Future by Diane Harley, Sophia Krzys Acord, March 4, 2011.
- Scholarly Communication: Academic Values and Sustainable Models C. Judson King, Diane Harley, Sarah Earl-Novell, Jennifer Arter, Shannon Lawrence, and Irene Perciali, July 27, 2006.
The CRA itself sponsored a “Best Practices Memo” on Evaluating Computer Scientists and Engineers For Promotion and Tenure, Computing Research News, Sep. 1999. A central point made in this report is that conference publication is both rigorous and prestigious in many sub-areas of Computer Science, and unlike most other disciplines.
Bibliometrics, including citation counts in particular, play a major role in assessing the reputation of a scholar. Some thoughtful pieces on the appropriate role for these are:
- A Report on Citation Statistics, from the International Mathematical Union, June 2008.
- An editorial, Fraud, the h-index, and Pasternak, by Nicholas Kotov, ACSNano 4(2):585-586, 2010.”
- Developing Bibliometric Indicators of Research Performance in Computer Science: An Exploratory Study, by Henk F. Moed and Martijn S. Visser. Research Report to the Council for Physical Sciences of the Netherlands Organisation for Scientific Research (NWO).
- Bibliometrics Pilot Exercise (Multiple Reports) Higher Education Funding Council for England.