Quality, not quantity: ending science's "academic prostitution"

As a graduate student, I have a particularly personal aversion to the “publish or perish” mentality that dominates the research world. I’m quite confident that most scientists—and many non-scientists—share my dissatisfaction. But, despite the grumbling, many people lack constructive ideas for turning the quantity-over-quality atmosphere into one that values strong research over a lengthy CV.

In the June issue of Nature, six contributors provided their opinions about what’s wrong with the current ranking systems used to score researchers—and how we might go about fixing them.

Most of the contributors agree that researcher rankings themselves aren’t inherently bad; instead, a major problem lies in hiring and tenure committees’ tendencies to misinterpret the metrics they are using to evaluate candidates.

For one thing, there are dozens of scientific ranking systems, including impact factors, the h-index, and page rank metrics, each with their own strengths and limitations. With only about 1,500 “scientometricians” in the world, however, it’s no wonder most of us don’t fully understand what goes into these metrics and how they should be used. On every hiring and tenure committee, it would be helpful to include someone with at least a basic understanding of scientometrics. Additionally, incorporating scientometrics into university curricula would raise awareness of the issues inherent in scientific rankings.

While metrics were originally meant to be just one factor in decision-making processes, committees often rely on rankings rather than using them as an aid. Now that rankings have become so prominent, researchers are indirectly encouraged to please referees and institutions via “academic prostitution,” as one of the essays terms it. Scientists are submitting ever-smaller bits of research to the most prominent journals to increase their appeal to evaluators. One commentator suggests looking at this positive feedback loop as a game theoretic problem and using that to generate novel solutions.

One particularly interesting suggestion is to have journals publish lists of rejected papers along with the accepted research. This would act as a sort of public punishment and might encourage scientists to submit their research to appropriate journals on the first try. Another method to decrease the science community’s focus on metrics would be to allow tenure candidates to submit just their top few papers for evaluation.

One commentator notes that, unlike in most fields, science output is not directly proportional to effort. Instead of ranking scientists purely on their publication records, credit should be given for all sorts of activities that don’t necessarily come across in traditional metrics. Establishing publicly available datasets, serving on committees, and developing new experimental set-ups should all be taken into account in hiring and tenure decisions. More journals should follow the leads of PLoS ONE and The Journal of Negative Results and begin publishing negative or inconclusive findings.

Understanding metrics, reducing reliance on rankings, and suggesting new ways to evaluate scientists are only the beginning; it’s going to take a sea change and lots of cooperation among scientists, journals, and academic and government institutions to banish the “publish or perish” mentality.

Nature, 2010. DOI: 10.1038/465870a (About DOIs).