In my experience, the publication pressure in today's science is to large extent inhibiting innovation. How can you innovate when you need to have X papers every year, otherwise you will not get that position of funding. To fulfill the quota, the only rational strategy is to focus on simple iterative papers that are very similar to what everybody else is doing. There is simply no time to innovate or be brave, you have to comfort. There is also barely time to make sure they what you are doing is actually methodologically correct. If you spend too much time, you will get scooped and forgotten.
Case in point, everybody is doing AI research nowadays and NIPS has like 15k submitted papers. But the innovation rate in AI is actually not that much higher than 10 years ago, I would even argue that it is lower. What are all these papers for? They help people build their careers as proofs of work.
kevinventullo · 2h ago
Follow-up papers by other authors which “only extend or expand on the specific finding in very minor ways” have a secondary benefit. In addition to expanding the original findings, they are also implicitly replicating the original result. This is perhaps a crucial contribution in light of the replication crisis!
Daub · 1h ago
Maybe. But that is a generous reading. I used to attend many computational aesthetic conferences. The sheer volume of non photorealistic rendering cross hatch algorithms was almost laughable.
Daub · 2h ago
For a few years I worked closely with computer engineers in a S E Asian university. I got to know quite well the sort of stuff they published. Some of the dodgy stuff i saw:
Recycling. Some papers seemed to be near duplicates of prior work by the same academic, with minor modification.
Faddishnes. Papers featuring the latest buzz technologies regardless of whether they were appropriate.
Questionable authorship. Some senior academics would get their name included on publications regardless of whether they had been actively engaged with that project. I saw a few academics get involved in risky and potentially interesting subjects, but they all risked their careers in doing so.
But most of all, there was a dearth of true innovation. The university noticed this and established an Innovation Centre. It quickly became full of second hand projects all frustratingly similar to projects in the US from a few years ago.
Of course there were exceptions, and learning from them was a genuine growth experience fir which I am grateful.
bonoboTP · 45m ago
It's not just about the academics but the expectations from higher ups and funding agencies in order to keep your job and have a chance at continuing your career. Over the last few decades the expected amount of papers at good and even mediocre institutions has exploded. Profs who want to be seen as productive and who want good funding publish 30-50 papers per year and sometimes "supervise" dozens of PhD students at the same time (who agree to the deal to get the brand name of the big prof, not for any real supervision).
Funding agencies can't evaluate the research itself, so they look at numbers, metrics, impact factors, citations, h-index, publication count etc. They can't simply say "we pay this academic whether he publishes or not because we trust he is still deep in important work when he is not at a work stage to publish" because people will suspect fraud and nepotism and bias, and often the funding is taxpayer money. Not that the metrics prevent that of course. But it seems that way. So metrics it is, so gaming the metrics via Goodhart's law it is.
I don't think it's super bad, but it increases administrative work and busywork overhead on top of the actual research. The progress slows somewhat per person, as the same work has to be salami sliced and marketed in chunks, but there's also way more people in it, but of course most of them produce vary low quality stuff but it's not a big loss because these people would not even have published anything some decades ago, they would just have some teaching professorship and publish every few years perhaps just in their national language. It increases the noise but there are ways to find the signal among it, and academics figure out ways to cut through the noise. It's not great, not super easy, and it pushes a lot of people out who dislike the grind but there are plenty who see it as a relatively good deal to move to a richer country and do this.
birn559 · 4h ago
The process is far from perfect, but it works well enough mid-term and works pretty well long-term.
It's also better than any alternatives, as far as I know. Haven't heard people pushing the idea of restructuring the process, the only exception being that journals shouldn't cost (that much) money and instead institutions should pay for publishing a paper. This wouldn't however change the foundation of the process.
agumonkey · 2h ago
What about the publish or perish effect ? no ideas on how to rebalance things to avoid it ?
Daub · 1h ago
Yes. It's simple. Establish peer review as the metric of tenure and make dam sure those peers know their stuff.
friendzis · 1h ago
> Establish peer review as the metric of tenure and make dam sure those peers know their stuff.
And you are back at square one: peer reviews become the currency used in academic politics. A relatively small group of tenured academics have all the incentives to independently form a fiefdom. Anonymization does not help as everyone knows work and papers of the rest anyway.
Case in point, everybody is doing AI research nowadays and NIPS has like 15k submitted papers. But the innovation rate in AI is actually not that much higher than 10 years ago, I would even argue that it is lower. What are all these papers for? They help people build their careers as proofs of work.
Recycling. Some papers seemed to be near duplicates of prior work by the same academic, with minor modification.
Faddishnes. Papers featuring the latest buzz technologies regardless of whether they were appropriate.
Questionable authorship. Some senior academics would get their name included on publications regardless of whether they had been actively engaged with that project. I saw a few academics get involved in risky and potentially interesting subjects, but they all risked their careers in doing so.
But most of all, there was a dearth of true innovation. The university noticed this and established an Innovation Centre. It quickly became full of second hand projects all frustratingly similar to projects in the US from a few years ago.
Of course there were exceptions, and learning from them was a genuine growth experience fir which I am grateful.
Funding agencies can't evaluate the research itself, so they look at numbers, metrics, impact factors, citations, h-index, publication count etc. They can't simply say "we pay this academic whether he publishes or not because we trust he is still deep in important work when he is not at a work stage to publish" because people will suspect fraud and nepotism and bias, and often the funding is taxpayer money. Not that the metrics prevent that of course. But it seems that way. So metrics it is, so gaming the metrics via Goodhart's law it is.
I don't think it's super bad, but it increases administrative work and busywork overhead on top of the actual research. The progress slows somewhat per person, as the same work has to be salami sliced and marketed in chunks, but there's also way more people in it, but of course most of them produce vary low quality stuff but it's not a big loss because these people would not even have published anything some decades ago, they would just have some teaching professorship and publish every few years perhaps just in their national language. It increases the noise but there are ways to find the signal among it, and academics figure out ways to cut through the noise. It's not great, not super easy, and it pushes a lot of people out who dislike the grind but there are plenty who see it as a relatively good deal to move to a richer country and do this.
It's also better than any alternatives, as far as I know. Haven't heard people pushing the idea of restructuring the process, the only exception being that journals shouldn't cost (that much) money and instead institutions should pay for publishing a paper. This wouldn't however change the foundation of the process.
And you are back at square one: peer reviews become the currency used in academic politics. A relatively small group of tenured academics have all the incentives to independently form a fiefdom. Anonymization does not help as everyone knows work and papers of the rest anyway.