Journalology #138: A little bit exaggerated
Hello fellow journalologists,
This week I have a poll AND a quiz for you. I’ll eat my hat if someone gets the quiz question right. Anyone can take part in the poll, though (and I hope you will).
News headlines
The ‘News headlines’ section includes snippets from newspapers and magazines that are likely to have broad appeal. This section is ‘free to read’ by everyone who subscribes to the Journalology newsletter.
Reproducibility crisis ‘a little bit exaggerated’, suggests ERC chief
The very fact that replication studies were conducted and that potentially unreliable findings were highlighted illustrates that science “autocorrects”, [Maria] Leptin emphasised. Researchers who, in the worst cases, try to cheat the system “are going to be discovered”, she said. Open science practices, in which methods and outcomes are shared for scrutiny and to facilitate replication studies, are “essential” so that further work can “find flaws”, she continued. Leptin also suggested that the “most important” way to shore up trust in science is for the public and policymakers to better understand the scientific method and its self-correcting nature.
JB: ERC = European Research Council, which will dish out €2.6 billion in funding this year as part of the Horizon Europe programme (total budget = €13 billion). Professor Maria Leptin, a well known biologist, is the head of the ERC. She poses an important, but potentially contentious, question. Let’s put it the vote:
Journal retracts nearly 150 articles for compromised peer review
The American Society For Testing And Materials (ASTM) International started an investigation into its Journal of Testing and Evaluation after an ASTM vendor noticed some “irregular patterns in the peer review” in a special issue, spokesperson Gavin O’Reilly told Retraction Watch. When the publisher confirmed those patterns, ASTM decided to investigate several related issues, he said. The investigation revealed the peer review process in the special sections or issues had been compromised, each of the retraction notices says.
JB: ASTM International publishes five journals, three of which published less than 20 research or review articles last year. The publisher’s article output increased rapidly in 2018 (to over 800 articles) and then fell precipitously in 2020 (to 400 articles). Last year ASTM International published less than 200 articles.
Retraction Watch notes that in 2024 the society generated nearly $82 million from “publications sales”. Journals are likely to be a tiny part of that total. Lets do the maths: even if they were generating $10,000 in revenue per article, that would only be $2 million of the $82 million total. ASTM likely generates most of its publishing revenues from ‘standards’.
Critical social media posts linked to retractions of scientific papers
Zheng and his colleagues examined thousands of tweets that referenced articles that went on to be retracted and articles that didn’t. Of the 604 studies that went on to be retracted, the researchers found that 8.3% had at least one critical post on X before retraction, compared with only 1.5% of articles that were not retracted. Zheng and his colleagues defined critical tweets as posts that contained sarcasm, criticism, accusations or doubt about the article. They say that nearly 1 in 12 of the retracted articles could have been flagged to publishers for greater scrutiny on the basis of the social-media posts.
JB: 1 in 12 doesn’t sound like a particularly strong signal to me.
ArXiv preprint server clamps down on AI slop
Until now, someone wanting to submit to arXiv for the first time only needed an email address affiliated with a reputable academic or research institution, such as a university. But a rule instituted on 21 January now requires first-time posters to be endorsed by an established arXiv author in their own field. People who have previously posted in the same disciplinary section of arXiv do not need an endorsement. The move is an attempt to clamp down on a rising tide of fraudulent submissions, says University of Amsterdam astronomer Ralph Wijers, chair of the arXiv editorial council. A large fraction, he says, are generated with artificial intelligence (AI). The new rule is “mostly to try and discourage very junior, unskilled people from trying to get something started by sending some rubbish to arXiv,” he says.
According to the 2024 annual report, the total budget for arXiv was around $7m. The majority of that was for personnel costs ($3.5m) and indirect costs covered by Cornell (for HR, building costs etc; $2.3m).
Publishing companies are investing millions of dollars into research integrity tools and teams to try to protect the academic record; arXiv doesn’t have the same access to those funds and so has no choice but to resort to blunt measures like this one, which makes arXiv feel a bit like an old boys’ club.
A related story ran this week: ArXiv says submissions must be in English: are AI translators up for the job?
More news and analysis continues after the subscription fold. Paid subscribers will learn about the latest developments in publishing integrity, AI, peer review, open access and much more. You can either subscribe individually or as a group.
