Journalology #139: Kind of Blue
Hello fellow journalologists,
I generally listen to music while I’m writing. This week I’ve had Miles Davis’ Kind of Blue on repeat. ‘So What?’, you may ask. The reasons relate to “clarity of purpose, depth of thinking, and new horizons”. More on that later in the newsletter.
I’ll be attending the Researcher to Reader conference in London the week after next (Feb 24-25). There are still tickets available, I understand. You can find out more here. I hope to see some of you there.
News headlines
The ‘News headlines’ section includes snippets from news outlets that are likely to have broad appeal. This section is ‘free to read’ by everyone who subscribes to the Journalology newsletter.
Open-source AI tool beats giant LLMs in literature reviews — and gets citations right
Researchers have published the recipe for an artificial-intelligence model that reviews the scientific literature better than some major large language models (LLMs) are able to, and gets the citations correct as often as human experts do. OpenScholar — which combines a language model with a database of 45 million open-access articles — links the information it sources directly back to the literature, to stop the system from making up or ‘hallucinating’ citations.
JB: The black-box nature of commercially available LLMs makes me deeply uneasy. OpenScholar could be the start of a solution. The big downside is that it only has access to open access articles; paywalled articles are not included in the dataset.
The researchers, who work at the University of Washington and the Allen Institute for AI, also developed a way of testing LLMs in a scholarly context:
ScholarQABench, a new large-scale benchmark, provides a standardized way to evaluate literature review automation across several scientific domains.
The authors provide data on how bad the hallucination problem is for GPT-4o:
Although GPT-4o hallucinates citations 78–90% of the time, OpenScholar achieves citation accuracy on par with human experts.
The Nature paper is an easy read and is worth spending some time with: Synthesizing scientific literature with retrieval-augmented language models.
Mega-journal Heliyon retracts hundreds of papers after internal audit
Heliyon has published fewer papers and ramped up its retractions since a major indexing service put the journal on hold and the publisher launched an audit of all papers published in the journal since its launch in 2016. Clarivate put Heliyon on hold in September 2024, citing concerns about the quality of the content. The “on-hold” status indicates a journal is being re-evaluated, and new content isn’t indexed, according to documentation on the Clarivate website. If Clarivate removes the hold by August 1, the journal will receive an impact factor this year. If not, its impact factor will be listed as “forthcoming,” a spokesperson for Clarivate told us.
JB: Heliyon was not originally part of Cell Press. It was moved there in around 2019 (from memory) to help it to grow after a sluggish start. The strategy worked. Heliyon published 4000 articles in 2022, increasing to 17,000 by 2024. Last year its output dropped to 3000 articles, possibly because it was put ‘on hold’ by Web of Science (but I suspect also because Elsevier hit the brakes — the drop in output was incredibly fast; I wrote about it here).
Last year Heliyon retracted 392 articles. In the first month of 2026 it retracted a further 144 articles. That’s a small proportion of its total output since it launched in 2016, but it’s not an ideal outcome for one of Elsevier’s flagship brands. It’s unclear if further retractions are to come or whether Heliyon will keep it’s impact factor. Retraction Watch notes:
A spokesperson for Clarivate told us they couldn’t comment on specific journals, but said a journal must be both taken off hold and have its missing content backfilled by August 1 in order to receive an impact factor for that year. If a journal is still on hold and content hasn’t been backfilled, the journal will not receive an impact factor, the spokesperson said.
I wouldn’t be surprised if Elsevier quietly closes Heliyon in the future or at the very least moves it out of Cell Press. After all, the Cell Press portfolio also includes iScience, which has an impact factor of 4.1 and could, in theory at least, compete with the likes of Scientific Reports and PLOS One.
Academic publishers defeat lawsuit over ‘peer review’ pay, other restrictions
A group of major academic publishers convinced a judge in New York to dismiss a lawsuit accusing them of thwarting competition by barring scholars from submitting papers to multiple journals simultaneously and denying pay for “peer review” services. In his ruling, opens new tab on Friday, U.S. District Judge Hector Gonzalez in the federal court in Brooklyn said the four scholars, scientists and professors who filed the lawsuit in 2024 had not shown sufficient evidence of a conspiracy involving publishers Elsevier, John Wiley & Sons, Sage Publications, Springer Nature, Taylor & Francis and Wolters Kluwer.
JB: Common sense prevails. The only winners in this class action lawsuit are the lawyers and their bank managers. You can read the ruling here.
Journal impact factors still exert ‘undue influence’, finds PLOS study
But a survey of almost 500 biology researchers who have served on either grant review committees or university hiring and promotion panels in the past two years found most respondents still used this and other “extrinsic proxies” to assess not just the strength of an applicant’s publication record but the trustworthiness of their outputs. In the study, published by the open access title PeerJ, 57 per cent of respondents say they normally use at least one of three indicators – journal reputation, lab reputation or JIF – to evaluate whether or not “research is credible”. Journal reputation was the most trusted “extrinsic proxy” when evaluating research credibility (it was used by 48 per cent of respondents) and 43 per cent used it to decide whether research was trustworthy or reliable, says the study, written by PLOS editors.
JB: This paper was published in PeerJ a few weeks ago; the authors work at PLOS. The findings are unsurprising in many ways. Humans have used brands and perceived reputation — for people and for products — as heuristics for millennia. That’s unlikely to change any time soon, as much as we might want it to.
Global aid cuts could lead to 9.4 million deaths by 2030, Lancet study says
Over the past year, sharp aid cuts have forced the closure of soup kitchens in war-riven Sudan, led to medicine shortages across sub-Saharan Africa, and resulted in reductions in food rations in places such as Somalia and Haiti. A new study published Monday in the Lancet puts a number on the potential human toll as the global humanitarian system cracks apart, projecting an extra 9.4 million deaths by 2030 if the current trends persist. The study amounts to an early picture of how funding reductions from the United States and other Western countries could undo decades of health gains, leading to upsurges in HIV/AIDS, malaria and hunger across the developing world.
JB: Impactful journals publish research that holds politicians to account for the decisions they make. This news story from The Washington Post about a recent Lancet study is a good example of that in action. The Post may not be able to cover important stories like this in the future after a third of the newsroom was let go this week.
More news and analysis continues after the subscription fold. Paid subscribers will learn about the latest developments in publishing integrity, AI, peer review, open access and much more. You can either subscribe individually or as a group.
