Discussion
Loading...

Post

  • About
  • Code of conduct
  • Privacy
  • Users
  • Instances
  • About Bonfire
petersuber
@petersuber@fediscience.org  ·  activity timestamp last week

Update. AI-assisted peer review shows "a strong and consistent institutional-prestige bias: identical papers attributed to low-prestige affiliations face a significantly higher risk of rejection, despite only modest differences in LLM-assessed quality."
https://arxiv.org/abs/2509.15122

#AI #Bias #PeerReview #Prestige #ScholComm

arXiv.org

Prestige over merit: An adapted audit of LLM bias in peer review

Large language models (LLMs) are playing an increasingly integral, though largely informal, role in scholarly peer review. Yet it remains unclear whether LLMs reproduce the biases observed in human decision-making. We adapt a resume-style audit to scientific publishing, developing a multi-role LLM simulation (editor/reviewer) that evaluates a representative set of high-quality manuscripts across the physical, biological, and social sciences under randomized author identities (institutional prestige, gender, race). The audit reveals a strong and consistent institutional-prestige bias: identical papers attributed to low-prestige affiliations face a significantly higher risk of rejection, despite only modest differences in LLM-assessed quality. To probe mechanisms, we generate synthetic CVs for the same author profiles; these encode large prestige-linked disparities and an inverted prestige-tenure gradient relative to national benchmarks. The results suggest that both domain norms and prestige-linked priors embedded in training data shape paper-level outcomes once identity is visible, converting affiliation into a decisive status cue.
  • Copy link
  • Flag this post
  • Block
Log in

Open Science

We are a network of scientists, developers and organizations building the next generation of digital spaces for open science.

Open Science: About · Code of conduct · Privacy · Users · Instances
Bonfire open science · 1.0.0-rc.2.30 no JS en
Automatic federation enabled
  • Explore
  • About
  • Members
  • Code of Conduct
Home
Login