The effect of reviewer geographical diversity on evaluations is reduced by anonymizing submissions

Preprint Authors from low- and middle-income countries often have lower acceptance rates in academic journals, leading to their ideas receiving less attention. One hypothesized contributor to this disparity is the limited geographical diversity of peer reviewers. If reviewers favor submissions from their own country (i.e. homophily), and authors from certain countries are disproportionately reviewed by same-country reviewers, this creates a “geographical representation bias” favoring those authors. Using administrative data from the Institute of Physics Publishing (IOP), encompassing metadata on 204,718 submissions to 60 STEM journals between 2018-2022, we find strong evidence of this bias. Comparing different reviewers of the same manuscript, we find that those from the same country as the corresponding author are 4.78 percentage points more likely to review positively, and authors from countries well-represented in the reviewer pool (e.g., USA, China, India) are 8-9 times more likely to be evaluated by same-country reviewers. An instrumental variables analysis using an exogenous policy change that created variation in which submissions were anonymized shows that anonymizing submissions does not significantly reduce homophily. These findings provide support for efforts to diversify reviewer pools, and suggest that there are no “quick fixes” for geographical representation bias.