Department of Computational and Data Sciences
SPEAKER : Nihar Shah (CMU)
TITLE : “Security and Privacy in Peer Review”
Date & Time : June 23, 2023, 12:00 Noon
Venue : # 102 CDS
Peer review is the backbone of scientific research, used to review hundreds of thousands of papers and allocate billions of dollars of grants annually. We will discuss two challenges, and proposed solutions, in peer review.
1) Security: A number of research communities are struggling with the breakdown of peer review where a group of reviewers make a pact with each other of trying to get assigned each others’ papers and then accept each others’ papers. We will review various such attacks as well as defenses proposed in the literature. We will discuss the assumptions underlying each of these defenses. We will then present a defense — based on randomized assignments of reviewers to papers — that does not require these assumptions. We will present theoretical guarantees as well as discuss practical deployments for review of thousands of papers. 2) Privacy: Anonymity of reviewers is a bedrock of peer review that enables reviewers to critique freely. However, in an analysis of a top conference with thousands of papers and reviewers, we find that the routine human habit of “batching” tasks — where people keep aside blocks of time for certain kinds of tasks — can allow authors to identify the reviewers of their papers. We then provide mechanisms that can impart anonymity despite batching, and provide strong theoretical guarantees of our mechanisms. A survey article on challenges, experiments, and computational tools in peer review can be found here: https://www.cs.cmu.edu/~nihars/preprints/SurveyPeerReview.pdf
Nihar B. Shah is an Assistant Professor in the Machine Learning and Computer Science departments at Carnegie Mellon University (CMU). His research broadly lies in the fields of statistics, machine learning, information theory, and game theory. The recent focus of his research is on addressing systemic issues in peer review, involving experiments, algorithm design and deployment, and theoretical guarantees. His algorithms for peer review are used for the review of tens of thousands of submissions, and his experiments have helped in evidence-based policy design in many peer-review venues. He is a recipient of a JP Morgan faculty research award, a Google Research Scholar Award, an NSF CAREER Award, the David J. Sakrison memorial prize from EECS Berkeley for a “truly outstanding and innovative PhD thesis”, and several paper awards.
Host Faculty: Dr. Danish Pruthi
ALL ARE WELCOME