skip to main content
10.1145/1458082.1458127acmconferencesArticle/Chapter ViewAbstractPublication PagescikmConference Proceedingsconference-collections
research-article

An algorithm to determine peer-reviewers

Published: 26 October 2008 Publication History

Abstract

The peer-review process is the most widely accepted certification mechanism for officially accepting the written results of researchers within the scientific community. An essential component of peer-review is the identification of competent referees to review a submitted manuscript. This article presents an algorithm to automatically determine the most appropriate reviewers for a manuscript by way of a co-authorship network data structure and a relative-rank particle-swarm algorithm. This approach is novel in that it is not limited to a pre-selected set of referees, is computationally efficient, requires no human-intervention, and, in some instances, can automatically identify conflict of interest situations. A useful application of this algorithm would be to open commentary peer-review systems because it provides a weighting for each referee with respects to their expertise in the domain of a manuscript. The algorithm is validated using referee bid data from the 2005 Joint Conference on Digital Libraries.

References

[1]
C. Basu, H. Hirsh, W. Cohen, and C. Nevill-Manning. Technical paper recommendation: A study in combining multiple information sources. Journal of Artificial Intelligence Research, 14:231--252, 2001.
[2]
M. Bianchini, M. Gori, and F. Scarselli. Inside pagerank. ACM Transanctions on Internet Technology, 5(1):92--128, 2005.
[3]
J. Bollen, M. A. Rodriguez, H. Van de Sompel, L. L. Balakireva, and A. Hagberg. The largest scholarly semantic network. ever. In ACM World Wide Web Conference, Banff, Canada, Banff, Canada 2007. ACM Press.
[4]
P. Bonacich. Power and centrality: A family of measures. American Journal of Sociology, 92(5):1170--1182, 1987.
[5]
S. Brin and L. Page. The anatomy of a large-scale hypertextual web search engine. Computer Networks, 30:107--117, 1998.
[6]
A. Collins and E. Loftus. A spreading activation theory of semantic processing. Psychological Review, 82:407--428, 1975.
[7]
W. J. Conover. Practical nonparametric statistics. John Wiley and Sons, New York, NY, USA, 1971.
[8]
F. Crestani. Application of spreading activation techniques in information retrieval. Artificial Intelligence Review, 11(6):453--582, 1997.
[9]
S. T. Dumais and J. Nielsen. Automating the assignment of submitted manuscripts to reviewers. In Research and Development in Information Retrieval, pages 233--244, 1992.
[10]
D. Fogaras and B. Rácz. Algorithms and Models for the Web-Graph, chapter Towards Scaling Fully Personalized PageRank, pages 105--117. Springer, 2004.
[11]
J. J. M. Guervós and P. A. C. Valdivieso. Conference paper assignment using a combined greedy/evolutionary algorithm. In Proceedings of the International Conference on Parallel Problem Solving from Nature, pages 602--611, 2004.
[12]
X. Liu, J. Bollen, M. L. Nelson, and H. Van de Sompel. Co-authorship networks in the digital library research community. Information Processing and Management, 41(6):1462--1480, 2005.
[13]
M. E. J. Newman. Scientific collaboration networks: I. network construction and fundamental results. Physical Review E, 64(1):016131, 2001.
[14]
M. E. J. Newman. Scientific collaboration networks: Ii. shortest paths, weighted networks, and centrality. Physical Review E, 64(1):016132, 2001.
[15]
M. E. J. Newman. Coauthorship networks and patterns of scientific collaboration. Proceedings of the National Academy of Science, 101:5200--5205, 2004.
[16]
M. A. Rodriguez. Grammar-based random walkers in semantic networks. Knowledge-Based Systems, {in press}, 2008.
[17]
M. A. Rodriguez, J. Bollen, and H. Van de Sompel. The convergence of digital libraries and the peer-review process. Journal of Information Science, 32(2):149--159, 2006.
[18]
M. A. Rodriguez, J. Bollen, and H. Van de Sompel. Mapping the bid behavior of conference referees. Journal of Informetrics, 1(1):62--82, 2007.
[19]
T. Sumner. Report on the fifth ACM/IEEE joint conference on digital libraries - cyberinfrastructure for research and education. D-Lib Magazine, 11(7/8), 2005.
[20]
S. Wasserman and K. Faust. Social Network Analysis: Methods and Applications. Cambridge University Press, Cambridge, UK, 1994.
[21]
S. White and P. Smyth. Algorithms for estimating relative importance in networks. In KDD '03: Proceedings of the ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 266--275, New York, NY, USA, 2003. ACM Press.
[22]
D. Yarowsky and R. Florian. Taking the load off the conference chairs: towards a digital paper-routing assistant. In Proceedings of the 1999 Joint SIGDAT Conference on Empirical Methods in NLP and Very-Large Corpora., 1999.

Cited By

View all
  • (2024)Systematic mapping of automated reviewer recommendation solutionsJournal of Computer Science and Technology10.24215/16666038.24.e1624:2(e16)Online publication date: 18-Oct-2024
  • (2024)A multilayer network diffusion-based model for reviewer recommendationChinese Physics B10.1088/1674-1056/ad181d33:3(038901)Online publication date: 1-Mar-2024
  • (2023)Science Communication as a Collective Intelligence Endeavor: A Manifesto and Examples for ImplementationScience Communication10.1177/1075547023116263445:4(539-554)Online publication date: 4-Apr-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CIKM '08: Proceedings of the 17th ACM conference on Information and knowledge management
October 2008
1562 pages
ISBN:9781595939913
DOI:10.1145/1458082
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 October 2008

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. co-authorship networks
  2. peer-review process

Qualifiers

  • Research-article

Conference

CIKM08
CIKM08: Conference on Information and Knowledge Management
October 26 - 30, 2008
California, Napa Valley, USA

Acceptance Rates

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)32
  • Downloads (Last 6 weeks)4
Reflects downloads up to 19 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Systematic mapping of automated reviewer recommendation solutionsJournal of Computer Science and Technology10.24215/16666038.24.e1624:2(e16)Online publication date: 18-Oct-2024
  • (2024)A multilayer network diffusion-based model for reviewer recommendationChinese Physics B10.1088/1674-1056/ad181d33:3(038901)Online publication date: 1-Mar-2024
  • (2023)Science Communication as a Collective Intelligence Endeavor: A Manifesto and Examples for ImplementationScience Communication10.1177/1075547023116263445:4(539-554)Online publication date: 4-Apr-2023
  • (2023)A Dataset on Malicious Paper Bidding in Peer ReviewProceedings of the ACM Web Conference 202310.1145/3543507.3583424(3816-3826)Online publication date: 30-Apr-2023
  • (2023)Reviewer Assignment Decision Support in an Academic Journal based on Multicriteria Assessment and Text Mining2023 IX International Conference on Information Technology and Nanotechnology (ITNT)10.1109/ITNT57377.2023.10139187(1-4)Online publication date: 17-Apr-2023
  • (2023)An Algorithm for Peer Reviewer Recommendation Based on Scholarly Activity AssessmentIEEE Access10.1109/ACCESS.2023.326385711(39609-39620)Online publication date: 2023
  • (2023)Literature ReviewKnowledge Recommendation Systems with Machine Intelligence Algorithms10.1007/978-3-031-32696-7_2(9-27)Online publication date: 1-Oct-2023
  • (2023)The Use of Blockchain in Fighting Medical Misinformation: A Concept PaperBlockchain in Healthcare10.1007/978-3-031-14591-9_15(225-239)Online publication date: 2-Jan-2023
  • (2022)Near-Optimal Reviewer Splitting in Two-Phase Paper Reviewing and Conference Experiment DesignProceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems10.5555/3535850.3536062(1642-1644)Online publication date: 9-May-2022
  • (2022)DDMF: A Method for Mining Relatively Important Nodes Based on Distance Distribution and Multi-Index FusionApplied Sciences10.3390/app1201052212:1(522)Online publication date: 5-Jan-2022
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media