\n <\/p>\n
Credit: Artem Cherednik\/iStock via Getty<\/span><\/p>\n The number of papers submitted to the top artificial-intelligence conferences has shot up, with some events seeing numbers rise more than tenfold over the past decade. <\/p>\n This is not just a side effect of the rapid increase in global AI research output, says Buxin Su, a mathematician at the University of Pennsylvania in Philadelphia. AI conferences tend to attract multiple submissions from the same author, which poses a serious problem: who\u2019s going to sort through them all to find the most exciting and high-quality work? <\/p>\n In a study posted on the preprint server arXiv in October1, Su and his colleagues describe a system that requires authors making more than one submission to directly compare their papers, ranking them by quality and potential impact. The rankings are calibrated against the assessments of peer reviewers (who do not see the self-ranking information) to highlight the top picks. <\/p>\n The approach was tested on 2,592 papers submitted by 1,342 researchers to the 2023 International Conference on Machine Learning (ICML), a leading AI event that was held in Honolulu, Hawaii. Sixteen months after the conference, Su and his team assessed each paper\u2019s real-world impact by tracking citation counts and comparing these to the calibrated peer-review scores.<\/p>\n Su says that the self-rankings and calibrated scores provide important insights that could be used to predict a paper\u2019s future performance. For example, the team found that papers with the highest self-rankings went on to receive twice as many citations as those with the lowest ones. \u201cThe authors\u2019 rankings are a very good predictor of the long-term impact,\u201d says Su. \u201cThe calibrated scores better reflect the true quality.\u201d <\/p>\n ICML 2026, which is due to be held in Seoul later this year, will be the first event to formally adopt the self-ranking methodology, says Su, who is a member of the conference\u2019s integrity committee.<\/p>\n He adds that although the method could be used at any research conference, it would be particularly useful for AI conferences because of the large number of multiple submissions they attract. His study found that in more than three\u2011quarters of ICML 2023 submissions, at least one author had more than one paper submitted to the conference. AI conferences are also being inundated with AI-generated papers, which is putting further pressure on those who have to sort and review them.<\/p>\n Self-rankings are \u201ca really novel and really cool idea\u201d, says Nihar Shah, a computer scientist at Carnegie Mellon University in Pittsburgh, Pennsylvania. But he is not convinced that authors can accurately predict which of their papers will have the most impact. \u201cThe claim that authors have better predictive power than reviewers could be an artefact of [Su and colleagues\u2019] methodology, rather than what\u2019s actually happening in the real world,\u201d he says.<\/p>\n That said, Shah, who served as scientific integrity chair at ICML 2025 in Vancouver, Canada, welcomes any efforts to address the surge in submissions to AI conferences. \u201cThat\u2019s absolutely a problem that we don\u2019t really know how to solve at this point.\u201d <\/p>\n Emma Pierson, a computer scientist at the University of California, Berkeley, agrees that self-rankings are an \u201cexciting possibility\u201d. \u201cYou know which papers are your \u2018baby\u2019, which papers you really love,\u201d she says. \u201cI think the author\u2019s own self-ranking would be one valuable source of input if you can get them to honestly provide it.\u201d<\/p>\n Both Shah and Pierson worry that researchers might deliberately try to game the system, however \u2014 for example, by giving their weaker papers higher rankings in an effort to balance out critical referee comments. <\/p>\n","protected":false},"excerpt":{"rendered":" Credit: Artem Cherednik\/iStock via Getty The number of papers submitted to the top artificial-intelligence conferences has shot up, with some events seeing numbers rise more than tenfold over the past decade. This is not just a side effect of the rapid increase in global AI research output, says Buxin Su, a mathematician at the University<\/p>\n","protected":false},"author":1,"featured_media":43696,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[58],"tags":[4092,1556,6582,17700,10246,4107,22803,1168],"class_list":{"0":"post-43695","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-science","8":"tag-asks","9":"tag-author","10":"tag-conference","11":"tag-deluge","12":"tag-paper","13":"tag-papers","14":"tag-selfranked","15":"tag-top"},"_links":{"self":[{"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/posts\/43695","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=43695"}],"version-history":[{"count":0,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/posts\/43695\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/media\/43696"}],"wp:attachment":[{"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=43695"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=43695"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=43695"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}An honest assessment?<\/h2>\n