\n<\/p>\n
A recent op-ed by David Randall, executive director of the Civics Alliance and director of research at the National Association of Scholars, argues that faculty hiring in American universities has become so corrupt that it requires sweeping legislative intervention. NAS\u2019s proposed Faculty Merit Act would require public universities to publish every higher ed standardized test score\u2014SAT, ACT, GRE, LSAT, MCAT and more\u2014of every faculty member and every applicant for that faculty member\u2019s position across different stages of a faculty search. The goal, they claim, is to expose discrimination and restore meritocracy.<\/p>\n
Letter to the editor<\/strong><\/p>\n A letter has been\u00a0submitted in response to this article. You can\u00a0read the letter\u00a0here,\u00a0and view all of our letters to the editor\u00a0here.<\/p>\n The proposal\u2019s logic is explicit: If standardized test scores are a reasonable proxy for faculty merit, then a fair search should select someone with a very high score. If average scores decline from round to round, or if the eventual hire scored lower than dozens\u2014or even hundreds\u2014of rejected applicants, the public, Randall argues, should be able to \u201csee that something is wrong.\u201d<\/p>\n But the Faculty Merit Act rests on a serious misunderstanding of how measurement and selection actually work. Even if one accepts Randall\u2019s premise that a standardized test score \u201cisn\u2019t a bad proxy for faculty merit,\u201d the conclusions he draws simply do not follow. The supposed red flags the proposed act promises to reveal are not evidence of corruption. They are the expected mathematical consequences of using an imperfect measure in a large applicant pool.<\/p>\n I am a data scientist who works on issues of social justice. What concerns me is not only that NAS\u2019s proposal is statistically unsound, but that it would mislead the public while presenting itself as transparent.<\/p>\n The proposed act depends on a simple idea: If standardized test scores are a reasonable proxy for faculty merit, then a fair search should select someone with a very high score. If the person hired has a lower score than many rejected applicants, or if average scores decline from round to round, something must be amiss.<\/p>\n This sounds intuitive. It is also wrong.<\/p>\n To see why, imagine the following setup. Every applicant has some level of \u201ctrue merit\u201d for a faculty job\u2014originality, research judgment, teaching ability, intellectual fit. We cannot observe this truth directly. Instead, we observe a standardized test score, which captures some aspects of ability but misses many others. In other words, the test score contains two parts: a signal (the part related to actual merit) and noise (everything else the test does not measure).<\/p>\n Now suppose a search attracts 300 applicants, as in Randall\u2019s own example. Assume\u2014very generously\u2014that the search committee somehow identifies the single best applicant by true merit and hires that person.<\/p>\n Here is the crucial point: Even if test scores are meaningfully related to true merit, the best applicant will almost never have the highest test score.<\/p>\n Why? Because when many people are competing, even moderate noise overwhelms rank ordering. A noisy measure will always misrank some individuals, and the larger the pool, the more dramatic those misrankings become. This is the same reason that ranking professional athletes by a single skill\u2014free-throw percentage, say\u2014would routinely misidentify the best overall players, especially in a large league.<\/p>\n Before putting numbers on this, we should ask a basic empirical question: How strongly do standardized tests actually predict the kinds of outcomes that matter in academia?<\/p>\n The most comprehensive recent research on the GRE\u2014the test most relevant to graduate education\u2014finds minimal predictive value. A meta-analysis of more than 200 studies found that GRE scores explain just over 3\u00a0percent of the variation in graduate outcomes such as GPA, degree completion and licensing exam performance. For graduate GPA specifically\u2014the outcome the test is explicitly designed to predict\u2014GRE scores explained only about 4\u00a0percent of the variance.<\/p>\n These studies assess near-term prediction within the same educational context: GRE scores predicting outcomes for the very students who took the test, measured only a few years later\u2014under conditions maximally favorable to the test\u2019s validity. The NAS proposal extrapolates from evidence that is already weak even under these favorable conditions. It would evaluate faculty hiring using test scores\u2014often SAT scores\u2014taken at age 17, applied to candidates who may now be in their 30s, 40s or older. Direct evidence for that kind of long-term extrapolation is scarce. However, the limited evidence that does exist points towards weak relationships rather than strong ones. For instance, Google\u2019s internal hiring studies famously found \u201cvery little correlation\u201d between SAT scores and job performance.<\/p>\n Taken together, the research suggests that any realistic relationship between standardized test scores and faculty merit is weak\u2014certainly well below the levels needed to support NAS\u2019s proposed diagnostics.<\/p>\n The proposed Faculty Merit Act raises an important practical question: Even if standardized test scores contain some information about merit, how useful are they when hundreds of applicants compete for a single job?<\/p>\n Taking the GRE meta-analysis at face value, standardized test scores correlate with relevant academic outcomes at only about 0.18. Treating that number as a proxy for faculty merit is already generous, given the decades that often separate testing from hiring and the profound differences between standardized exams and the actual work of a professor. But let us grant it anyway.<\/p>\n Now, consider a search with 300 applicants. With a correlation of 0.18, I\u00a0calculate that the single strongest candidate by true merit would typically score only around the 70th\u00a0percentile on the test\u2014roughly 90th out of 300. In other words, it would be entirely normal for around 90 rejected applicants to have higher test scores than the eventual hire.<\/p>\n Nothing improper has happened. No favoritism or manipulation is required. This outcome follows automatically from combining a weak proxy with a large applicant pool.<\/p>\n Even if we assume a much stronger relationship\u2014say, a correlation of 0.30, which already exceeds what the evidence supports for most academic outcomes\u2014the basic conclusion does not change. Under that assumption, I\u00a0calculate that the best candidate would typically score only around the 80th\u00a0percentile, corresponding to a rank near 60 out of 300. Dozens of rejected applicants would still have higher test scores than the person who gets the job.<\/p>\n This is the point the proposal gets exactly backward. The pattern it treats as a red flag\u2014a hire whose test score is lower than that of many rejected applicants\u2014is not evidence of corruption. It is the normal, mathematically expected outcome whenever selection relies on an imperfect measure. Scaling this diagnostic across many searches does not make it informative; it simply reproduces the same expected misrankings at a larger scale.<\/p>\n The same logic applies to the claim that average test scores should increase at each stage of a search.<\/p>\n Faculty hiring is not one-dimensional. Early stages might screen for general competence; later stages may emphasize originality, research direction, teaching effectiveness and departmental fit\u2014traits that standardized tests measure poorly or not at all. As a search progresses, committees naturally place less weight on test scores and more weight on other information. When that happens, average test scores among finalists can stay flat or even decline. That pattern does not signal manipulation. It signals that the committee is selecting on dimensions that actually matter for the job.<\/p>\n Randall\u2019s op-ed, published by the James G. Martin Center for Academic Renewal, frames the proposal as a response to injustice. But transparency based on invalid diagnostics does not mitigate injustice; it produces it.<\/p>\n Publishing standardized test scores invites the public to draw conclusions that those numbers cannot support\u2014and those conclusions will not fall evenly. Standardized test scores are strongly shaped by socioeconomic background and access to resources. Treating them as a universal yardstick of merit\u2014especially for faculty careers\u2014will predictably disadvantage scholars from marginalized and nontraditional paths.<\/p>\n From the standpoint of justice, this is deeply concerning. Accountability mechanisms must rest on sound reasoning. Otherwise, they become tools for enforcing hierarchy rather than fairness.<\/p>\n If the goal is genuine academic renewal, it should begin with renewing our understanding of what numbers can\u2014and cannot\u2014tell us. Merit cannot be mandated by publishing the wrong metrics, and justice is not served by statistical arguments that collapse under careful inspection.<\/p>\n Chad M. Topaz is a faculty member at Williams College; co-founder of the Institute for the Quantitative Study of Inclusion, Diversity and Equity; and winner of the Mary and Alfie Gray Award for Social Justice from the Association for Women in Mathematics. He is the author of <\/em>Unlocking Justice: The Power of Data to Confront Inequity and Create Change, forthcoming from Princeton University Press in May, and can be found on Bluesky at @chadtopaz.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":" A recent op-ed by David Randall, executive director of the Civics Alliance and director of research at the National Association of Scholars, argues that faculty hiring in American universities has become so corrupt that it requires sweeping legislative intervention. NAS\u2019s proposed Faculty Merit Act would require public universities to publish every higher ed standardized test<\/p>\n","protected":false},"author":1,"featured_media":41852,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[57],"tags":[4308,3350,17653,22204,440],"class_list":{"0":"post-41851","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-education","8":"tag-act","9":"tag-faculty","10":"tag-merit","11":"tag-meritless","12":"tag-opinion"},"_links":{"self":[{"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/posts\/41851","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=41851"}],"version-history":[{"count":0,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/posts\/41851\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/media\/41852"}],"wp:attachment":[{"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=41851"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=41851"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=41851"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}A Statistical Mistake<\/h2>\n
How Strong Is the Test-Merit Relationship, Really?<\/h2>\n
What This Means in Practice<\/h2>\n
Why \u2018Scores Dropped Each Round\u2019 Proves Nothing<\/h2>\n
Transparency, Justice and Bad Diagnostics<\/h2>\n