\n<\/p>\n
The last time you interacted with ChatGPT, did it feel like you were chatting with one person, or more like you were conversing with multiple individuals? Did the chatbot appear to have a consistent personality, or did it seem different each time you engaged with it?<\/p>\n
A few weeks ago, while comparing language proficiency in essays written by ChatGPT with that in essays by human authors, I had an aha! moment. I realized that I was comparing a single voice\u2014that of the large language model, or LLM, that powers ChatGPT\u2014to a diverse range of voices from multiple writers. Linguists like me know that every person has a distinct way of expressing themselves, depending on their native language, age, gender, education and other factors. We call that individual speaking style an \u201cidiolect.\u201d It is similar in concept to, but much narrower than, a dialect, which is the variety of a language spoken by a community. My insight: one could analyze the language produced by ChatGPT to find out whether it expresses itself in an idiolect\u2014a single, distinct way.<\/p>\n
Idiolects are essential in forensic linguistics. This field examines language use in police interviews with suspects, attributes authorship of documents and text messages, traces the linguistic backgrounds of asylum seekers and detects plagiarism, among other activities. While we don\u2019t (yet) need to put LLMs on the stand, a growing group of people, including teachers, worry about such models being used by students to the detriment of their education\u2014for instance, by outsourcing writing assignments to ChatGPT. So I decided to check whether ChatGPT and its artificial intelligence cousins, such as Gemini and Copilot, indeed possess idiolects.<\/p>\n
If you’re enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.<\/p>\n
To test whether a text has been generated by an LLM, we need to examine not only the content but also the form\u2014the language used. Research shows that ChatGPT tends to favor standard grammar and academic expressions, shunning slang or colloquialisms. Compared with texts written by human authors, ChatGPT tends to overuse sophisticated verbs, such as \u201cdelve,\u201d \u201calign\u201d and \u201cunderscore,\u201d and adjectives, such as \u201cnoteworthy,\u201d \u201cversatile\u201d and \u201ccommendable.\u201d We might consider these words typical for the idiolect of ChatGPT. But does ChatGPT express ideas differently than other LLM-powered tools when discussing the same topic? Let\u2019s delve into that.<\/p>\n
Online repositories are full of amazing datasets that can be used for research. One is a dataset compiled by computer scientist Muhammad Naveed, which contains hundreds of short texts on diabetes written by ChatGPT and Gemini. The texts are of virtually the same size, and, according to their creator\u2019s description, they can be used \u201cto compare and analyze the performance of both AI models in generating informative and coherent content on a medical topic.\u201d The similarities in topic and size make them ideal for determining whether the outputs appear to come from two distinct \u201cauthors\u201d or from a single \u201cindividual.\u201d<\/p>\n
One popular way of attributing authorship uses the Delta method, introduced in 2001 by John Burrows, a pioneer of computational stylistics. The formula compares frequencies of words commonly used in the texts: words that function to express relationships with other words\u2014a category that includes \u201cand,\u201d \u201cit,\u201d \u201cof,\u201d \u201cthe,\u201d \u201cthat\u201d and \u201cfor\u201d\u2014and content words such as \u201cglucose\u201d or \u201csugar.\u201d In this way, the Delta method captures features that vary according to their authors\u2019 idiolects. In particular, it outputs numbers that measure the linguistic \u201cdistances\u201d between the text being investigated and reference texts by preselected authors. The smaller the distance, which typically is slightly below or above 1, the higher the likelihood that the author is the same.<\/p>\n
I found that a random sample of 10 percent of texts on diabetes generated by ChatGPT has a distance of 0.92 to the entire ChatGPT diabetes dataset and a distance of 1.49 to the entire Gemini dataset. Similarly, a random 10 percent sample of Gemini texts has a distance of 0.84 to Gemini and of 1.45 to ChatGPT. In both cases, the authorship turns out to be quite clear, indicating that the two tools\u2019 models have distinct writing styles.<\/p>\n
To better understand these styles, let\u2019s imagine that we are looking at the diabetes texts and selecting words in groups of three. Such combinations are called \u201ctrigrams.\u201d By seeing which trigrams are used most often, we can get a sense of someone\u2019s unique way of putting the words together. I extracted the 20 most frequent trigrams for both ChatGPT and Gemini and compared them.<\/p>\n
ChatGPT\u2019s trigrams in these texts suggest a more formal, clinical and academic idiolect, with phrases such as \u201cindividuals with diabetes,\u201d \u201cblood glucose levels,\u201d \u201cthe development of,\u201d \u201ccharacterized by elevated\u201d and \u201can increased risk.\u201d In contrast, Gemini\u2019s trigrams are more conversational and explanatory, with phrases such as \u201cthe way for,\u201d \u201cthe cascade of,\u201d \u201cis not a,\u201d \u201chigh blood sugar\u201d and \u201cblood sugar control.\u201d Choosing words such as \u201csugar\u201d instead of \u201cglucose\u201d indicates a preference for simple, accessible language.<\/p>\n
The chart below contains the most striking frequency-related differences between the trigrams. Gemini uses the formal phrase \u201cblood glucose levels\u201d only once in the whole dataset\u2014so it knows the phrase but seems to avoid it. Conversely, \u201chigh blood sugar\u201d appears only 25 times in ChatGPT\u2019s responses compared to 158 times in Gemini\u2019s. In fact, ChatGPT uses the word \u201cglucose\u201d more than twice as many times as it uses \u201csugar,\u201d while Gemini does just the opposite, writing \u201csugar\u201d more than twice as often as \u201cglucose.\u201d<\/p>\n
Eve Lu; Source: Karolina Rudnicka (data)<\/p>\n
Why would LLMs develop idiolects? The phenomenon could be associated with the principle of least effort\u2014the tendency to choose the least demanding way to accomplish a given task. Once a word or phrase becomes part of their linguistic repertoire during training, the models might continue using it and combine it with similar expressions, much like people have favorite words or phrases they use with above-average frequency in their speech or writing. Or it might be a form of priming\u2014something that happens to humans when we hear a word and then are more likely to use it ourselves. Perhaps each model is in some way priming itself with words it uses repeatedly. Idiolects in LLMs might also reflect what are known as emergent abilities\u2014skills the models were not explicitly trained to perform but that they nonetheless demonstrate.<\/p>\n
The fact that LLM-based tools produce different idiolects\u2014which might change and develop across updates or new versions\u2014matters for the ongoing debate regarding how far AI is from achieving human-level intelligence. It makes a difference if chatbots\u2019 models don\u2019t just average or mirror their training data but develop distinctive lexical, grammatical or syntactic habits in the process, much like humans are shaped by our experiences. Meanwhile, knowing that LLMs write in idiolects could help determine if an essay or an article was produced by a model or by a particular individual\u2014just as you might recognize a friend\u2019s message in a group chat by their signature style.<\/p>\n","protected":false},"excerpt":{"rendered":"
The last time you interacted with ChatGPT, did it feel like you were chatting with one person, or more like you were conversing with multiple individuals? Did the chatbot appear to have a consistent personality, or did it seem different each time you engaged with it? A few weeks ago, while comparing language proficiency in<\/p>\n","protected":false},"author":1,"featured_media":10441,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[50],"tags":[2214,3405,3408,3406,3407],"class_list":{"0":"post-10440","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-environment","8":"tag-chatgpt","9":"tag-gemini","10":"tag-styles","11":"tag-uniquely","12":"tag-writing"},"_links":{"self":[{"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/posts\/10440","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=10440"}],"version-history":[{"count":0,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/posts\/10440\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/media\/10441"}],"wp:attachment":[{"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=10440"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=10440"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=10440"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}