On this page you can download the discussion paper that was submitted for publication in the Journal for Media Linguistics. The blogstract summarises the submission in a comprehensible manner. You can comment on the discussion paper and the blogstract below this post. Please use your real name for this purpose. For detailed comments on the discussion paper please refer to the line numbering of the PDF.
Blogstract of
Appeals to Generative Authority and the AI Reference Paradox. A Pragmatic Analysis of Reference to LLMs in Online Debates
by Dominik Hetjens
How LLMs are Reshaping Online Debate
Digital discourse has entered a new era. On platforms like Reddit, the traditional exchange between users is being disrupted by a silent third party: the Large Language Model (LLM). In an explorative study of the German-language subreddit r/de, I investigated how references to ChatGPT in comments are altering the nature of online debates. The results suggest that users have developed nuanced ways to reference the LLM and frame their interaction with it, ascribing differing degrees of authority to the AI.
Different Types of AI-references
By analysing their function, four distinct types could be identified:
- The Scribe Reference: The AI acts as a creator of primary material, providing a text for use outside of the discussion.
- The Informant Reference: The AI is framed as a neutral data source, often introduced with the casual phrase “Ich habe mal ChatGPT gefragt” (“I just asked ChatGPT”).
- The Corroborator Reference: The user strategically employs the AI to validate a pre-existing position, using specific markers like the conditional “Wenn” (“if/when”) to imply the results are reproducible.
- The Arbiter Reference: In its most extreme form, the AI is weaponised as a final authority to abruptly terminate debates, proclaiming a “higher truth” that leaves no room for further human deliberation.
The AI-Reference Paradox
One of the most striking findings is the emergence of what could be called the “AI-reference paradox.” Users frequently exhibit overt metadiscursive scepticism by using hedges like “aus Spaß” (“for fun”) to save face, while simultaneously placing deep-seated trust in the AI’s factual claims. This creates a “buffer of deniability”: by framing the interaction as a casual experiment, users can leverage the AI’s authority while insulating themselves from criticism should the output prove incorrect.
Interactional Profiles
The study also found that where a comment appears in Reddit’s “tree” structure correlates with its function. Informant references (Type 2) typically appear early in a thread (most often on Level 1), serving as a foundation for the discussion. In contrast, Arbiter references (Type 4) tend to appear deeper in the nested comment structure, surfacing only after a human-to-human debate has reached a stalemate. At this point, the AI is brought in not to inform, but to “win.”
AI–Human–Human–AI Interaction
We are witnessing a shift in digital debating culture. Traditional markers of expertise and epistemic authority are being supplemented, and in some cases supplanted, by the ability to strategically “interrogate” generative systems. This creates an AI–human–human–AI interaction where every interlocutor must account for the separate, private conversations their peers might be having with an algorithm, while at the same time framing their own interactions with it in the best possible way.
As LLMs become further embedded in our digital lives, the power to win an argument may no longer rest on what you know, but on how effectively you can recruit an AI Arbiter to your side.
Review for the submission „Appeals to Generative Authority and the AI Reference Paradox“
reviewed by Antje Wilton-Franklin
Recommendation: major revisions
Thank you very much for inviting me to review this interesting study – I was certainly intrigued by the topic and I think it is indeed worthy of exploration, as reliance on and trust in AI as a source of knowledge has and will have a great impact on the way humans interact with each other. Therefore, I like the fact that the study takes a discourse analytic approach and uses very systematic and transparent methodology to source and analyse the data. The structure of the text makes it easy to follow and to understand the respective steps in the analysis.
However, after careful and reiterated reading some questions and issues remain for me:
First, I would like a broader discussion as to why a discourse analysis is needed – what does the fact that people act like shown in the data on reddit do within tech discourse, or CMC discourse or even media discourse in general?
Second, I feel rather uncomfortable with the (in my opinion) very low number of comments (especially when sorted into categories) that make up the interesting part of the corpus – I assume it would have been easy to source more comments? While I think that the categories themselves do make sense, having a category with just one representative is difficult in my view. With the methodology presented in the text, I guess it would not have been a problem to widen the search until more comments with the phenomena of interest were available. As the dataset is now, all findings seem rather preliminary and wouldn’t justify the rather broad (and somewhat bold) claims made in the discussion.
Third, I fail to see the truly innovative character of the way people reference AI (ChatGPT) in the data presented here. Asking as a reference to a knowledge source, in particular a digital knowledge source, is not new with the advent of AI, but has been done for sources like Google and Wikipedia before. Do you have any research or sources on this to substantiate your claims that there is a qualitative difference in referencing? If there are any (older) studies that can support your (implicit) claim that “asking ChatGPT” is done differently from “asking Google”, then I would strongly recommend to include those studies, discuss them and use them as a backdrop for your own investigation of AI reference. Does Leuckert (2024) shed any light on this? Within this context, I would also clearly separate anthropomorphisation from referencing external authorities as knowledge sources – I even think that anthropomorphisation could be a concept that is very helpful here and should maybe be a) introduced earlier and b) discussed in more detail and applied more consistently in the analysis. In the data, I see how people treat ChatGPT as an agent, but I wonder if they treat it as a specifically human(-like) agent – after all, you can also ask a dog or, as I said earlier, another machine/system/tool. The use of the personal pronoun “es” seems to be quite interesting: what exactly ChatGPT *is* to users would be an interesting idea to pursue.
Finally, I have a feeling that the issue of the AI reference paradox is actually the most interesting that emerges from the data. It has relevance for discourse dynamics, human-AI relations and human-human relations in a broader sense. Again, I would like to know if this phenomenon is indeed new (have people asked Wikipedia “aus Spass”, too, or is this something that comes up exclusively with AI?) and if not, whether is it maybe different in quality. However, for an investigation of this aspect I would also strongly recommend to work with a broader and more substantial data base.
Gutachten zur Einreichung „Appeals to Generative Authority and the AI Reference Paradox“
begutachtet von Derya Gür-Şeker
Empfehlung: Überarbeitung
Der Beitrag behandelt ein aktuelles und für das Journal für Medienlinguistik sehr passendes Thema: die Einbindung generativer KI (konkret ChatGPT) in deutschsprachige Reddit-Online-Debatten. Die Verbindung von Medienlinguistik, Pragmatik, Diskurslinguistik, Evidentialität und Fragen epistemischer Autorität ist überzeugend und eröffnet eine relevante Forschungsperspektive. Besonders gelungen ist die Beobachtung, dass ChatGPT in Reddit-Kommentaren nicht nur als Werkzeug, sondern von User:innen als Informationsquelle, Beleginstanz oder argumentative Autorität herangezogen wird. Die vorgeschlagene Typologie der Rollen von KI als Scribe, Informant, Corroborator und Arbiter stellt einen innovativen Beitrag zur Analyse KI-bezogener Online-Kommunikation dar. Diesbezüglich sollten jedoch relevante Quellen im Theorieteil aufgegriffen bzw. benannt werden (Woher stammen diese Kategorien? Gibt es Quellen?).
Gleichzeitig besteht punktueller Überarbeitungsbedarf. Die Datenauswahl und das methodische Vorgehen sollten präziser beschrieben werden. Es bleibt unklar, wie groß das Sekundärkorpus ist, was genau unter „same comment trees“ verstanden wird und nach welchen Kriterien die Kontextkommentare als relevant für die Analyse ausgewählt wurden. Auch die Beschränkung auf das Suchwort „ChatGPT“ sollte stärker reflektiert werden, da dadurch andere Bezeichnungen für generative KI (Gemini, Claud etc.) ausgeschlossen bleiben. Was also begründet den Fokus auf ChatGPT genau? (z.B. Begründung durch Nutzer:innenzahlen in Deutschland, Verbreitung etc.).
Zudem sollte die zentrale Forschungsfrage einheitlicher formuliert werden. Sie wird an mehreren Stellen (S. 1, 4 oder 6 etc.) wiederkehrend aufgegriffen, erscheint aber nicht immer in exakt derselben Fassung bzw. mit derselben Schwerpunkten. Für die Stringenz des Beitrags wäre es wichtig, die Forschungsfrage früh klar zu formulieren und sie im weiteren Verlauf konsistent beizubehalten. Auch einige allgemeine Aussagen zur Forschungslücke und zur Veränderung von Online-Debatten durch GenAI sollten vorsichtiger formuliert oder stärker belegt werden. Im Beitrag finden sich einzelne Stellen, an denen Literaturbelege oder genauere Seitenangaben fehlen bzw. Aussagen zu breit oder ungenau erscheinen (siehe Anmerkungen im PDF).
Insgesamt handelt es sich um einen originellen und thematisch sehr relevanten Beitrag. Die Argumentation ist plausibel, die Beispiele und untersuchten Kommentare sind aufschlussreich. Auch die Typologie ist anschlussfähig für weitere Forschung. Vor einer Veröffentlichung sollten jedoch Methodik, Datengrundlage, Forschungsfrage und Belegpraxis geschärft werden. Auch empfiehlt sich eine Änderungen des Titel (statt LLM –>ChatGPT).