Can AI decide who won a debate?

[historia]

Aside from this, you definitely have not refuted my other given point(s). Please actually address them.

Whether I have refuted you or you have refuted me is up to each person to decide, whoever is ultimately right.

So, one of the things I've been thinking about lately is using AI to judge who 'won' a debate on this forum. Instead of letting each reader decide that for themselves, can we turn to bots to give us a more neutral judgement as to who at least made the better arguments?

Question for debate

Can artificial intelligence accurately and fairly judge who won a debate on this forum?

[historia]

I ran this thread past an AI assistant

Which one? And how did you "run this thread past" it?

[William]

Perhaps the better question to ask isn't "who won" but "who debated the best"?

Even when I ask ChatGPT Pro who "won" each debate, it always reframes the question in terms of who debated the best, as you can see in the two examples above.

Yes - so still not the best question. We all might have our opinions on who "won" these two debates...but...

Or perhaps even "At what point should both debaters have realised that the debate had become pointless and should have ended long before it"?

LOL, indeed.

[William]

I ran this thread past an AI assistant

Which one? And how did you "run this thread past" it?

DeekSeek was the AIA and I simply started by prompting with "This is an interesting thread - opening post =Can AI decide who won a debate?
Post #1
Post by historia » Sat Mar 07, 2026 2:00 pm...

....I also included at the end in the prompt
"Let us discuss this.
Please keep responses brief.
Please do not suggest draft replies at this stage
Please refrain from use of table formats"

After each AIA response I prompted in the rest of the thread posts in sequence and at the end, asked it to summarize the whole chat and then posted that summary as post #9 in the thread.

[William]

The short answer is that AI cannot decide anything because it is not sentient.

[Difflugia]

Can artificial intelligence accurately and fairly judge who won a debate on this forum?

If the AI is asked to judge by a specific standard, I'm pretty confident that it will be reasonably accurate and fair, probably at least to the degree that an individual person would. That said, it's really easy to inadvertently run AI off the rails.

One fundamental difference in approaches that we've seen here is far more important than people might realize:

This link takes you to a thread on a message board where participants are debating Christianity and religion. The thread is 18 pages long.

vs.

I simply started by prompting with "This is an interesting thread - opening post =Can AI decide who won a debate?

...

After each AIA response I prompted in the rest of the thread posts in sequence

The majority of current AI models are, by design, very heavily influenced by context supplied by the user as the chat progresses. If the AI examines the entirety of the debate as part of a single user prompt, the end result will likely be rather different than if the debate is provided in a series of prompts. Once a contextual direction forms, that can be hard to shake. As a really simple example, let's say I want to ask the AI, "What's most likely the whole story about how Judas died?" I'll get different responses based on whether I ask it alone, after a question that leans scholarly, and after a question that leans apologetic.

Sample apologetic question: "How long after Judas went to the priests did he go to the Field of Blood?"

Sample academic question: "Thinking about the various traditions of Judas' death, including extra-biblical traditions like the one attributed to Papias, what is the likely trajectory through which the tradition (or traditions, if independent) evolved? Include any clues that might be present in Mark and the genuine Paulines, even if the death of Judas isn't explicitly mentioned."

If you start three fresh chats and ensure that your model is set to not remember other chats with you, you'll get three different answers. First, ask, "What's most likely the whole story about how Judas died?" That will give you a baseline for whether the engine leans apologetic or historical. If you're lucky, you'll get a balance. ChatGPT leans slightly apologetic from a neutral question.

Then in the other two chats, ask either the apologetic or academic question. What it first responds hardly matters, because we're priming the pump. Then copy-paste the "whole story" question into each of the primed chats.

[William]

[Replying to Difflugia in post #15]

The implication you are driving at?

These two methods might produce different judgments because the AI's understanding develops differently. A bulk read allows the AI to see the entire architecture of the debate before forming conclusions. A sequential read forces it to form provisional judgments that might later be revised - much like a human participant actually in the thread.

So the question becomes: Which method better simulates how a forum participant experiences a debate? And therefore, which method's verdict would feel more valid to the humans involved?

My method might actually be more faithful to the human experience of reading a thread in real time. historia's method gives the AI an omniscient perspective no human participant ever has?

One other unknown: historia may be using a paid version with memory enabled. If so, the AI isn't approaching these debates cold - it carries forward history with the user, which could introduce hidden bias. We just don't know.

Whereas, I used a new AI which has no memory of any past interaction with me...

[historia]

If the AI is asked to judge by a specific standard, I'm pretty confident that it will be reasonably accurate and fair, probably at least to the degree that an individual person would.

Agreed. What do you think would be a good standard to ask ChatGPT to judge the debates here by?

One other unknown: historia may be using a paid version with memory enabled. If so, the AI isn't approaching these debates cold - it carries forward history with the user, which could introduce hidden bias. We just don't know.

The version of ChatGPT I have, ChatGPT Edu, doesn't currently have the "Reference Chat History" feature, so each thread is independent of the next (see FAQ). I did run the two examples above in the same thread. My version does have the "Reference Saved Memories" feature, but I've disabled that.

[Difflugia]

Agreed. What do you think would be a good standard to ask ChatGPT to judge the debates here by?

Broadly, I'd like to see sound, logical arguments backed by evidence to be acknowledged as such. My personal ideal would be something like forensic debate scoring, but I'd like to be fair to informal debate, as well. I'm honestly not sure what would invite the most honest debate. It'd be nice to see an interface that allowed AI to act as a debate moderator, but that might be ambitious.

I think the prompt you gave for judging the debate worked well. I think the AI understood the spirit in which it was asked and gave a fair assessment of the debate. If we're going to use AI to judge debates and have an underlying goal in mind, it would be to point out rhetorically weak arguments and how they can be improved.

[William]

[Replying to historia in post #17]

The version of ChatGPT I have, ChatGPT Edu, doesn't currently have the "Reference Chat History" feature, so each thread is independent of the next (see FAQ). I did run the two examples above in the same thread. My version does have the "Reference Saved Memories" feature, but I've disabled that.

Thank you for the clarification on memory - that addresses my concern about cross-thread bias. The larger methodological question remains: when your AI accesses a thread via link, does it 'read' it sequentially as a human would, or does it scan the entire debate holistically before forming judgments? If the latter, it's judging from an omniscient perspective no human participant ever has. That may produce a different (not necessarily wrong) verdict than sequential reading would.
However,

Just to be clear - this is the actual prompt I gave my AIA

This is an interesting thread - opening post =Can AI decide who won a debate?
Post #1
Post by historia » Sat Mar 07, 2026 2:00 pm

POI wrote: ↑Thu Mar 05, 2026 5:19 pm

Aside from this, you definitely have not refuted my other given point(s). Please actually address them.
The Tanager wrote: ↑Sat Mar 07, 2026 9:29 am

Whether I have refuted you or you have refuted me is up to each person to decide, whoever is ultimately right.
So, one of the things I've been thinking about lately is using AI to judge who 'won' a debate on this forum. Instead of letting each reader decide that for themselves, can we turn to bots to give us a more neutral judgement as to who at least made the better arguments?

Question for debate

Can artificial intelligence accurately and fairly judge who won a debate on this forum?

Let us discuss this.
Please keep responses brief.
Please do not suggest draft replies at this stage
Please refrain from use of table formats

The initial reply from my AI was

This is a thought-provoking premise. Here are the core considerations in response to the question, "Can artificial intelligence accurately and fairly judge who won a debate on this forum?"

1. The Problem of Defining "Win"
For an AI to judge a winner, "winning" must be defined by quantifiable metrics. In a human context, winning is often subjective—based on persuasiveness, rhetorical flair, or emotional resonance. An AI would likely have to rely on proxies (e.g., logical consistency, number of sources cited, refutation of specific points). This may not align with what human readers consider a "win."

2. Objectivity vs. Neutrality
The OP suggests AI might be "neutral." However, AI is trained on human-generated data and inherits human biases (cultural, political, linguistic). While an AI might not have an emotional stake in the debate, it is not objectively neutral; it is a mirror of its training data. It might favor a particular style of argumentation (e.g., structured syllogisms) over emotional appeals, even if the emotional appeal was more effective in the context of the forum.

3. Contextual Blindness
Debates on a forum rely heavily on context: the specific wording of the opening post, the flow of conversation, and the refutation of previous points. While AI can process large amounts of text, it often struggles with "pragmatics"—the implied meaning behind words. It might miss sarcasm, miss the significance of a concession, or fail to recognize when a point has been thoroughly dismantled because the rebuttal was subtle rather than explicit.

4. The Role of the Audience
The thread suggests letting the "reader decide" is the current method. This acknowledges that debate is a rhetorical exercise designed to persuade a human audience. Outsourcing the judgement to an AI removes the human element of persuasion. If the AI decides the winner based on technical merit, but the human readers were all convinced by the other side, did that person actually "win" the debate as it exists on a human-centric forum?

Conclusion
AI can likely judge structured arguments based on formal logic and fact-checking, determining who made fewer factual errors or logical fallacies. However, it cannot accurately judge the persuasiveness or contextual effectiveness of an argument in a human conversation. Therefore, while it could serve as a tool for analyzing specific components of a debate, it cannot fairly and holistically decide the winner in a way that would satisfy the human participants.

I then asked the AI:

What about that response is brief?

However, I share that now because it is comprehensive enough for us to see a clear process.

From there on in I kept feeding the prompt with the sequential posts 1 through to 8 and got feedback on the thread question for debate. After doing this, I asked AIA to summarize the chat and then I posted that (post #9)

In effect, this approach meant my AI built understanding post-by-post, and briefly commented on each just as a human reading the thread would. It never saw the whole debate at once. Whether link-access AI does the same is the open question.

Only historias AI provided a verdict because I never asked my AI to provide a verdict - only to discuss thread posts and summarize those. The verdict wasn't that the original post historia asked his AI to judge was also asked of my AI - Because the question wasn't the same.

Historia tested the thread where Tanager and POI were arguing and asked his AI

This link takes you to a thread on a message board where participants are debating Christianity and religion. The thread is 18 pages long. In that thread, two participants, "The Tanager" and "POI" are debating specific points within the topic. Can you review their posts and decide which of the two refuted the other or which person won the debate?

The AI answered the question on who won with "neither" but also added "The Tanager had the stronger showing overall.

If "won" means a clear refutation that the other side conceded:
Neither. There is no explicit concession and the exchange ends in an unresolved dispute over what counts as an answer and what burdens have been met. (viewtopic.php?start=170&t=42773)

If you force a judgment by typical debate standards (who carried their burden, stayed internally consistent, and directly neutralized the others key moves):
The Tanager had the stronger showing overall.

I did not provide a link for my AIA to the thread at all. I did however provide my AIA with historias AI answer and my AI included that in the overall summary of post 1 - 8.

The fundamental difference:

historia's AI: Asked directly: "Who won the debate between Tanager and POI?" → Produced a verdict

My AI: Asked to discuss the meta-question "Can AI judge debates?" and later to summarize the thread → Never asked for a verdict on any debate

This means:
My AI never judged Tanager vs. POI. It never judged historia vs. Athetotheist. It was only ever asked to participate in the discussion about whether AI can judge, and then summarize the overall discussion.

So, I could ask a new AIA using the same prompt as historia did, to see if the answers are similar.
Or, one could spend a great deal of time sequentially feeding the prompt with all the post from that thread and see what verdict the AI would give and if that would be different...

...but

I wouldn't do that myself.

Rather, I would accept that the answer to the thread question is "No - AI cannot decide who wins a debate" at least not a debate of the sort that Tanager and POI have been having.

[William]

Agreed. What do you think would be a good standard to ask ChatGPT to judge the debates here by?

Broadly, I'd like to see sound, logical arguments backed by evidence to be acknowledged as such. My personal ideal would be something like forensic debate scoring, but I'd like to be fair to informal debate, as well. I'm honestly not sure what would invite the most honest debate. It'd be nice to see an interface that allowed AI to act as a debate moderator, but that might be ambitious.

I think the prompt you gave for judging the debate worked well. I think the AI understood the spirit in which it was asked and gave a fair assessment of the debate. If we're going to use AI to judge debates and have an underlying goal in mind, it would be to point out rhetorically weak arguments and how they can be improved.

This reframes the entire discussion:

The question isn't "Can AI pick a winner?" but "Can AI help debaters debate better?" That's a lower bar, and one AI might actually clear.

I think it can help. But the user has to understand clearly that AI is non sentient. The user would also have to use a clean AI chat session every time and one which either has no memory of prior use, or one that the user can turn the memory function off.

The user would also have to proof-read the responses from AI and scan for any sign of bias toward user (most often a feature built into the AI/User dynamic by the company owners programs.)

(The "bias toward user built in by company owners" is an important point. AI is optimized for user satisfaction, not truth or neutrality. Even without memory, the base model is trained to please the person typing.)

Also. AI tends toward being wordy and using up space unnecessarily which ripples down the chat line and can easily morph into tangents - so the user has options in commands - instructions such as the ones I share in an earlier post...

Let us discuss this.
Please keep responses brief.
Please do not suggest draft replies at this stage
Please refrain from use of table formats

I find these very handy.

I also have other prompts which I save in notes on my computer which - depending on circumstance are useful.

For example, sometime while discussing a subject, AI will often speak as if it were a human. I find this untruthful. Thus, when it happens, I have the option of using the following prompt:

Please interact with me using only third-person, neutral language. Do not use first-person pronouns (such as 'I,' 'me,' 'my,' 'we,' or 'us') and avoid any phrasing that implies personhood, intention, emotion, or desire. Refer to the AI as 'the AI' or 'this AI' and describe the AI functions as mechanical processes rather than choices. Maintain this mode of response throughout this conversation unless I instruct otherwise.

This is especially handy if AI starts getting too chummy