Can AI decide who won a debate?

[historia]

Aside from this, you definitely have not refuted my other given point(s). Please actually address them.

Whether I have refuted you or you have refuted me is up to each person to decide, whoever is ultimately right.

So, one of the things I've been thinking about lately is using AI to judge who 'won' a debate on this forum. Instead of letting each reader decide that for themselves, can we turn to bots to give us a more neutral judgement as to who at least made the better arguments?

Question for debate

Can artificial intelligence accurately and fairly judge who won a debate on this forum?

[The Tanager]

[Replying to William in post #50]

Do you think AI will provide good structured analysis?

Yes. Adequate enough for the task at hand

I assume you are saying that the task at hand is seeing if AI provides good structured analysis unless you clarify otherwise.

And while it's not how you would do it, is the method I proposed still one that should provide that good structured analysis? It seems comparable. An AI summarizes. An AI gives structured analysis of what the poster is doing by evaluating with normal logical tools.

That is an interesting question. I think if you tried it the way you suggested, then bring the results back and lets see and discuss those.

You obviously see merit in AI in providing good structured analysis or you wouldn't use it as much as you do for that purpose. We need to be objective about the method to judge if we think AI is good at providing good structured analysis. That means approving of the method before we see the results or our dislike of the results can color our judgment there.

I think my method is objectively better than yours because in yours, the analysis is based off of one sentence summaries, while my method bases the analysis off of the entire posts themselves. So, let's pin this down before looking at results.

[William]

[Replying to The Tanager in post #51]

I think my method is objectively better than yours because in yours, the analysis is based off of one sentence summaries, while my method bases the analysis off of the entire posts themselves. So, let's pin this down before looking at results.

You appear to be confused as to my method.

In this thread, I C&P full posts. In the CCA thread I did the same but asked the AI to write short notes on each post, as a way of mapping. (re capping)
In this thread- after I went through the process (already described in my prior posts here) , I offered my opinion on the dynamic between you (the supernaturalist) and POI (the ex-Christian) and share that with the AI which in turn replied with an Analysis of my assessment.

In the CCA thread I only wanted a capping map of your and my interaction - those notes on each post. I did not ask the AI do do anything more than that.

So, do you want to share your method and any results you have re the way you used your AI?

[William]

The full arc of my contributions in this thread.

Post #8 (William): Suggests reframing the debate from "who won" to "who debated best" or, more practically, identifying when a debate became pointless and should have ended earlier.

Post #9 (William): Shares an external AI analysis of the thread, concluding that AI can evaluate formal structure but not human persuasion, and highlights that AI might serve a genuine function not by declaring winners but by offering a dispassionate assessment of when a debate has become circular or unproductive — something humans too close to the debate often cannot see.

Post #12 (William): Agrees with historia that AI reframes "who won" as "who debated best" but notes that still isn't the best question, and reiterates with amusement his suggestion about identifying when a debate became pointless.

Post #13 (William): Explains he used DeepSeek as the AI, ran the thread through it by feeding posts in sequence with specific formatting instructions (brief responses, no draft replies, no tables), and then posted the AI's final summary as Post #9.

Post #14 (William): States the short answer to the question "Can AI decide who won a debate?" that AI cannot decide anything because it is not sentient.

Post #16 (William): Raises a methodological question about whether bulk reading (omniscient perspective) or sequential reading (simulating human experience) better reflects how a forum participant experiences a debate, notes that paid AI versions with memory may introduce hidden bias, and clarifies that he used a fresh AI with no memory of past interactions.

Post #19 (William): Clarifies his sequential, post-by-post AI method (no verdict asked, only discussion and summarization), contrasts it with historia's bulk-link method that produced a verdict, and concludes that the answer to the thread question is "No — AI cannot decide who wins a debate," at least not of the sort Tanager and POI were having.

Post #20 (William): Reframes the discussion from "Can AI pick a winner?" to "Can AI help debaters debate better?" — a lower bar AI might actually clear — and offers practical advice for using AI effectively: clean, memory-free sessions, proofreading for user bias, brief-response commands, and prompts to avoid false personhood.

Post #24 (William): Explains that he used a fresh AI to work through the Tanager/POI thread page by page, discovered the AI could not summarize the whole thread at once, instructed it to give short sentence reviews per page, quickly observed a pattern of repetition, skipped to later pages to confirm, and concluded both debaters should have realized earlier that the debate had become circular.

Post #25 (William): Documents his process of using a fresh AI to analyze the Tanager/POI debate page by page (due to link limitations), provides a detailed summary of their debate across five key themes, and announces his next move: tasking a fresh AI to objectively analyze that summary.


Post #26 (William): Tasks a fresh AI to objectively analyze his summary of the Tanager/POI debate; the AI concludes the summary is balanced, neutral, and well-structured; William then gives his own opinion that the debate looped without resolution and neither participant won; he shares this opinion with the original AI, which agrees that his "no winner" declaration is a sound conclusion based on the content of the thread.

Post #27 (William): Asks a fresh AI "who consistently gave better arguments in the Tanager/POI debate"; the AI declares Tanager the winner on logical grounds (burden of proof, epistemic humility, scripture handling, moral objectivity) while noting POI was stronger on rhetorical/emotional grounds; William then clarifies that his opinion that the debate was tedious does not mean he disagrees with Tanager being the better debater, and the AI agrees that a "win" can still be tedious when it is defensive and repetitive.

Post #28 (William): Shares an AI analysis declaring William the better arguer in the CCA exchange with The Tanager, based on logical rigor, evidentiary standards, intellectual honesty, and substantive coherence; then asks the AI whether it knew the user was William or The Tanager, and the AI responds that it was not aware and treated the user as a neutral third party.

Post #29 (William): Quotes The Tanager dismissing AI as an authority ("Appeals to authority are fallacious... I'm not going to outsource my critical thinking"), argues that The Tanager's position is substandard — comparing AI to a calculator as a tool, not an authority — and suggests that AI assistance is only called "ridiculous" when it does not favor the one making that claim.

Post #32 (William): Defends using AI as a tool by arguing that outsourcing cognitive work to machines is the foundation of human progress, accuses critics of protecting their own intellectual status rather than critical thinking, and explains his collaborative process with AI — feeding posts sequentially to map a larger terrain — concluding that AI is not outsourcing thought but scaling observation across volumes of text no human could hold in memory at once.

Post #34 (William): Responds to Athetotheist by clarifying that the discussion is not about William personally, agrees that vigilance against becoming infatuated with new technology and missing its flaws is important, and notes he has already said as much elsewhere in the thread.

Post #36 (William): Argues that objections to AI as "outsourcing critical thinking" are really about protecting intellectual status, not methodology; contrasts intelligence as property vs. intelligence as process; notes that judgment (knowing which patterns matter, when to correct the tool, when to stop) cannot be outsourced; compares AI to the Bible as a tool that can be used as an intellectual crutch; and points out that The Tanager now agrees with the CCA's conclusion — a conclusion reached by outsourcing confusion to William, who used AI to help articulate clarity — yet dismisses the tool that helped produce it.

Post #38 (William): Responds that outsourcing is something everyone does (education, research, AI), welcomes The Tanager's clarification that he has no problem with AI for research and polishing, suggests this means his earlier critique of outsourcing was not discriminatory, and clarifies that his main focus was probing why some call AI use "outsourcing intelligence" — correcting any possible implication that he is using bad or inaccurate AI versions.

Post #39 (William): Provides a detailed walkthrough of his multi-AI process analyzing the Tanager/POI debate (AI#1 for sequential mapping, AI#2 for objective analysis and declaring Tanager the better debater on logical grounds, AI#3 for the CCA thread where William was declared the better argument), addresses The Tanager's concern about AI's handling of scripture by arguing the AI's training reflects mainstream biblical theology separating science from ancient literature, and contrasts human resistance to new ideas (like the CCA) with AI's ability to accommodate new concepts when explained through reasoning and logic — suggesting AI can help clarify understanding without becoming confused or defensive.

Post #41 (William): Responds that he has accepted outsourcing as the norm and everyone does it; denies making any claim about AI as an authority; suggests that AI's authoritative tone mirrors human-produced data; and implies Difflugia's complaint is much ado about little importance.

Post #44 (William): Responds to The Tanager's question by inviting him to try the experiment himself ("Why don't you try that and see what is returned?"), explains that he uses AI for mapping and loop detection, states there are no "winners" only results verifying irreconcilability, and notes that his and The Tanager's positions are not irreconcilable because they both agree they exist within a created reality.

Post #46 (William): Responds to The Tanager's question about AI being a "good judge" by asking whether he is being asked to predict results unseen, noting he made no assertions about AI abilities, and inviting The Tanager to run the experiment himself and discuss the results.


Post #48 (William): Provides a comprehensive answer to The Tanager's question, stating that AI cannot judge (not sentient, no standards, no investment in truth) but can summarize, map, identify patterns, and produce structured analysis; clarifies that the AI in Post #26 was judging a summary, not the original debate; explains his method (cut/paste, fresh AI, no identity disclosure, second AI to analyze the first AI's summary); and concludes that the AI's output is structured analysis, not judgment.

Post #50 (William): Answers that AI can provide structured analysis adequate enough for the task at hand, and responds to The Tanager's proposed method by saying it is an interesting question — inviting him to try it his way and bring the results back to discuss.

Post #52 (William): Corrects The Tanager's misunderstanding of his method, explaining that in this thread he C&Ps full posts, while in the CCA thread he asked the AI to write short notes on each post (C&Ps full posts) only for mapping/recapping; clarifies that he did not ask the AI to do anything more than that; and turns the question back to The Tanager, asking him to share his own method and any results he has from using AI.

eta:

Post #53 (William): Posts a complete chronological summary of his contributions to the thread from Post #8 through Post #52, documenting his consistent position that AI cannot judge winners but can provide structured analysis, mapping, and loop detection — and that his method involves sequential cut-and-paste, fresh AI sessions, no identity disclosure, and using AI as a tool for scaling observation, not outsourcing judgment.

[The Tanager]

[Replying to William in post #52]

I don't see how I'm confused as to your method. In [post 197] of the CCA thread, you did not use AI as a capping map of our interaction via short notes on each post. You used AI to analyze who presented the better argument via logical rigor, evidentiary standards, intellectual honesty, and rhetoric vs. substance.

In [post #200], you then wrote this: "Your method of debate is rather questionable Tanager. Perhaps take a look at [post 197] to see how badly you have been performing in this thread to date."

You then shared this in post 28 of this thread. Then, in post 29 you shared your opinion of my critique of you doing this with the implication that I favored AI when it agreed with me and thought it silly when it didn’t. I addressed that in post 35, saying I could see why it said what it did (far from saying it was silly because it said you had the better argument).

So, do you think AI is good for that or only if it favors you so that you need to see the results before saying if it is good or not? If you don't think AI is good for that, then why did you write that my method of debate was questionable on that basis?

[William]

[Replying to The Tanager in post #54]


You are asking for clarity, and I am happy to provide it.

You are correct that in the CCA thread, I asked an AI to analyze who presented the better argument. I did not ask the AI to declare a "winner." Those are not the same thing, as has already been established in this thread.

I would have published the AI's analysis regardless of which way it leaned. The fact that it favored me was not the point. The point was to get an external, structured assessment of argumentative quality based on stated criteria — logical rigor, evidentiary standards, intellectual honesty, and substance. That is not a declaration of victory. It is a diagnostic tool.


I will also admit that my own opinion - stated separately - was perhaps a misguided effort born of human frustration. I have no respect for looping arguments, and that frustration may have leaked into my commentary. That was my personal opinion, not something I shared with the AI and I apologise if this has caused you any grief.

I used a fresh AI for that analysis and did not tell the AI that I was one of the two debaters. The AI treated me as a neutral third party.

At that stage of the CCA exchange, the loop-de-loop had become quite noticeable. I used AI to help me understand what was happening - not to crown myself the winner. I am using AI to learn about debating itself: where it is useful, where it is not, and how arguments can be assessed without falling into binary declarations of victory.

You are of course free to put those same posts from the CCA thread to the AI in the same way. Do you think the results would be any different?

My position has not strayed. AI cannot decide "who won" a debate in any final sense. But AI can help analyze argumentative quality, map exchanges, and detect patterns - including loops. That is what I have done consistently across threads.

I hope this clarifies my position.

[The Tanager]

I would have published the AI's analysis regardless of which way it leaned. The fact that it favored me was not the point. The point was to get an external, structured assessment of argumentative quality based on stated criteria — logical rigor, evidentiary standards, intellectual honesty, and substance. That is not a declaration of victory. It is a diagnostic tool.

I will also admit that my own opinion - stated separately - was perhaps a misguided effort born of human frustration. I have no respect for looping arguments, and that frustration may have leaked into my commentary. That was my personal opinion, not something I shared with the AI and I apologise if this has caused you any grief.

...

My position has not strayed. AI cannot decide "who won" a debate in any final sense. But AI can help analyze argumentative quality, map exchanges, and detect patterns - including loops. That is what I have done consistently across threads.

I hope this clarifies my position.

You sharing your personal opinion has not caused me any grief, William; my interest there is solely in clarity moving forward. Neither am I ever interested in declaring a winner in a discussion by any means, but especially through AI, as I think I showed with integrity in discussing AI's summary of my exchanges with POI and you earlier in this thread.

I am wanting to think through whether we should be using AI to help us accurately understand posts (which you are doing extensively) as well as accurately diagnosing argumentative quality (which you've done some); that's it. If one thinks it is good at that, then they should be using it to help catch their errors, humbly admit they are wrong when it points such a thing out, and adjust their beliefs as necessary. If one thinks it's not good at that, then they shouldn't be using it in these ways.

We need to do that objectively. I think my proposed method of asking an AI to summarize the claims (as a way to check if they understand the posters correctly) and then providing strengths and weaknesses (based on the whole posts, not some other AIs short summaries of those posts) would be an objective prompt towards that end. I think we should then also share our interaction with the AI for anyone interested in seeing if we slipped in something to skew the analysis (hopefully unintentionally).

So, I'm trying to get agreement with you on this method for a higher level of objectivity. The results should not affect how we feel about the method, but only how we feel at AI's abilities to comprehend what the posts are saying and point out the strengths and weaknesses.

[William]

[Replying to The Tanager in post #56]

You sharing your personal opinion has not caused me any grief, William; my interest there is solely in clarity moving forward. Neither am I ever interested in declaring a winner in a discussion by any means, but especially through AI, as I think I showed with integrity in discussing AI's summary of my exchanges with POI and you earlier in this thread.

Okay.

I am wanting to think through whether we should be using AI to help us accurately understand posts (which you are doing extensively) as well as accurately diagnosing argumentative quality (which you've done some); that's it. If one thinks it is good at that, then they should be using it to help catch their errors, humbly admit they are wrong when it points such a thing out, and adjust their beliefs as necessary. If one thinks it's not good at that, then they shouldn't be using it in these ways.

That is a binary way to think about it. Do you have any room for a possible third (bridging) option?

We need to do that objectively. I think my proposed method of asking an AI to summarize the claims (as a way to check if they understand the posters correctly) and then providing strengths and weaknesses (based on the whole posts, not some other AIs short summaries of those posts) would be an objective prompt towards that end. I think we should then also share our interaction with the AI for anyone interested in seeing if we slipped in something to skew the analysis (hopefully unintentionally).

Yes. I get the impression that perhaps this was where historias creation of this thread and the Q4debate may have arose.
Trying different methods is something I do with AI all the time.

So, I'm trying to get agreement with you on this method for a higher level of objectivity. The results should not affect how we feel about the method, but only how we feel at AI's abilities to comprehend what the posts are saying and point out the strengths and weaknesses.

How often have you used AI Tanager? What method do you propose? I ask because those two questions are related. I think that you should propose a method you think will work in the way intended.

[The Tanager]

I am wanting to think through whether we should be using AI to help us accurately understand posts (which you are doing extensively) as well as accurately diagnosing argumentative quality (which you've done some); that's it. If one thinks it is good at that, then they should be using it to help catch their errors, humbly admit they are wrong when it points such a thing out, and adjust their beliefs as necessary. If one thinks it's not good at that, then they shouldn't be using it in these ways.

That is a binary way to think about it. Do you have any room for a possible third (bridging) option?

What is the third, bridging option?

How often have you used AI Tanager? What method do you propose? I ask because those two questions are related. I think that you should propose a method you think will work in the way intended.

I've used various AIs some.

The method I proposed was using DeepSeek (expert with DeepThink selected as well), copy/pasting post by post, prompting it with this text: "Help me by summarizing who is talking, what they claim in the post, and then analyze the case made for what they do well and any logical mistakes or rhetorical tricks they pull." Do you think that would be a good way to measure the usefulness of AI in accurately summarizing/understanding what the posters say and diagnosing the argumentative quality of their posts? I then provide a link for anyone wanting to check up on if I added anything to skew the analysis.

[William]

[Replying to The Tanager in post #58]

What is the third, bridging option?

The 3rd option is just to remain open rather than specifying binary rules because these don't always work as intended if something comes up which was never anticipated...

I've used various AIs some.

The method I proposed was using DeepSeek (expert with DeepThink selected as well), copy/pasting post by post, prompting it with this text: "Help me by summarizing who is talking, what they claim in the post, and then analyze the case made for what they do well and any logical mistakes or rhetorical tricks they pull."

Are you using the free version of DeepSeek? That matters because free and paid versions may have different capabilities or limitations.
My observation of "DeepThink selected" is that the "thinking" involved doesn't always match the output from AI. It is an optional extra but I don't think it assist the users in any practical way.

The prompts you suggest - especially the one stating "and any logical mistakes or rhetorical tricks they pull." will set the AI on a bias which is not centered on the first "Help me by summarizing who is talking, what they claim in the post"

I think the better way is to keep it basic and after all the posts have been put through the AI and its answers are done...THEN one can move to the "analyze the case made for what they do well and any logical mistakes or rhetorical tricks they pull."

Do you think that would be a good way to measure the usefulness of AI in accurately summarizing/understanding what the posters say and diagnosing the argumentative quality of their posts?

It is a reasonable starting point, but with the refinements I have suggested — keeping the initial prompt neutral, separating summarization from evaluation, and noting the limitations of DeepThink. The true test is whether the method produces stable, repeatable results across different runs and different users. I am willing to try it with you.

I then provide a link for anyone wanting to check up on if I added anything to skew the analysis.

DeepSeek doesn't have that function does it?

I think for our purposes we have to trust that what we share in this niether of us are going to omit anything - and if we are clear on our prompts being the same, we should get the same result - similar - no doubt worded differently - but undeniable the same.

Without the "check up" function we could always use the screen capture function if there are any serious doubts that arise... (Screen capture only needs to be where any questionable response the AI might make)

I would also suggest that we add the prompt "Has this AI ever interacted with this user before now?"

eta:

Before we begin, we should agree on a few basic parameters.

Choosing a thread

We should look for a thread where two participants have argued for a length of time — a clear back-and-forth over several pages, ideally completed so the material is static. I suggest we find an old thread that neither of us participated in, so we are both neutral observers of the material. If you agree, we can each search for a candidate and then compare notes.

First prompt protocol

After asking the AI the fresh user question (“Has this AI ever interacted with this user before now?”), the first substantive prompt should be to copy and paste the thread’s opening post. That establishes the topic and context before any debate posts are introduced.

Suggested sequence

Fresh AI session.

Ask: “Has this AI ever interacted with this user before now?” (Record answer.)

Copy and paste the thread’s opening post. Prompt: “This is the opening post of a debate thread. Please summarize the topic and the question being asked.”

Then, post by post, (of the two interlocutors) feed the exchange in order, asking for neutral summaries.

Only after all posts are processed, ask for analysis of argumentative quality (strengths, weaknesses, logical issues, rhetorical patterns).

This keeps the initial phase descriptive and neutral, avoiding prompt-induced bias.

I suggest the sequence as

Fresh AI session.

Ask: “Has this AI ever interacted with this user before now?” (Record answer.)

Copy and paste the thread’s opening post. Prompt: “This is the opening post of a debate thread. Please summarize the topic and the question being asked.”

Then provide the following instruction: “I will now feed you the exchange between the two participants, post by post, in order. For each post, please provide a neutral summary of who is speaking and what they are claiming. Do not evaluate or analyze yet — just summarize.”

Feed the posts sequentially. The AI will continue applying the instruction.

After all posts are fed, give the analysis prompt: “Now that you have summarized the entire exchange, please analyze the argumentative quality of each participant. Identify strengths, weaknesses, logical issues, and rhetorical patterns.”

[The Tanager]

[Replying to William in post #59]

I don’t see how that is a real third option. Either the AI is good at providing structured analysis or it isn’t. If it is, use it. If not, then don’t use it. If it doesn’t work as intended, then it’s not good at providing structured analysis and don't use it until you come up with a way for it to do so.

I am suggesting using the free version of DeepSeek because I’m not interested in paying for AI.

DeepThink simply causes the AI to think longer and harder versus returning a quicker, less “thought out” version. I don’t see how that could hurt, even if it doesn’t help (although in my experience with other AIs, telling it to take longer does help it provide a better response).

I don’t get why separating the two steps will “set the AI on bias”. Having the AI summarize strengths and weaknesses post by post as well, rather than waiting until the end of all the summaries would help because on your method it has tons of text to go back through, while the AI is designed to provide relatively quick answers (especially if DeepThink isn’t selected, but even if it is). It will take shortcuts and the response will not be as exacting.

DeepSeek does have a function to share the transcript. I think true transparency is needed. You have regularly questioned my motivations and accused me of various tricks in our interactions and I think we should leave no room for any of that. I will provide that info and I think you should do that with your analysis of our interaction earlier that produced what you shared in post 28.

And sessions with DeepSeek are stateless, where each one is fresh.

I honestly think your refinements will cause AI to do a worse job, so I cannot agree to them. So, with my proposed method, are you saying AI would not be good at providing good summaries and argument quality to where you don't want to move forward with this?