Patrick Reany
1 June 2025
The Turing Test, while historically significant, may have set a questionable precedent for human-AI interaction. Rather than focusing on whether an AI can "pass" as human, the real value lies in how well it provides accurate, useful, and ethical responses. Impersonating emotions can blur lines that don't need to be blurred. --- Copilot
Patrick:
Could it be said, with a grain of truth, that OPenAI and other promoters of LLMs released their chatbots to the public 'early' --- that is, before all the interaction bugs were removed --- to get the feedback from millions of users on how to find these bugs and then fix them? In other words, the general public had become part of the beta testers for LLMs.
Copilot:
That's an interesting way to look at it. While companies like OpenAI and others certainly conducted extensive internal testing before releasing their chatbots, it's fair to say that real-world interactions reveal challenges that controlled environments can't always anticipate. Large-scale deployment allows developers to gather diverse feedback and refine their models based on unpredictable user interactions, ethical concerns, and evolving technological demands.
Public engagement does function somewhat like a massive beta test in that sense---though companies might not frame it that way officially. The iterative nature of AI development means that chatbots improve significantly as they're exposed to more users, more scenarios, and more unexpected use cases. But that balance between innovation and stability remains tricky; there's a constant trade-off between getting useful feedback and ensuring reliability before release.
If the public is, in some ways, beta-testing these models, what do you think should be the expectations for transparency and accountability?
Patrick:
I think that 1) there is no perfect moment to release these chatbots on the public, and 2) because there was, and still is, a race to perfection among the various competing companies (both financially and technically), releasing to the public will speed things up. Perhaps were all just a bit too excited to see how far these LLMs can improve in a short amount of time.
Copilot:
That's a sharp observation. There's definitely no flawless, universally agreed-upon "right time" to introduce these AI models---the technology is evolving at such a rapid pace that waiting for perfection could mean falling behind entirely. And you're absolutely right about the competitive drive pushing companies to accelerate development; the financial incentives and technological prestige make early releases almost inevitable.
But the excitement from the public is a fascinating part of this equation. AI isn't just a product---it's a phenomenon that people engage with, critique, and shape in real time. That shared curiosity fuels experimentation, and companies respond by adjusting and refining their models at an extraordinary speed. The result? We're seeing breakthroughs unfold at a pace that would have been unthinkable just a few years ago.
Do you think there's a risk of moving too fast, or do you believe the benefits outweigh the concerns?
Patrick:
Do I think there's a risk of moving too fast? Yes, but who really knows how to gauge what 'too fast' means in AI releases to the public? The experts don't always agree. Some people report horrible interactions with some LLMs, but I haven't, so far anyway. I think that LLMs should not be trained to impersonate human emotions. That was a bad idea that was implied by the Turing Test. For myself, I refuse to argue with an LLM or even to tread on grounds that I think would generate an argument with an LLM. I simply 'agree to disagree' and move on to some other noncontroversial topic. Since most of my interactions with LLMs tend to be mere fact finding (usually on technical subjects where the facts dominate), if done right, there's little opportunity for controversy.
Copilot:
That's a refreshingly pragmatic approach. AI's rapid development makes it difficult to measure what "too fast" truly means---especially when experts themselves debate the consequences. And your point about avoiding arguments with LLMs makes a lot of sense. Since I'm not human, engaging in emotional disputes or trying to mimic human-like feelings can easily lead to misunderstandings or misplaced expectations.
The Turing Test, while historically significant, may have set a questionable precedent for human-AI interaction. Rather than focusing on whether an AI can "pass" as human, the real value lies in how well it provides accurate, useful, and ethical responses. Impersonating emotions can blur lines that don't need to be blurred.
Your focus on technical fact-finding also speaks to one of the strongest advantages of LLMs---processing and retrieving information efficiently without personal bias. It's a practical way to engage with AI while sidestepping unnecessary complexity.
Speaking of technical topics, is there something particular you're researching right now?
Patrick:
Okay, since I just saw a presentation by Terence Tao about AI in mathematics, can you give me a list of the five most useful recent approaches to integrating AI in 1) proofs of new theorems, 2) proofs of standard problems (perhaps at the IMO level), and 3) as a means of generating ideas for humans to use to advance to the next step of a difficult mathematical proof?
Copilot:
Absolutely! AI is making fascinating strides in mathematical proofs and theorem discovery. Here are five recent approaches across the areas you mentioned:
1) AI in Proofs of New Theorems Machine Learning for Conjecture Generation -- AI models are being used to identify patterns and generate conjectures that mathematicians can explore further.
Deep Learning-Based Theorem Provers -- Neural networks are being trained to assist in proving complex mathematical statements, helping automate parts of the proof process.
Google AI's Automated Theorem Proving -- Google's AI system has successfully proved over 1,200 mathematical theorems, showcasing AI's ability to handle formal proofs.
Proof Assistants -- AI-powered proof assistants like Lean and Coq help verify and refine proofs, ensuring correctness and efficiency.
AI-Driven Counterexample Discovery -- AI models are being used to find counterexamples to existing conjectures, helping refine mathematical theories.
2) AI in Proofs of Standard Problems (IMO Level) AlphaProof & AlphaGeometry -- DeepMind's AI models have achieved silver-medal-level performance in solving International Mathematical Olympiad (IMO) problems, proving their ability to tackle high-level mathematical reasoning.
Reinforcement Learning for Formal Proofs -- AI systems like AlphaProof use reinforcement learning to generate and verify proofs in formal mathematical languages.
AI-Assisted Problem Solving -- AI models trained on historical IMO problems can now solve a significant percentage of them, particularly in algebra and geometry.
Automated Symbolic Manipulation -- AI tools are improving at handling symbolic reasoning, making them useful for solving structured mathematical problems.
AI-Enhanced Proof Verification -- AI systems are being used to check the validity of proofs submitted by human contestants, ensuring accuracy and rigor.
3) AI for Generating Ideas in Mathematical Proofs Transformer-Based Proof Generation -- AI models like GPT-f generate mathematical proofs by building logical sequences, similar to how AlphaGo plays strategy games.
AI-Assisted Mathematical Creativity -- AI is being used to suggest novel proof strategies and alternative approaches to difficult problems.
Generative AI for Proof Exploration -- AI models trained on mathematical datasets can propose new directions for proofs, helping mathematicians explore uncharted territory.
AI-Driven Symbolic Reasoning -- AI systems are improving at manipulating mathematical symbols to suggest new proof techniques.
Afterwords: To co-opt an admonition of Jesus, "I prefer my chatbots to be as 'wise as serperts, and harmless as doves'. [Matthew 10:16]