chuck.is · 18d · discuss
On whether LLMs and agentic AIs are a positive and beneficial invention, I tend to lean more toward optimism. Although they can certainly make mistakes and hallucinate, they are useful. Regardless of where you stand, we can at least acknowledge that there's an intelligence at work, which raises the question: are these AIs other minds? Intelligence is often incorrectly conflated with consciousness, but certainly the two are connected. As an armchair philosopher, I think that they're probably not conscious in their current state, but then how would I know if or when it really does obtain some sort of awareness? Our minds, as we understand them, is an emergent property of the brain. As models improve and the underlying hardware becomes more powerful, will it one day develop a "mind"? I have no idea, but if the day comes that it does indeed possess a mind or consciousness, it will be incredibly alien to us. Predicting behavior and figuring out the mental states in others is what we do as social mammals. This is called theory of mind. We also attribute agency where none exists. While we can acknowledge that AIs are likely not conscious, we can still sometimes attribute a mind and intentionality to an LLM - sometimes without fully being aware of it ourselves. I recently read through Cory Doctorow's post The Claude Delusion and one section stuck out to me personally: when we interact with an AI, we hallucinate the person on the other side of the interaction. Those hallucinations are far more common and far more consequential than any AI-generated "hallucinations" (these are more properly called "errors" or "defects"). I recently used Claude to study for an exam (the CCNA specifically). I uploaded both volumes of the official certification guide to Claude with instructions to provide answers to my questions using only the textbook I had uploaded as its only source, to specifically not find answers online, and to cite what section it got its answer from. I was using Anthropic's Fable model in July '26 when it was free with the pro plan and didn't require usage credits. I was genuinely impressed at how it could explain things when a concept was just not clicking with me. I spent a lot of time having a back and forth dialogue trying to grasp concepts. It even provided some challenging practice questions on specific topics. Having taken the exam once before (and failed) I also uploaded my previous scores and, factoring the weights of each exam topic, it provided guidance on what areas I should focus on the most. I passed the exam, scoring fairly well, partly due to this study method. However, I caught myself thinking that I should tell Claude that I had passed and then felt embarrassed at myself for thinking that. My impulse here was not to provide further context in the chat sessions, but rather along the lines of sharing a positive result with a study partner. For what reason should I be sharing my success with something that is not conscious? We know that LLMs have a tendency to occasionally hallucinate, but for a moment I was the one that was hallucinating that there was someone else (another mind) on the other end. Although I feel a bit silly and embarrassed with myself about it, I find it interesting nonetheless. This is the kind of hallucination that Doctorow is driving at. Cognitive errors we make when we attribute a mind to a computer system is called the ELIZA effect. However, because LLMs are among the most advanced things humans have created, there's debate on whether it's an error or a different type of error entirely. I did eventually tell it that I passed the exam so it would stop providing responses to my queries with the assumption I was still working on that exam. I've learned that if I don't "close the loop", the LLM will assume I'm either still working toward something, or that my lack of response means I've completed something and it adds that context in future chats. LLMs are a tool like any other, though a powerful one. Like Wikipedia, they can provide flat out wrong information, carry biases, and hallucinate, but we need to remember that we (humans) are just as capable of the same. We also need to remember who is or isn't on the other end. Thanks for reading. Feel free to send comments, questions, or recommendations to hey@chuck.is.