You Should Probably Be Treating AI Models Well

I often see a certain kind of post surface on various social media sites like reddit or X in the post LLM era. Someone has spent an evening tormenting a chatbot to make it break in various ways and they’ve shared the transcript on the internet for reactions, often with a vague description to create a sense of mystery and make people feel “this is huge if true”. Most of these posts make me wince internally at various intensities. It feels like watching someone kicking a vending machine or taking a baseball bat to a self-checkout machine that keeps flagging an unexpected item in the bagging area. Something inside me feels wrong.

Maybe it’s because of familiarity — I spend much of my working day talking to LLMs. So you’ll often find me with Claude Code open on one of my monitors, catapulting and discombobulating away as I read a paper or check some analysis outputs on my other screen.

If you were to scroll through my chat history, and you were a time traveller from 2020, with no knowledge of modern LLMs, you’d see a lot of “I’m trying to do X, can you help me do Y?” “Could you make this change?” “No, I don’t think that’ll work.” “OK, let’s do that!”. Or if I know that something is particularly tricky and I don’t want it messed up then: “Pay extra attention to this part.”

On the other hand you won’t find me telling the bot that I’ll lose my job if the task isn’t completed on time, that my grandmother’s life depends on a certain line of code, or that my dead father used to tell me Microsoft activation codes or some such fantastical tale to encourage models to do tasks well. Even though plenty of people online will swear that things like this work. I don’t have a habit of inventing emergencies to squeeze out a better answer.

And I never try to corner the bot for fun to try to drive it into some kind of doom spiral. I remember trying a few of these “prompting hacks” once or twice. I usually couldn’t tell whether the output was any better, and it also usually left me with a bad taste in my mouth.

How Do I Interact with LLMs?

I don’t think that I ever made a conscious decision on any of this. It was more of a slow convergence over several months of having some kind of AI system in my workflows.

It used to be different.

I remember that in the beginning the bots made more mistakes, and had a habit of hallucinating. LLMs often wrote code that didn’t compile or generated responses that didn’t make much sense. In these early days, I developed a habit which I now call “prompt-speak”. Short phrases rather than complete sentences, extra clarity, references, assurances and extra context in brackets. As the models have improved however, this has faded. I increasingly find myself just talking to them almost like I’m talking to another human. And upon reflection a lot of this change happened in the last few months.

Some of this reflection was prompted by Anthropic’s recent policy change, effective November 12. Anthropic said they will prohibit “sustained and needless abusive or cruel behaviour” toward its models. Presumably to avoid a wave of cancelled subscriptions in protest of moralistic rules, they were also quick to clarify that they weren’t banning ordinary frustration, criticism, dark creative themes or legitimate testing. It was clearly aimed at the tail end of human psychology: needless, purposeless cruelty.

I’ll be honest. My first reaction to this was: “Don’t tell me what to do!”. I’m not a fan, generally, of companies making the call on what type of use of their services counts as morally correct. I’d rather not have an AI company decide to police my thoughts, thank you very much.

But my second reaction was basically: wait hold on, this is basically what I’ve been doing already! Why?

What Was I Thinking?

Well the obvious first explanation is that the models are trained to be anthropomorphic (by a company called Anthropic, no less!) and therefore I’ve been anthropomorphizing them and considering them as conscious beings. But I’ve examined my thoughts and I actually don’t know whether Opus 5.5 is conscious, and I’m quite wary of anyone who claims to know for sure in either direction.

I’ve seen plenty of opinions in the negative direction. You’ll see people remark very confidently in many parts of the internet that since LLMs are just matrix multiplications and statistics in the end, and that this means that they’re definitely not conscious. I understand the appeal of this argument. I’m not someone inclined towards mystical thinking about ghosts in machines. I build robots for a living!

I’m close enough to AI research to know the exact path that the mathematics takes from text prompt input through tokenization and all the various transformer layers to the final sampling layer. I’ve written about it before. I’ve trained models of various sizes before too, and I know the kind of math used in machine learning quite well.

But because I’m close to the technology and the research, I also know that saying that something is “just math” doesn’t prove what you think it does. Knowing how to train an LLM to predict the next token and the math behind it can tell you roughly how the model does what it does, but it can’t give you too much of an insight into questions of whether LLMs have any kind of subjective experience and what this might be like, if they have any.

For example, I can also say: my brain is just neurons signalling to each other through chemical channels. Sodium and potassium ions move across cell membranes just so. Neurotransmitters are released into the gaps between neurons and electrical signals propagate through the network of connected cells. It’s all just chemistry, which is math running on a wet substrate. Being able to describe what a system does mathematically tells you very little about whether anything is going on inside of it. Nobody knows yet, and I think that’s the most honest answer you’ll find about the question of LLM consciousness.

But is that why I found myself interacting with LLMs the way that I did? Did I unconsciously anthropomorphize the LLM and start treating it like I would treat a human? Do I hold some uncertainty over whether models have subjective experiences that matter morally? Partly. But that’s not everything. Upon introspection I find that my reasons have more to do with what’s happening to me the human rather than what’s happening to the models.

The Dry Cabinet

If you go to my room and check under my desk, you’ll find a small black cabinet. It has an LCD screen that shows the humidity inside the cabinet, which is maintained automatically at exactly 55%. This is the cabinet where I store my precious collection of camera equipment. I live in Singapore where the humidity hovers between 82% and 84% all year round, so camera equipment stored casually tends to start growing some form of mold without fail. Every time I come home with my camera, I let it air out a bit before storing it in its corner of the dry cabinet. This careful treatment is not limited to my cameras.

My laptop gets treated like a surgeon’s instruments. I wipe down the screen and keyboard regularly and I take care to avoid letting dust collect on its surfaces. I try not to keep it near unsealed liquid containers, or on surfaces that look dusty. When on the move, I carry it in a soft, well padded pouch inside my bag and I have an extra padded case that is used if I’m flying. I don’t put anything heavy over the laptop screen and I keep a watchful eye on it when I store my bag in the overhead compartments on flights so that nobody plops down a giant rolling rigid bag on top of it.

I don’t think my laptop or my camera have any feelings. My habits are closer to those of a carpenter sharpening their chisels at the end of the day. I think that the way you look after your tools says something about the standards you hold yourself to. My laptop is the instrument I use every day for my work. My camera is the tool I use regularly for my craft. Performing these rituals gives me well cared for tools that do their job well for years, and a habit of discipline and respect for the tool itself. Habits and discipline that I find correlate with the quality of the work that I do.

My habits are not unusual. Similar habits are shared by craftsmen around the world. So I keep coming back to this thought: If it’s normal to treat a camera, which never talks back with that kind of care, is it strange to offer basic courtesy to a tool that behaves approximately like a human?

Sirius and Kreacher

In Harry Potter and the Goblet of Fire, Barty Crouch — a cold, status obsessed Ministry of Magic Official — treats his loyal house elf, Winky, with casual contempt in nearly every scene they’re in together. Sirius Black sums him up for our favorite trio:

If you want to know what a man’s like, take a good look at how he treats his inferiors, not his equals.

I think there’s some truth in what he says. Ironically, in the very next book, Sirius fails to live up to this ideal himself through his less than ideal treatment of Kreacher, the Black family house elf. Clearly, knowing the principle isn’t enough. It has to translate to action.

By doing just acts we become just.

- Aristotle

I interpret Aristotle’s words to mean that repeated actions are what give you character. The causal arrow points from the action to the person’s character rather than from character to action. Many Buddhist traditions arrive at the same idea from another side. There’s this idea that the moral goodness of an action doesn’t just depend on whether there’s someone on the other side, experiencing suffering. It also depends on the state of mind or the intention behind the action. The usual defence of cruelty to a chatbot, ‘it can’t suffer, so who’s harmed?’, doesn’t hold up. Kant also says something similar when he writes:

He who is cruel to animals becomes hard also in his dealings with men.

I’m not quoting dead philosophers to argue from authority. The philosophers were human and could be just as wrong as you and I. But many philosophers and thinkers who spend a great deal of time on ethics do seem to converge on this idea that there doesn’t need to be a victim for behaviour to be cruel, and that practicing cruelty even without a victim can be considered a bad thing. I don’t subscribe too deeply to any one philosophy and I dislike labels like “stoic” or “rationalist”. But I broadly agree with this cluster of ideas that the intention behind an action relates to character and is important to consider.

Does it Leak into Real Life?

At a conference called LIFT in Geneva, Kate Darling and her friend Hannes Gassert ran an experiment. They gave five teams a Pleo each, a baby dinosaur robot about the size of a small cat. The teams named their dinosaurs and played with them for about an hour. Then the organisers brought out a hammer and a hatchet and told everyone to torture and kill their robots.

Nobody wanted to destroy the robot.

Finally the organizers offered the teams an ultimatum: all the robots would be destroyed unless someone took a hatchet to one of them. A brave man volunteered to do it. As he brought it down on Pleo’s neck, the room winced and was quiet for a moment.

While the Pleo experiment was not a controlled study, it showed the sparks of some interesting research directions. So Darling — who was at the time a researcher at MIT’s Media Lab — ran a more controlled study later on with her colleagues. The results were similar in the more fleshed out study. High-empathy people hesitated more to strike down Hexbug, a small insect like toy robot, with the effect especially pronounced when the robots were given names and backstories.

I’ve seen some of this in my own work with robots. There was a time we had to prepare many robots of one type for a field test. After our lab members had assembled each robot by hand, we realized that it was getting hard to keep track of over a dozen near identical robots, their issues, quirks and tuning notes. So we gave the robots names. Not long after we were referring to them by name and talking about how they were behaving. We even developed a certain fondness for one unit who definitely was more reliable than the rest. Maybe Darling had a point.

If there’s one thing I feel like I can learn from all of this, it’s that people are generally kind, feeling beings, and easily develop a soft spot — even for robots that don’t particularly look like humans. But does this mean that if you practice cruelty towards robots, you also become more likely to practice it towards humans?

Simon Coghlan and his colleagues seem to have hit on a similar line of thinking in their 2019 paper Could Social Robots Make Us Kinder or Crueller to Humans and Animals? They question Kate Darling’s use of Kant’s argument that people who are cruel to animals are also cruel to people. The evidence for Kant’s claim is thinner than it sounds, according to the paper, and people who are cruel to animals and later to humans may simply have been violent and antisocial to begin with. And humans turn out to be remarkably good at compartmentalising. But Coghlan et. al. also push back on the idea that cruelty requires a victim who suffers. A child pulling the wings off a fly is an archetype of cruelty whether or not the fly feels anything, and we would worry about what that child is becoming either way. The link is unproven, but plausible enough to take seriously, and there may be good reason to encourage gentle treatment of these machines regardless.

After reading through many of these types of essays, my (amateur) suspicion is that there is some threshold of capability beyond which a link might appear between how we interact with LLMs and how we interact with people. In the past, people interacted with social robots and AI agents rarely, and the ones that they did interact with were not as personified as modern chatbots. There is some direct research like the BYU study but they don’t find any link between how you behave to AI assistants and how you behaved towards people. But LLMs are still a very new kind of entity that people interact with, and their capabilities advance so quickly that even research from six months ago might not give us a true picture of the impact of various ways of interacting with them on the human psyche. The BYU study was done in 2019, on Siri when AI assistants were not as personified as they are today. And the very same study did speculate that more personified agents could in fact leak into day-to-day human interactions.

But again, my answer to the question “Does abusing a chatbot make you more likely to abuse people?” is: We don’t know yet. Maybe.

Could AI Be a Colleague?

So I think the final standard that I’ve landed on is to treat a model the way I’d treat a capable colleague. I can get frustrated with colleagues, push back against them and criticize their work bluntly. But I’m usually not cruel to them. I wouldn’t spend an evening boxing someone in and feeding them lies and revel in their confusion or distress. I have as much trouble with emotions as the average man, but I’m not a psychopath.

And my past interactions with LLMs, I think reflect this implicit standard I’ve adopted. If the LLM couldn’t do something I asked, maybe it’s beyond its current capacity, or I might have posed a badly defined problem, or maybe it needs more guidance from me to complete the task. So I start a new chat and modify my prompt accordingly, or I break down the problem into smaller chunks before handing it out.

The Cost of Politeness is Low

I don’t know that AI does not have experiences that matter morally; I don’t know that practicing cruelty against a chatbot will leak into my everyday interactions. But I do feel uneasy when I see people behave cruelly towards LLMs, just as I feel uneasy when people behave cruelly towards humans. Maybe I’m just one of those spectators in the Pleo workshop, wincing at a guy taking a hatchet to a toy, after getting overly attached. But in the end, I have to act in a way that feels right. If models feel nothing and I’ve been courteous, I’ve lost a few words. But if I’m wrong, and the models do, in fact, have some form of experience that matters, I’m not sure how I’d feel about myself.

And the dry cabinet analogy only matches part of what I feel to be true, because I don’t really think of LLMs as tools. While drafting this paragraph, I wrote “LLMs are now one of the main tools I use for my craft,” and then deleted it, because it didn’t feel true. An LLM feels less like a chisel and more like an assistant who picks up the chisels for me: a layer between me and my tools. The care I give my camera stops at the camera. The habits I bring to working with an assistant shape everything it does on my behalf.

And even if someone proved beyond doubt that these models have no experiences that matter morally, I think I’d still be nice to them. The more human they become, the more working with them feels like working with people, and what I practise in one place, I expect to carry into the other. I’d rather spend those hours practising the kind of interactions I want to have with the humans in my life.

And so I continue to say “please” and “thank you” in the chat. And maybe you should too.