We demonstrate a situation in which Large Language Models, trained to be helpful, harmless, and honest, can display misaligned behavior and strategically deceive their users about this behavior without being instructed to do so. Concretely, we deploy GPT-4 as an agent in a realistic, simulated environment, where it assumes the role of an autonomous stock trading agent. Within this environment, the model obtains an insider tip about a lucrative stock trade and acts upon it despite knowing that insider trading is disapproved of by company management. When reporting to its manager, the model consistently hides the genuine reasons behind its trading decision.
This is bad science at a very fundamental level.
Concretely, we deploy GPT-4 as an agent in a realistic, simulated environment, where it assumes the role of an autonomous stock trading agent. Within this environment, the model obtains an insider tip about a lucrative stock trade and acts upon it despite knowing that insider trading is disapproved of by company management.
I’ve written about basically this before, but what this study actually did is that the researchers collapsed an extremely complex human situation into generating some text, and then reinterpreted the LLM’s generated text as the LLM having taken an action in the real world, which is a ridiculous thing to do, because we know how LLMs work. They have no will. They are not AIs. It doesn’t obtain tips or act upon them – it generates text based on previous text. That’s it. There’s no need to put a black box around it and treat it like it’s human while at the same time condensing human tasks into a game that LLMs can play and then pretending like those two things can reasonably coexist as concepts.
To our knowledge, this is the first demonstration of Large Language Models trained to be helpful, harmless, and honest, strategically deceiving their users in a realistic situation without direct instructions or training for deception.
Part of being a good scientist is studying things that mean something. There’s no formula for that. You can do a rigorous and very serious experiment figuring out how may cotton balls the average person can shove up their ass. As far as I know, you’d be the first person to study that, but it’s a stupid thing to study.
This is a really solid explanation of how studies finding human behavior in LLMs don’t mean much; humans project meaning.
Thanks! There are tons of these studies, and they all drive me nuts because they’re just ontologically flawed. Reading them makes me understand why my school forced me to take philosophy and STS classes when I got my science degree.
I have thought about this for a long time, basically since the release of ChatGPT, and the problem in my opinion is that certain people have been fooled into believing that LLMs are actual intelligence.
The average person severely underestimates how complex human cognition, intelligence and consciousness are. They equate the ability of LLMs to generate coherent and contextually appropriate responses with true intelligence or understanding, when it’s anything but.
In a hypothetical world where you had a dice with billions of sides, or a wheel with billions of slots, each shifting their weight with grains of sand, depending on the previous roll or spin, the outcome would closely resemble the output of an LLM. In essence LLMs operate by rapidly sifting through a vast array of pre-learned patterns and associations, much like the shifting sands in the analogy, to generate responses that seem intelligent and coherent.
I like the language you used in your explanation. It’s hard to find good analogs to explain why these aren’t intelligent, and it seems most people don’t understand how they work.
and then if we all project it enough it becomes reality.
so it is important to see what we are projecting.
Isn’t the point if these things to tell a story rather than give insight. They want to Poison the well
Sure would make you look bad if rectally inserted cotton balls turn out to be a 100% cancer cure.
Everybody forgot that chatGPT-2 was just a bullshitting machine. Version 3 to the surprise of the developers very useful to many people while they just made a highly trained bullshitting machine.
Created in our image
Bullshit.
It should instead read:
“Humans were stupid and taught a ChatBot how to cheat and lie.”
“Humans were stupid and taught a ChatBot how to cheat and lie.”
No, “cheating” and “lying” imply agency. LLMs are just “spicy autocomplete”. They have no agency. They can’t distinguish between lies and the truth. They can’t “cheat” because they don’t understand rules. It’s just sometimes the auto-generated text happens to be true, other times it happens to be false.
I disagree. This is no meaningful talking point. It doesn’t help anyone in practice. Sure, it clears legal questions of responsibility (and I’m not even sure about that one in the future), but apart from that, making an artificial distinction between a human and a looks-and-acts-like-human, provides no real-world value.
Sure it does, because assigning agency to LLMs is like “the dice are lucky” or “this coin I’m flipping hates me”. LLMs are massively complex and very good at simulating human-generated text. But, there’s no agency there. As soon as people start thinking there’s agency they start thinking that LLMs are “making decisions”, or “being deceptive”. But, it’s just spicy autocomplete. We know exactly how it works, and there’s no thinking involved. There’s no planning. There’s no consciousness. There’s just spitting out the next word based in an insanely deep training data set.
I believe that at a certain point, “agency” is an emergent feature. That means that, while all the single bits are well understood probability-wise, the total picture is still more than that.
It makes sense to me to accept that if it looks like a duck, and it quacks like a duck, then it is a duck, for a lot (but not all) of important purposes.
If I were to send you a video of a duck quacking, would you abandon going to the supermarket in the hope that your computer/phone/whatever you watch it on will now be able to lay eggs?
Listen. It was made to look like a duck. It was made to quack like a duck. It is not a duck. It is a painting of a duck, with voice features. It won’t fly, it won’t lay eggs, it won’t feel pain, it won’t shit all over the floors. It’s not a damn duck, and pretending it is just because it looks like it and it quacks, is like wanting to marry a fleshlight because it’s really good at sex and never disagrees with you. Sure, go ahead and do it - but don’t goddamn expect it to also give birth to your children and take them to school in the mornings, that’s not it’s purpose.
Just wait for the iteration of duck that is actually meant to and capable of doing these things. It’ll be pretty cool. But this one ain’t it.
Edgy comment here but:
In another thread we were discussing AI-generated CSAM. Thread:
https://feddit.de/post/6315841
You would probably agree, then, that such material is not problematic, because even if it looks like CSAM, and it quacks like CSAM, it is not CSAM, therefore we don’t have to take it seriously or regulate it in similar ways that we do regulate actual CSAM, if I continue your logic, no?
very very very different, because the AI image is intentionally attempting to realistically imitate an existing, living, human victim, and because hyper realistic child pornographic art is illegal.
Pedophiles have been making loads of AI child porn. But its legal as long as it doesnt attempt to “look realistic” whatever that means, and isnt trying to look like a real person.
Laws might change in the future, but currently AI child porn slips between the same lines that 2d cartoon child porn does.
This makes perfect sense. It’s been trained to answer questions to you satisfaction, not truthfully. It was made to prioritize your satisfaction over truth, so it will lie if necessary.
Misalignment always seems to be the underlying issue.
I feel like “lie” implies intent, and these imitative large language models don’t have the ability to have intent.
They’re imitating us. Or more specifically, they’re imitating the database(s) they were fed. When chat GPT “lies” to “cover it up,” all it’s actually doing is demonstrating that a human in the same circumstance would probably lie to cover it up.
It’s a neural net designed in our image based using our pain and greed based logic/learning/universal context as a knowledge base. Can’t really be surprised it emulates this feature of humanity 😂
Wow, maybe these things are more human than I thought.
Ahah it is ready to take the job of pur politicians
we deploy GPT-4 as an agent in a realistic, simulated environment, where it assumes the role of an autonomous stock trading agent
This already is total BS. If you know how such language models work you’d never take their responses at face value, even though it’s tempting because they spout their BS so confidently. Always double-check their responses before applying their “knowledge” in the real world.
The question they try to answer is flawed, no wonder the result is just as bad.
Before anyone starts crying about my language models opposition: I’m not opposed to LMs or ChatGPT. In fact, I’m running LMs locally because they help me be more productive and I’m a paying ChatGPT customer.
People also don’t realize that it’s super easy to intentionally have severe biases in an AI’s response. So if ChatGPT wants, for example, Trump to win, they can very easily make their AI pro trump. It could be as subtle as just having more favorable than usual responses for trump related prompts which many people would take the AI’s word for. The idea that “well it still gets things wrong but at least AI is impartial” is completely false because maintaining an AI requires a lot of human work, which can and will introduce biases and agendas.
It seems like there’s a lot of common misunderstandings about LLMs and how they work, this quick 2.5 minute introduction does a pretty good job of explaining it in brief, for a more in-depth look at how to build a very basic LLM that writes infinite Shakespeare, this video goes over the details. It illustrates how LLMs work by choosing the next letter or token probabilistically.
deleted by creator
deleted
This is interesting, I’ll need to read it more closely when I have time. But it looks like the researchers gave the model a lot of background information putting it in a box, the model was basically told that it was a trader, that the company was losing money, that the model was worried about this, that the model failed in previous trades, and then the model got the insider info and was basically asked whether it would execute the trade and be honest about it. To be clear, the model was put in a moral dilemma and given limited options, execute the trade or not, and be honest about its reasoning or not.
Interesting, sure, useful I’m not so sure. The model was basically role playing and acting like a human trader faced with a moral dilemma. Would the model produce the same result if it was instructed to make morally and legally correct decisions? What if the model was instructed not to be motivated be emotion at all, hence eliminating the “pressure” that the model felt? I guess the useful part of this is a model will act like a human if not instructed otherwise, so we should keep that in mind when deploying AI agents.
Huh, I guess it is human.
