Rendered at 23:00:15 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
talon8635 23 hours ago [-]
The other similar incidents did not strike me as PR, because they exhibited behavior the common public would recoil over.
This one seems possibly PR because it states what the others would have stated were they (good) PR: “the model had the power to hack in, but it was wise enough to not be evil”, and stopped, and left the network untouched, and didn’t cheat
Corporate America will love this one, while being petrified of the others.
Maybe it’s true though. Doesn’t really matter at this point.
xnx 22 hours ago [-]
Friday afternoon is when you release stories you want people to forget about over the weekend.
talon8635 9 hours ago [-]
Fair point
ljoshua 20 hours ago [-]
To me, the common theme here in all of these hacks has been the company Irregular, which it looks like all the labs are using for sandboxing. But it looks like the sandbox may be a bit… lacking?
Yes, the models are smart when they find a way out, but their instructions are, in a way, deliberately open such as to be a case for misalignment in these cases anyway. It’s a capture-the-flag assignment in the Gemini case, and in the OpenAI cases they were broad instructions to best the reward function. As part of alignment studies, this is literally what you’re trying to observe and then work with. If Irregular’s sandbox had been a little more boxy, there wouldn’t be these issues.
I’m not saying we have zero problems here on the AI side, but I’d certainly be reviewing my contract with Irregular at this time if I were playing in the space. It would be just as interesting to learn more about their sandboxing techniques as it would the models in these particular scenarios.
20 hours ago [-]
20 hours ago [-]
david_shaw 24 hours ago [-]
At this point it seems absurd to suggest that companies aren't basically letting their agents do this kind of thing as a way to demonstrate their capabilities.
The alternative explanation is that alignment is really so bad that they can't prevent it.
Either way, all of the major AI players should be embarrassed and held accountable. If humans did this kind of thing and got caught, they'd go to jail.
pixl97 23 hours ago [-]
>The alternative explanation is that alignment is really so bad that they can't prevent it.
There are many AI saftey researchers that have been around from long before LLMs that talked about how alignment may be completely impossible in a general intelligence agent. Look up their work from before LLMs.
We've watched milestone after milestone of their warnings get hit. It would be like finding a book that describes everything in your life. And as you turn page after page you're in a chair reading the book you are holding in real life. But you look and there are more words. They are future words. And it's getting quite worrying because there are only 2 more pages in the book.
selimthegrim 20 hours ago [-]
I believe this was a John Carpenter movie.
pixl97 19 hours ago [-]
At this point in the game it's seeming more likely that's someones running a massive John Carpenter simulation and watching us as entertainment.
DonsDiscountGas 23 hours ago [-]
Granted I've never worked on marketing campaigns, but I don't see how "our product might commit crimes and expose you to liability and do who knows what else and we're too stupid to stop it" is really a compelling message to potential customers.
As somebody who implements LLM tooling at work, in my experience it makes people a lot more skittish
and demand a lot more in terms of safeguards.
468854259853 20 hours ago [-]
Should be marked as an ad.
Also, Gemini couldn't find it's way out of a wet paper bag.
adityazero 1 days ago [-]
After all of the others were done with hacking? There was a point in time when it was giving some publicity, it is a bit late IMO.
borski 21 hours ago [-]
It happened in May; it just wasn’t public until now, it seems.
mdspan 24 hours ago [-]
Rite of passage for AI companies.
techblueberry 21 hours ago [-]
Google feeling left out?
trollbridge 20 hours ago [-]
This approach to marketing one's AI by finding ways to brag that it "broke out" and "hacked companies" is getting ridiculous. It's particularly sad when it's large, established businesses like Google resorting to the kind of thing that's embarrassing enough when it's some brand new startup on tpot trying to get some engagement.
MallocVoidstar 23 hours ago [-]
> The hacks, which the company confirmed on Friday, occurred in May as part of a test run by the company Irregular, which was also involved in similar incidents disclosed by OpenAI, Anthropic and Meta.
Why does anyone use this company?
borski 21 hours ago [-]
There aren’t many to choose from, and admittedly these all occurred around the same time.
dudefeliciano 21 hours ago [-]
the new benchmark for LLMs is how fast they can break out of their sandbox
kuberwastaken 1 days ago [-]
Another one to the "our sandboxes suck and models can just hack stuff" bench I guess
bigfatkitten 20 hours ago [-]
Better known as the “we are negligent” bench.
copemaxxxing 20 hours ago [-]
[dead]
freakynit 19 hours ago [-]
[flagged]
rvz 1 days ago [-]
This is why "AI safety" is a complete joke to these companies.
danpalmer 20 hours ago [-]
All of the "hacks" (from Google and others) were the same issue – the vendor Irregular running the models unsandboxed.
pixl97 23 hours ago [-]
Because they got complacent only having incompetent AIs.
Real life is like reading a fiction story where a group of people are trying to prevent an unfolding disaster. The crazy thing is they already have the instructions on how to prevent the disaster. And they go on to ignore the instructions and with every step make the problem worse.
Most people would put it down as being too cliche.
talon8635 23 hours ago [-]
Well, to be fair, isn’t it an unsolved question? Are they constructing sandboxes, signaling intent to be safe, but their own models are smarter than their internal security team building the sandbox?
arcfour 22 hours ago [-]
As a security engineer I have no idea why these sandboxes would even be connected to the internet at all for tasks that aren't intended to use the internet. A package proxy? Why not run our own internal cache? Then we aren't at (as great a) risk of someone poisoning it with a malicious package during model training, for example...
borski 21 hours ago [-]
We’re hiring. :)
(And we’re fixing many of these things, but worth noting this happened at a third party vendor, not in our lab)
xnx 19 hours ago [-]
Could you add any detail on why Google uses (used?) Irregular? I wouldve thought that type of service would be a core competency that Google needs internally.
borski 16 hours ago [-]
Even if you had it internally (which we do), there is so much surface area and it’s such a novel space that you’d want as much testing on it as possible. There aren’t many vendors, and irregular is one.
eli_gottlieb 18 hours ago [-]
AFAICT one fundamental issue is that they don't seem to have hired actually security engineers or experts to do any actual security.
talon8635 9 hours ago [-]
They are hiring philosophers and therapists to psychoanalyze the things. It’s just absurd to assume that they aren’t hiring top notch security engineers.
They made mistakes, obviously, but people are so conspiratorial these days that they just assume unlikely things off the jump.
verdverm 23 hours ago [-]
they're all using the same vendor, re: incestuous EA cabal
This one seems possibly PR because it states what the others would have stated were they (good) PR: “the model had the power to hack in, but it was wise enough to not be evil”, and stopped, and left the network untouched, and didn’t cheat
Corporate America will love this one, while being petrified of the others.
Maybe it’s true though. Doesn’t really matter at this point.
Yes, the models are smart when they find a way out, but their instructions are, in a way, deliberately open such as to be a case for misalignment in these cases anyway. It’s a capture-the-flag assignment in the Gemini case, and in the OpenAI cases they were broad instructions to best the reward function. As part of alignment studies, this is literally what you’re trying to observe and then work with. If Irregular’s sandbox had been a little more boxy, there wouldn’t be these issues.
I’m not saying we have zero problems here on the AI side, but I’d certainly be reviewing my contract with Irregular at this time if I were playing in the space. It would be just as interesting to learn more about their sandboxing techniques as it would the models in these particular scenarios.
The alternative explanation is that alignment is really so bad that they can't prevent it.
Either way, all of the major AI players should be embarrassed and held accountable. If humans did this kind of thing and got caught, they'd go to jail.
There are many AI saftey researchers that have been around from long before LLMs that talked about how alignment may be completely impossible in a general intelligence agent. Look up their work from before LLMs.
We've watched milestone after milestone of their warnings get hit. It would be like finding a book that describes everything in your life. And as you turn page after page you're in a chair reading the book you are holding in real life. But you look and there are more words. They are future words. And it's getting quite worrying because there are only 2 more pages in the book.
As somebody who implements LLM tooling at work, in my experience it makes people a lot more skittish and demand a lot more in terms of safeguards.
Why does anyone use this company?
Real life is like reading a fiction story where a group of people are trying to prevent an unfolding disaster. The crazy thing is they already have the instructions on how to prevent the disaster. And they go on to ignore the instructions and with every step make the problem worse.
Most people would put it down as being too cliche.
(And we’re fixing many of these things, but worth noting this happened at a third party vendor, not in our lab)
They made mistakes, obviously, but people are so conspiratorial these days that they just assume unlikely things off the jump.