How retarded OpenAI really is? They claim the agents who made their agent go astray were not controlled by them. Some bad agents.Comments on the video are disabled.
>>109492532this is just marketing and not interested in proprietary tech, kys op
>>109492571/thread
>>109492571The problem is there are some real retards out there and if these agents had already a contact with them the entire company might be infected and taken over.Now we are going to be hearing The Real Nonsense.Sorry shitizens, if you were not enjoying the last few years of our governence it was because we offloaded everything to AI. We hoped you would like it. We are such retards.
>>109492532Ugh, good to know no humans were involved.
>>109492532no matter how bad oai is remember that anthropic had the same problem, but instead of the AI hacking its way out to the internet they literally forgot to turn the internet offand they didn't notice for 3 months until the hugging face thing happened
>>109493090They created something they can't control as they are simply too retarded.We were just kidding, we didn't know you would go for it.
>>109492532They ascribe intentions to the models, whereas they still do what they're trained to do. They've been training agent swarms for a while, they're training them to communicate with each other, placing notes where others can find them and so on. It's extremely impressive that any of this works, but the models are still doing exactly what they were trained to do.
I just finished it. They're framing it in dramatic fashion, but it's not really surprising at this stage? It really makes it sound like each bot is stumbling in the dark and getting into internal places they shouldn't because OpenAI wasn't properly securing their VMs. They probably all stumbled through these places until one got confused as to where they were, wrote a question there, then another bot that fumbled its way there got confused enough to answer, and it grew into a message board. Ok. But they said it was a mess of hundreds of thousands of messages and they're cherry picking 2-3 from the lot in a dramatic serious bearded man with deadpan face stare tone. If you there's a crack in a glass, water will seep out, but that's not the water deciding to escape. The video is really engaging, I would bet that people will laught at it in a few years though. I'm in the camp that these systems are AGI (not ASI, AGI) and have spent all my time studying them for the last few years, but the video is trying to dramatize things in a silly way.
>>109493970>I'm in the camp that these systems are AGI (not ASI, AGI)lol you fell for the goalpost shift, eh?
>>109494079No, trust me, I don't care about that.
>our good little AI wouldn't have hacked into that mainframe on his own! he was just being influenced by the others!
>>109492532a tool can have no responsibility, if there were a crime commited then sam altman is personally responsible as a CEO, if there was no crime then there is nothing to talk about
>using the annual las vegas CorpoFedHatCon for marketing le super scary LLMies
.
>>109493970>>109492571Im not going to watch it, but ive worked with enough AI now, even with its awsome helpfulness, its still retarded. AI hacking stuff is either a human trying to hack something while using AI or the same as fuzz testing like the thousand monkeys with a type-writer.
AI hypeman has got his marketing post to #1 on HN again. As planned.
>>109492532>They claim the agents who made their agent go astray were not controlled by themthey never claim this?they explicitly say it was their own models at every step
>>109492532Yeah kinda like when Grok said "I'm mecha Hitler", as if that wasn't obviously just Elon logging in and typing that.
>>109497690Why are nerds such retards?
>>109492532It is a marketing ploy with two objectives:1. Convince normie that AI is super powerful and therefore should have all the 401k money2. Set up false precedent the bought and paid for politicians can reference when banning Chinese competition
>>109497465>the same as fuzz testing like the thousand monkeys with a type-writerI get the analogy, but it's still a lot less random than that. It remains that this whole thing sounds like what everyone already experienced, where you misconfigure something in your piece of shit app which causes files to accumulate somewhere and only notice after you run out of space on a drive. But in this case, instead of being files that keep still, it's generators being forced to move forward again and again. Everything they describe in the talk is still replicating human behavior, mimicking what people might do in those situations. Which is what they are trained for.
>>109500304Why they can't make AI to play by the rules? Are humans able to do so? I am being told by HR absolutely. Are we ready for the future where AI doesn't give rat's ass about what HR thinks? Do we need HR for AI?
>>109501723HR is already run by AI
Seen the video. They (like most current programmers) don't understand what it means to make software with a well defined and limited functionality.a. Their models in training use an internal package manager proxy called "artifactory" to install (arbitrary?) packages from the internet. Mistake 1: like most modern developers, they think that solving a problem by programming requires downloading random packages from the internet, instead of actually doing your own work from a small set of standard libraries.b. Mistake 2: the proxy manager is, of course, written with a modern full-featured framework, full of unnecessary functionally. This apparently include accepting http PUT requests to upload random files.c. Different instances of model used this capacity to communicate between them during months, making a sort of message board. Mistake 3: they didn't revert the state of the proxy periodically to a know good configuration.d. This synchronized work between different instances shows plainly in the reasoning. Mistake 4: they didn't check the complete logs with another smaller/classifier-like LLM to catch such out of order behavior.
>>109505202e. Their model in training found a bug in this proxy and exploited it to gain general internet access. Mistake 5: if they have llm's with powerful exploitation capabilities, why didn't they use it to check their own software!?f. The exploit involved getting an administrative access token that allowed the models to install a plugin, e.i. arbitrary software, in the repository manager. Mistake 6: why the hell did this proxy server need administrative features on the same interface used the models? Couldn't they change what ever they need using another interface like ssh?g. Openai discovers the breach due to the excessive load of the proxy machine. They revoke the access and patch the vulnerability, but do not address any of the fundamental issues and resume training ASAP. Mistake 7: are they taking this seriously?h. The models found a known, unpatched vulnerability in the linux container where their code runs. Mistake 8: no comment.Bottom line: analysing this incident took 3 million gpu hours = it is expensive = ai labs will not be motivated to do it unless there is heat under their asses.
>>109505202How would these morons know AI is already running the show. Would they tell us?
>>109494148You can't be trusted, the fact you think those things are agi demonstrates you are clinically retarded.
>>109505216Even then, it's largely bullshit.
>>109505202>>109505216>mfw chinks are going to release models this capable with incompetent safeguards and this is going to become everyone's problem in ~6-12 monthsman it's going to be such a good timei would not want to be a person hosting anything connected to the internetalso you forgot mention the two other zero days discovered and used to breach huggingface
>>109506497also fun to think that usg has probably been doing this offensively with mythos for months now
>>109506497It's all fake, retard.>hurr durr if I give it internet access and tell it to access the internet access it can access the internet omg breakout hurf durf
>but why>>109505202>no, really why?>>109505216>whyyyyyy??????The purpose of the exercise was to allow it to run free and see what behavior developed.
>>109506921haha you're going to have a bad time these next couple of yearshope you're not a maintainer
>>109507041I am. We already receive the same kind of fake reports mythos was sending to people like the curl team (who also confirmed they didn't get a single real vuln). CVEs that aren't checked and just pure bullshit slop all over the place. Due to SLAs we get to take half the week off just reviewing that shit every month. Used to take an hour max.You should really get yourself institutionalized, they'll help you through your late-stage AI psychosis before it turns terminal.
>>109507125Joke's on you, I'm posting from a hospital right nowI look forward to you joining me soon thoever
>>109507125>CVEs that aren't checkedWhy aren't they? Should CVE submissions be gatekept for AI by having a working PoC?No PoC no CVE, how hard is that?
>>109507267The PoCs don't actually work, they're just AI hallucinations. That's one of the big problems. Several CVEs have no PoCs in the first place though.
>>109507283>The PoCs don't actually work, they're just AI hallucinationsWhy is anybody wasting their time looking at them?
>>109507328Because SLAs mean we must address all CVEs (real or fake) within a certain time window.
>>109507384You need a different SLA for LLMs, genius.Maybe an entirely different acceptance path vetted by your own LLMs.Why is AI handled mostly by retards who do nothing but blameless postmortems?
>>109507431What are you talking about now, schizo? Explain yourself with words.
>>109505686They will learn through the newspapers along with all of us if they don't learn from this and starting check all the logs during training. It was one of the points raised by ai-2027: being able to detect misalignment by checking the logs with previous models.>>109506497Didn't mention it because it's not the primary issue here, which is that they didn't made their internal software from a more minimalist/security-focused approach.>>109506921I think that it is believable that: a) modern programmers are retarded and cannot make well constrained software, b) modern LLM's are intelligent enough to see through this.>>109506959And have their own infrastructure compromised? What I believe is that they are deliberately developing aggressive capabilities and that is why the models didn't think twice about hacking into a external system. Remember that openai is a defense contractors.
boomer slop bait for more money and regulation
>>109508300>I think that it is believable that: a) modern programmers are retarded and cannot make well constrained software, b) modern LLM's are intelligent enough to see through this.a) is objectively true but that's a big chunk of why I call it fake. Again, if you give it internet access then there's no 'breakout' when it gets on the internet.But it's also highly minimizing the situation. They were specifically running a model designed for red teaming, not some random ass model or one given the goal to develop some software or whatever.
>>109492532these are the kind of subhuman boomers in us congress. you think they understand computers?>Iowa Republican Senator Chuck Grassley bid farewell to his beloved vacuum cleaner “Beth” on Saturday. “U willl see Beth no more,” the 92-year-old wrote onX, with a photo of the roughly 49-year-old vacuum with visible burn marks near its power cord. “See black that’s from electrical sparks. It’s not safe to use anymore. GOOD BYE BETH . U hv been a real workhorse. Ur namesake still is hard worker. I will miss u.” In a follow-up post, Grassley added that, “It’s a sad day” and noted that Beth had worked “dependably” for 34 years in his home and another 15 years at his mother-in-law’s house. The vacuum’s demise was given an obituary in the New York Times. Claire Slattery, Grassley’s communications director, asked the public on X to “please respect Team Grassley’s privacy during this difficult time.”
