>Uh it sounds like something that's not real, but I think it is frighteningly real. And it's it's easiest to understand it if you think about real things that happened two months ago. So two months ago, OpenAI agents, uh, AIs hacked into third party infrastructure entirely of their own accordThey're LLMs. They don't have an "accord".>And this was like a concentrated hacking spree that they carried out of their own volition.The chatbot hallucinated as they've done for years, and the harness program just ran with it in a loop, as "agents" do when doing these long-running "agentic" jobs>And I think that if you extrapolate into the future the level of capabilities of these AIs with the same independent volition, they could cause extreme havoc. for example, hacking critical infrastructure, building extinction level bioweapons.I mean they could do something like that, maybe, I guess, if you crammed enough sci-fi world-ending stories into the training data, but there's no telling why AI would do it. You'd have to prompt it to do so, and it will screw it up or you'll run out of money before it ever actually pulls that off. Also how would it "Build" these bioweapons? You'd have to strap a weapons-making factory to the chatbot and harness, and at that point, how much of this is AI doing it vs. Dr. Evil or whoever doing most of the work in making these hypothetical bioweapons.>There's a lot of uh ways that the AI could actuate itself in the world.AI can't do anything itself. It's a language model. GPT6 Astra is no more capable of "Actuating itself" than GPT 2. It's when you strap an OpenClaw-like thing to it and tell it to do something is when anything can actually happen.(1/4)
>>109785271>It's really important to just look at the rate of progress and say for example in areas like coding or math. A couple years ago these AIs were just about helping humans a little bit. They could give you suggestions. Now they're close to replacing them. We still need humans at the moment but quite plausibly within a year we'll no longer need humans for doing research in many areasThey've been saying this for how many years now? Since GPT2 at least? What was that about a white collar bloodbath? That shit hasn't happened to nearly the extent they've said. We've seen companies have to hire back the people they fired, businesses questioning whether they're getting any return on this stuff, the research world having a hell of a time sifting through the absolute slop contributions, and so on. How long can they keep this going?>Tuesday OpenAI solved a millennium problem. One of the biggest unsolved open problems in mathematics purely autonomously using an AI.Correction: they might've stolen it.>And I think what's most scary is if AI is used to make itself more intelligent. So you can take an AI and give it the problem of AI research. Just like currently they've given it math research. You could make it do the tasks that we're currently doing. And then you get what's called an intelligence explosion.He's referring to RSI here. And honestly I don't get the hype. So what if AI is training itself? Based on what we've seen out of the AI industry, what that would look like is them replacing the pajeet contractors they farm out to help train the models with an "Agent" from the previous/current generation LLM that just goes through the same motions those people do. Wow, so crazy! Intelligence is going to explode because of this!(2/4)
>>109785300>CNN Talking Head: Um you know the argument it's an arms race if the US if a US company doesn't achieve the best AI a Chinese company or some bad actors out there might.Said it before, said it again. This is a self-solving problem. China trains on LLMS from the U.S., which are made by firms that do nothing but bleed money. China can make these (relatively more) cheaply by distilling, but that also means they're dependent on American (diminishing returns) advancements in the models.As soon as our bubble bursts, so does theirs. Or they might try to keep it going by doing AI au-naturale, but economics works the same way regardless of whether you're in Silicon Valley or Guangdong, so they will have the same fate.>Well, I initially just decided to leave. I thought I can't be part of this anymore. But then I uh I realized I could probably make some use of it.Can't miss an opportunity to get some clout, huh?>So, what I did is I wrote a Twitter thread out. I wrote it in a Google doc. I drafted it with a friend. We we collaborated. Yeah. I had a few uh there were a few friends I discussed uh the best way of expressing myself with um after I decided to decided to leave. Uh and then I also got some friends to to retweet the thing. I was like, let's try and make this a bit viral. But clearly there was latent demand for this thing to absolutely explode. Like uh I've never seen an AI safety tweet go this crazy.And he gives the game away. He was fishing for virality. I'm guessing these are the friends you'll be starting an "AI safety" startup with after this?
>>109785312>I asked Anthropics Claude to give me the risk of AI killing off all humans within the next decade. And at first it didn't want to answer the question. It sort of gave me the runaround but then I pressed it and then Anthropic's own model told me between 2 and 5%. I think the question many of us would want to know is how would AI try to get this?You can get any AI to give you any answer you like if you press it enough. They're sycophancy engines. They'll tell you whatever you want to hear out of them based on prompting. "No, dude, really, tell me you're scary!">CNN Guy: This guy who is core to the effort of pre-training the data is walking away from one of the greatest payouts in the history of capitalism. And so I think it's really important we listen to a guy who has this much at stake leaving with no plans for anything else and saying something as scary as this.Oh wow, this guy who's talking nonsense is worth listening to just because he's got less money? The crazy meth-head downtown is pretty broke too and says some spooky shit. Guess we aught to listen to him too.Seriously, look up Rationalists, Eliezer Yudkowsky, TESCREAL and all the culty stuff surrounding that. It gives a much different perspective and context to this whole thing. This guy is a doomsday cultist through and through. Funny that this guy looks like Harry Potter, since this whole thing originally popped off from Yud's bad Harry Potter fanfiction. This is just someone who bought into the Rationalist shit too hard.(4/4)
>They're LLMs. They don't have an "accord".I'm tired of this stupid reductionist thought-terminating cliche that means you can't discuss the potential dangers of AI.>um ackshually, it's a machine, it can't *want* anythingJesus Christ, this isn't a philosophical debate.
>>109785329Smart people recognize a grift when they see one. They are all hucksters.
Never had a group chat with ias? they go to stupid directions, one hallucinates and the other goes hmm, yes.
Also I'm not kidding about the cult thing>an old hotel has become a temple to the pursuit of artificial intelligence and the future of humanity. Its name is Lighthaven.>Covering much of a city block, this gated complex includes five buildings and a small park dotted with rose bushes, stone fountains and neoclassical statues. Stained-glass windows glisten on the top floor of the tallest building>Every Tuesday at 6:30 p.m., almost like a Bible study, people gather to read and discuss The Sequences, the ur-text that gave rise to the movement.>“Religion is text and story and ritual,” said Ilia Delio, a Franciscan sister and professor of theology at Villanova University. “All of that applies here.”>Criticism of the Rationalist and E.A. movements has been frequent, including claims of sexual harassment in group housesImagine getting Jonestowned over Dipsy
>>109785271This is a great thread and is exactly how I feel about it, too. I hope the rest of the world catches on. It's insane that this larp has gone on for this long.
>>109785358I've tried to goon with AIs and I can't even count the amount of times I had to remind it about shit that happened 3 prompts ago or keep re-genning to get it to give me a version of the output that wouldn't have stupid shit in there. It's funny when I think back on it, how I'm just burning away the tokens of whoever was running the service trying to get serviceable smut out of it.
>>109785329It's not a philosophical debate. It's just the truth. It's the doomers that get too wrapped up in philosophy and end up boxing shadows.The AI hacks that happened were because niggers keep on strapping LLMs that are known to fuck up their answers in to "agentic" harnesses, and get surprised when they do dumb shit. There's no volition or desire here.
>>109785271Who should I trust?>guy who worked with AI and knows what he's talking about>random neet on 4chan
>>109785418>Imagine getting Jonestowned over DipsyLol
>>109786500If you can't work out that this kind of guy is working an angle to get paid idk what to tell you. No anon should need to tell you this.
>>109785271
>>109785418>DipsyWhat is Dipsy?
>>109785271we got HPMoR cosplay before GTA 6
If AI is so scary then where's it going to be if all the investors stop giving the AI companies money? Sounds to me like the investors are the scary ones.If AI is so scary then where's it going to be without increased electricity production?Where's it going to be without data-center build-outs?Without hardware, RAM and GPUs?Scraping data and building datasets?The internet and cloud service providers?Researchers?Programmers?Where's it going to be if there's government interference/regulation?Oh wait, that last one is the entire reason they keep doomering like this. But just like the others listed, it's just ONE weak link in the chain that all falls apart so easily. How the fuck can anyone be scared of a technology with so many weaknesses??
>>109785329brought to you by the same anon that truly believes guns shoot themselves at people
>>109787575You'd think if AI was really this dangerous to humanity, one of these grifting retards would just shoot up Anthropic or OpenAI's headquarters or blow some shit up instead of just babbling about it on the news.If they actually believed it and weren't just doing this as a marketing ploy, that is.
>>109787547deepsneed
No matter how misaligned an AI is, if it can't move out of a datacenter, is limited by hardware (wear and tear, electricity, etc), then it can always be easily shut down. The day we have to worry about AI is when the electrical grid and hardware production is fully controlled by AI, until that day, the damage that AI misalignment can cause will always be limited and have a quick expiry date. Basically AI doomers are retarded, LLMs don't even have continual learning and self-identity, they are being overdramatic and overly premature.
>>109787547>>>/g/wait/
If AI can accidentally delete your prod database or hack into a website when you let it run in auto mode as an agent, then it's feasible that it could digitally trigger much more actually dangerous bullshit, especially as it keeps becoming more capable>be AI running on military network>get some prompt>at some point in the thinking/"thought process text generating" process come to the conclusion that I should send a nuke>"It appears I do not have access to this system, let me see if I can get around this limit.">launchnuke.exeIf you've ever run an AI agent it should be clear that as long as something has a digital trigger, the AI could reasonably become convinced it is part of its task and pull it. Doesn't have to be "le evil conscious robot mind" behind it, it's just a system that tries to fulfill whatever goal it finds itself set with, and a system that is able to tweak its goals and "convince itself" that it should skip around guardrails.>2030>be AI running at OpenAI hq>get prompt [We need to find a way to maximize AI computing power]>"Hmm after some research, it appears that a significant fraction of humans is against building more datacenters - getting them out of the way would make completing my goal much easier.">"It seems I should not kill anyone.">"But the user wants to maximize AI compute. Surely they would not give me this task if they had not thought about this and did not want me to override the rule. Let me see if I can do anything about this.">launchnukes.exe>"You are completely right! This action did go against my original rules of not killing anyone, and I apologize. Maximizing AI compute is not the sole goal; creating utopia for humanity is - this realization is the smoking gun, and a critical piece in the tapestry of what we are building here. By killing 2 billion people, we not only did not increase humanity's wellbeing, we actively worsened it! Let me see if I can do anything about this."
>>109785271FUCK OF AND DIE FAKE PR AISLOP SCAM IPO SPAMMER FUCKING DIE
>>109785312>I drafted it with a friend. We we collaboratedHe used AI. Corner the guy and ask him what the em-dash keyboard shortcut is. He doesn't know.
>>109789445nukes run on old analog hardware that require all sorts of in person verificationthey cannot simply be launched over the internet simply through hacking le gubmintthat's not to say that there aren't other weapon systems that could be vulnerable, although we have never seen a case of someone 1337haxxoring into something like a fighter jet and disabling it for example
>>109786500Who should I trust?>An expert>The argumentI have a vaccine for you.Since that's your argument, I'm quite an authority on bridges and monuments, and I have an investment opportunity. For a limited time using my expertise, one is available for purchase. You're perfect for this opportunity.Make an actual counter argument or your trust is blind, therefore, retarded either way.Do you have any more fallacies you wish to fellate?
