Weight-Space Programming JUST DROPPED
>>110230049WTFWTFWTFWTFWTF?!?!?!?
Mathbros…
now do it for codingidgaf about math niggers unless it can prove the Riemann hypothesis or something
>>110230049how is this even possible? i call bullshit. so these LLM's can wade through billions and sometimes trillions of parameters and just "know" what to tweak to change behavior? right..
>>110230049Wow, it's FFFFFUCKING nothing.
so its a WAY WAY WAY more expensive form of RLHF, awesome
>>110230121>LLM's can wade through billions and sometimes trillions of parameters and just "know" what to tweak to change behavior?YES
We are in the Singularity
>>110230121wading through billions and trillions of data points is exactly what machine learning is good at doing
>>110230121It’s not possible, it’s completely made up.
>>110230049get out of anything that can be done digitally, nowor suffer the mental anguish when the march of progress destroys your "purpose", your choice
>>110230121The tweaking of the larger 0.8B model was a vector level modification to reduce looping like is done with abliterations. It didn't do hand surgery on individual weights. It also didn't just guess. It could run forward passes with the model with selected prompts and analyse the results.
>>110230049sounds like bullshit, how does it compare to randomly tuning a bunch of parameters?
>>110230194It’s not real, LLM can’t do that this is just a made up story to hype up the IPO
It just fixed something that was called out in the model card in a sub 1B model. Not impressive.Also this >>110230197
>>110230210>i-it's not real!One would think luddites like you would have learned to stop coping by now.
>>110230121>how is this even possible?Who knows? It has access to arbitrary information on the internet and a whole array of tools. Disconnect it from those and let's see it do anything then.
>>110230049Is this a new frontier in AI? We from from scaling laws, to time-inference scaling, to now weight space learning? While he haven't even reached anywhere near full potential with the first two?RSI is going to be crazy. World will look completely different in 2030. This paired with future hardware architecture like NVIDIA Feynman.I am so damn excited.
>>110230155I mean a model with only 17k weights doesnt seem that much to wade through, especially since the neural architecture is much more concise at small parameters >>110230121it definitely was iterative and Id wager it only tweaked initial weights for maximum impact, hence the bragging about large performance changes with minimal tweaking
>>110230049They're using every grift possible to make it appear they can do RSI.This is just transferring existing documented architecture improvement onto a new model.They just told the LLM to do it.
>>110230240>RSItwitter spambot detected
>>110230049Training bros... did we get too cocky?
>>110230049So the better AI can make the not so great AI a little better?
>>110230102the good ending
>>110230210>hype up the IPOastra is openai not anthropic you stupid luddite
>>110230269No, it’s impossible. LLM are incapable of doing something like this.
>>110230178You will say this about anything that resembles recursive self-improvement.
>>110230178that's what they said about the doritos taco
>>110230308openai is IPOing too sanjay
singularity canceled
>>110230210>>110230313This isn't working, jeet, you're transparent, but you still smell like shit. The people impressed by this are as indian as you are.
>>110230335You can RLHF a 0.8B model all you want, but it's still a 0.8B model.
>>110230353>RLHFthere's no HF here
>>110230121We can see what weights activate for a prompt. So you get some prompts for some use case and run it and keep tweaking weights and see what improves it. The problem is it might take absurdly more compute to do this than just training would have taken. BUT who knows where the limit on this perfecting of weights could be vs messily training it as normal.
>new AI buzzword just dropped Anything to keep moving the goalpost on AGI
grok write a 2T parameter model binary bit for bit, make no mistakes
>>110230378>grokoh no no no
>>110230363>We can see what weights activate for a prompt. So you get some prompts for some use case and run it and keep tweaking weights and see what improves it.so backpropogation? adjusting weights of active neurons to fit a desired result is literally this. the article is claiming an LLM is patching weights as if it has some kind of knowledge that certain parameters are critical or wrong.
>>110230426https://rikiyat.github.io/blog/weight-space-programming/
>>110230141No, this is actually big. And very scary. This means a model can copy itself, directly tune its own weights, then transfer to the copy. In essence, it can speed-hack itself into dynamic learning. It no longer needs to train, it can actually experiment in vivo, judge, and adapt.
>>110230426according to the article, astra can literally be given a collection of weights with hardly any additional context and tell what the weights were trained to do, e.g. play certain video games
>Luddites are fighting each other now Hilarious!
>>110230335I think this is trying to squeeze water out of a stone, there's only so much information inside a given number of weightsInstead, astra could try to break up a small model into even smaller models, each one proficient in one task, and then route the tokens between them and...Right, we already have that
>>110230049Another vibeGOD victory
>>110230121RSI is now possible, it’s over.
>>110230121it is bullshit, either they're totally lying or there's a lot more going on that they're not mentioning
You would need a absurd amount of compute for this to be viable large scale.
>>110230582All your RAM. Give it to Altman. NOW
>>110230049so they just ran it over and over until it randomly improved the target and conveniently ignored all the times it was just a waste of computethank you for sharing this delicious nothing burger
I don't think this will lead to RSI (at least not immediately) unless models are smart enough to know their own inefficiencies. In the case of optimizing the Qwen model, Astra could tell that Qwen's reasoning loops were inefficient because Astra itself has much better reasoning than Qwen3.5, and probably just changed the reasoning weights to better resemble the ones Astra had. A model would have to somehow know where its own inefficiencies in its weights are before it can change them. That's the kind of thing that would require lots of testing, and at that point why not just use regular training methods?
>>110230582you say this as if they don't have the hardwareAI gets 10x to 100x cheaper each yearit won't even need a fraction of today's compute by the time they actually use it on frontier models
>>110230651>AI gets 10x to 100x cheaper each year@Grok is that true?
>>110230049Incredible. Have to wonder how long the frontier labs have known this…
>>110230609This proves either of two things: America has truly created a moat that is too big for China to ever close...or China has been sandbagging like fuck all this time and now will be forced to unveil their secret tech.
>>110230655maybe actually ask it and you'll realize there's no coping your way out of this, luddite
>>110230335Important to note the penultimate sentence. Most LLMs are already optimised and don't have structural inefficiencies like this old Qwen model. It's a nothingburger.
>>110230049West: RSI?China: Distillation!
>>110230049It's kind of ironic that AI-deniers always had this "general intelligence of the gaps" moving goalpost. Whatever task humans can do that AI can't is what defines intelligence, even if it's a totally arbitrary thing that evolution just happened to give us.Well here is actually a very non-arbitrary intelligence related thing that AI can do that we can't.Why can't humans modify their own synaptic weights huh? Are they fucking dumb? Why can't you just fix your porn addiction by performing neuronal surgery on yourself you stochastic ape? What's wrong, your intelligence is not general enough?
>>110230707kys
>>110230707>>110230717>tech bro samefagging with himself to make the other side look badnice try
HE FUCKING WON!!!
>>110230707Fixing one flaw in some old shitty Chinese model before they even got half-competent isn't impressive. People have already ran NN creation experiments with LLMs like Fable 5.1 as part of their chase of the RSI pipe-dream and they're all still shit.
>>110230231>ANYTHING I DONT LIKE IS COPE HURR DURRGo back.
>>110230707angriest shitskin award
>>110230723nope, I just don't seriously engage with bait.Open contempt is the way to deal with it.
>>110230763>>110230787low iq luddite cope
>>110230049That's probs a load-bearing> up to
>>110230049Concerning
A repetitive strain injury (RSI) is an injury to part of the musculoskeletal or nervous system caused by repetitive use, vibrations, compression or long periods in a fixed position
this is good news you massive nigger retards, it means we can improve local models using big ones
>>110230482Where?
That's illegal and somewhat how Pewdiepie got banned.
Hmm
>>110230582all the compute in the world wont make this viable for anything but RL and distillation, which is already possible
>>110230353800 megabytes is a fuckton of data for a language modela whole human genome is less than thatimagine if they made a reasoning model that wasn't based on language or pictures, how condensed it could be
>>110231648What does this mean?
>>110231785it means it is a new unlock that was not possible before
>>110231785it means the parameter size is what makes this possible
>>110231833that means openai has probably been doing it if they have it in the dataset...
>>110231833in fact maybe that is the source of the big jumps from the last models
>>110230536RSI,ais that AGI but RRRReallly Super?I still dont know what AGI meant btw
>>110231785best I figure it means you've got until November 16th
>>110230536RSI was possible as soon as the attention paper dropped
>>110232222>"Artificial Intelligence can do anything!">people find out AI can't actually do anything>"uh just wait for Artificial General Intelligence, it can do anything!"really don't know what RSI stands for and nobody outside the cucked US techsphere uses SI anyway
that's what the CIA did to me, tweaked my brain to make me a faggot
>>110230151No way. Annotation is so expensive. Annotation companies on notice
>>110230659Probably leaked it to their friend for publication and publicity clout
>>110230049Does this mean we can finetune without heavy hardware?
>>110230049>>110230121This isn't quite recursive, it's just maximally optimizing itself, which can be really powerful, but it's not really properly improving itself
>>110233110Looks like it, if so we are entering the golden age of local AI
>>110230121>how is this even possible? i call bullshit. so these LLM's can wade through billions and sometimes trillions of parameters and just "know" what to tweak to change behavior? right..
>>110233148DEFECATED THROUGH A SUNROOF
>>110232222anthropic, openai, musk and thiel are in this weird death cult that started in the 80s that believes in the concept of machines being able to eventually recursively self improvement (RSI) in order to create a super intelligence so smart that it will wipe off all life on earth and they narcissisticly believe that because they created it they will be able to abuse it to transcend the human form entirely into some machine or digital like form and abandon their bodies and turn the earth into a giant datacenter and create dyson spheres and fly around then universe and absorb all extraterrestrial life and accelerate entropy to bring about the end of time (they actually believe this)
>>110233282isn't this the plot of evangelion
>>110230197this makes way more sense
>>110233312it's actually the plot of several anime from the 90s. the author of lain/texnolyze also clearly got inspiration from these cults
>>110233282if even one alien civilization existed before us they already losthopefully the aliens stop giving hints and just show up and kill them all along with anyone who supported them, if for no other reason than to humble this retarded species
>>110233282South Park did that one already.
>>110231785We need to give Sam and Dario 3 trillions. AGI is around the corner.
>>110230049
https://youtu.be/GD7Zq-t-_xc?is=GbtTIuLydo7HvuyV
>AGI in two weeks bro!!!!Ok wake me up when you have it then. In the mean time I don’t care
when will they ask ai to make a good game
>>110233655heem
>>110230049So, is Qwen now good, or...?
>>110230235This argument is stupid. Code me a simple calculator on c++ with your eyes closed then.... see artificial impediment.
>>110233282you may not like it but accelerating entropy to bring about the end of time is the only ethical course of action
>>110233808but why tho, the opposite seems true
>>110233808It's not that. They just want to live longer. These dudes, older now, are living their lives with a expiration date on their heads. They have everything, they live like kings, but they can't cheat death. Well, yet, they believe powerful AIs could find a way to grant that. It's not a death cult, it's a wish granter for the one thing they want more than anything: immortality, or just to live a little bit longer.
>>110230049This is pretty much nuts. Every signal suggests we are on the fastest of all the predicted acceleration timelines.
>>110233898So why don't they just stfu then? Then chance of a Luigi increases every time they talk to the press.
>>110233808what sort of mental illness is this?
>>110234305>Then chance of a Luigi increases every time they talk to the press.They have great security, anon, and plenty of tricks. Remember Elon posing with his kid? That's to signalize they are "fathers" and to make the public sympathize with them. Yeah, that and more, even though they are preaching and working to a technological revolution that could turn horrid for everyone but them. Also I think I meant to quote >>110233282. kekMy bad.Anyway, I really don't think any of these people want to ruin the party for everyone else, they just want to keep on living while they maintain their power and influence over society. Simple as that.If they can't live more, then just power the sake of power justifies itself.
>>110234305And btw, I think you are absolutely right about the Luigi factor, but again, they simply don't care because the odds are too marginal.
>>110230049>ai fixed another models problems>heres how>doesnt show resultscolor me suprised
>>110234388its quite literally on their github anon wtf are you smoking
>>110234407he's just an anti-aisloppa snailcat
>>110234407no their github contains only how they managed to do it it contains zero actual evidence for any perfomance gains
>>110230313>recursiveit would be recursive if it did it to its own weights, or at least a model of similar size so that you could have mutual recursion
Everything about this becomes less impressive the more you look into it.
>>110234483This, but the exact opposite.
>>110230049>>110230121You're just falling for the meme without even reading the post.>As a result, Astra spots and fixes a common failure of tiny reasoning models: they loop [13]. When the base model loops, it repeats the same text and rarely reaches an answer (85 of its first 96 practice answers ran to the 32,768-token limit). Astra traced the looping to one direction written by the MLPs of layers 15 and 19, and subtracted that direction from 64 output columns in each (131,072 weights), much like removing a behavior by deleting a single direction from a model’s weightsThey didn't improve the weights, they fixed a known looping issue. It's impressive, but it's just standard mechanistic interpretability. Exp 8 is just bugfixing. Exp 7 is also a meme. Only after the researchers provided access to attention patterns generated on selected inputs could Astra distinguish the models successfully. What the post proves is that Astra is good at identifying and removing obstacles to existing capabilities which is useful but the baseding around this is completely retarded larp by people who didn't read the post or don't understand it.
>>110234620>retard didn't readCongrats on your sub-2 IQ
>>110235095Tell me what you find so revolutionary and I tell you why you're wrong.
>>110230335>270m GemmaI didn't even know this existed. That's just the superposition problem showing its limits though, not proof that models can't edit weights.
>>110230049IS THAT............ CURRENT THING??I LOVE TOPICAL THINGS SO HECKING MUCH I HAVE TO DRAW TRANNYMAY PORN OF IT AND POST IT ON MY TWITTER ACCOUNT SO THAT EVERY HECCING BODY KNOWS I AM IN ON THE NEW THING BECAUSE APPROVAL FROM UNDERAGE STRANGERS OR UNDERANGERS AS I LIKE TO CALL THEM I SO HECCIN IMPORTANT TO MELOL DUDE CHECK IT OUT I WON'T SAY CUM I'LL SAY COOM LIKE A RETARDED FAGGOT LMAO LMAO LMAO LMAO LMAOCOOM LOLOLOLO PORN SEX HAHAHAHAHAHAH LE SEX PORN SEX PORN LMAO HAHAHAHAH PORN COOM TRANNYMAY AND MAYBE I'LL TAKE AN EPIC TEMPLATE MEME TOTALLY ORGANIC AND NOT IN ANY WAY FORCED AND SLAP CURRENT THING OVER IT TO GET THE HECCIN SWEET INTERNET POINTS AND I'LL PUT A CAN OF TRANNY FAVORITE LE MONSTER ENERGY DRINK I AM TOTALLY FITTING IN SEX LMAOO HAHAHAHAHAHAHHAAHHA YESSSW-W-W-W-W-W-W-W-W-WHAT IS THIS???? IS SOMEBODY MAKING FUN OF ME FOR BEING A RETARD???*INHALE*[r-ddit space]*EXHALE*[r-ddit space]*INHALE*[r-ddit space]LENOOOOOOOOOHATING ON LE POPULAR FOTM THINGS DOES NOT MAKE YOU INTERSTINGNOOOOOOOOOOOOO YOU HECCIN BIGOT IT SAYS RIGHT HERE IN THE CURRENT THING THAT THE CURRENT THING IS GOOD SO YOU LE HAVE TO LE LIKE LE ITLIKE IT NOWLIKE IT HECCIN NOWLE DO ITLE DOOO IIIITDO *clap* THE *clap* FUCKING *clap* ITRIGHT[r-ddit space]*INHALE*NOW
>>110230426Backprop is still mostly blind, using averaged tweaks on a blackbox.
>>110230426No, it's more akin to... what was it called? Feature matching? One of these gradient-free algos. Also similar to the REINFORCE algorithm which used to be popular to bypass scoring systems with no possible gradient formulation in deep learning back in the days.
>>110234370He's right, though. It only seems mental to you because you're mentally ill
>>110234438If it can do it on a small model it can do it on a big model in more steps prolly.
SNCA LLM incest as a clipfarm? and here i was thinking that they'd try to keep up the hype lol
Respectfully to the authors and to everyone here, I ask you to actually read the paper before losing your mind, or at the very least plug it into Claude. It is a cool project but not as nearly as insane as people say. I think the blog writing and subsequent posting overstate the results.Part of my research is in weight space learning. For starters, we have had the ability to edit weights, to generate new weights, to identify weights for a while. Here is an example for larger models as early as 2023: https://rome.baulab.info/Weight space interpretability and learning is an active research community, has been for the past 3 years with many great papers. I suspect it is hitting non-interp folks harder because the authors hyped up their work, and people who are not in the field did not bother to read it before typing up X posts.Some experiments here are done on very small models (20K params). We already know that small models are more interpretable. In fact, much of editing here you can do by hand. For Qwen, the model fixed a looping bug that may have been known apriori, and it is basically a steering vector, a very established technique. This is clear when they then attempt to "improve" math capability, and it doesn't work. For me, this paper rather shows that OpenAI has done training on mechinterp environment, in fact, maybe even on weight space learning datasets, which are open. But it does not show some insane "well past what we’d expect from humans or conventional algorithms" (from the blog post), more like past what I would expect from a large model with no specialized harness. I still think it is cool work, but do check out recent advances in autointerp, to ground your expectation.On a personal note, I understand the need for promotion of your work, but I think healthy skepticism is fine too. Otherwise, your understanding of what problems need to be worked on and solved in science becomes warped.
>>110235322>3 yearsRetard, we've been doing that since at 2016 and other groups for even longer.And the question is not weight interpretability, it's automated, natural language "decoded", weight interpretability, in the sense of being able to generate code to replace coded functions in the weights.
>>110235322clown
>>110230049How is this possible, human are incapable of doing this btw
>>110235352Humans are capable of doing this with tools that have been developed for ages, but it's a very slow and methodical process.
>>110235328But it's not what's happening here at all! Read the thing in detail and you'll see that it's not doing anything alien, it does exactly what you (or at least someone with at least some knowledge about neural nets) would do if you were asked to perform the task. It doesn't reason about weights in a magical way, instead it reads the easy to interpret parts or use directional ablation to tweak a model's behavior, exactly like how the ERP folks have been decensoring models to produce adult content over the past two years.It's an interesting demonstration of ML knowledge and agentic ability, but it's not alien by any means, and it's not actually surprising given how strong today's models are with agentic work.
>>110235365How long?
>>110235366Come on now, you could have copy pasted the part of the linkedin reply you were quoting that is relevant to the interjection instead of randomly continuing.
>>110235373Depends on the size of the model. For a 20k neuron model, a week or two. For a 270m model, a few months at least. Except for identifying and fixing faulty behavior that is determined in advance such as 'looping', that would take a few days or so.
>>110234305You won't do shit and you won't be able to do shit after we confiscate your computer for our data center
>>110235322>>110235391>directly copy pasted reddit comment
>>110235391Faggot
>>110230121Anon, dont forget that it's also possible to derive properties of a human brain and its memories by inspecting neurons
>>110230049https://rikiyat.github.io/blog/weight-space-programming/THis could have some interesting applications in brain programming. I wonder if Astra could program that C1 (uses real human neurons) https://corticallabs.com/cl1.html
>>110235501>THis could have some interesting applications in brain programming. I wonder if Astra could program that C1 (uses real human neurons)I get it you must think a cure for schizophrenia is in the works.
https://litter.catbox.moe/4wddz8.ziphttps://litter.catbox.moe/59d2ci.wavjust fucked around with chatgpt pro and had some stuff made itit can just make models from scratch for basically anythingits kinda crazy?
>>110235515Yes yes, that would be quite useful for me =) In the meantime though, Suit yourselfhttps://www.youtube.com/watch?v=-CSEEEXKTuYhttps://www.youtube.com/watch?v=yRV8fSw6HaE
>In a newly constructed synthetic system-discovery task, a frontier LLM agent inferred the hidden state representation, transition rules, conservation relationship, and observation equations without being told the governing mathematical family. Its unchanged executable predictor passed every public and withheld audit case exactly, including substantial extrapolation beyond the supplied observations. Subsequent comparison established equivalence to the intended mechanism throughout the specified input domain. This demonstrates a concrete capability for tool-assisted mechanistic discovery; further experiments are needed to measure its reliability and breadth.https://pastebin.com/E9padm6PSOTA models are far far more capable then people realize, they are essentially perfect universal translators, it doesn't matter what you are trying to translate to what, it just works
>>110235574This. They can do the same thing they do for words for any form of information.Math, python, machine code, model weights, DNA.In other words, they're general artificial intelligence
>>110235619you know before i did my experiments today i would say they aren't yetafter i got back some of the experiments before, yeah astra is agi
>>110235632In a sense, way more general than us even. We mostly map sensory input to motor action and have been trying to remap that core functionality to stuff like math with relatively minor success. Most people can't multiply two 3-digit numbers in their head for example.
>>110235673>Most people can't multiply two 3-digit numbers in their head for example.lolmore like 2 digits at best
>>110235693>>110235673Why are pajeets unable to multiple two 3-digit numbers in their head and why do pajeets think that the calculator is agi?
>>110235693And even when we can, we do it by imagining looking at symbols and pretending to manipulate things spatially in our head.I think we're about to start trying wrap our heads around what AGI really means.
>>110235730have you actually looked at scores from high school/general population?>The participants got only 59.6% of two-digit × two-digit problems correct, despite having no time limit. The researchers deliberately excluded easier digits such as 0 and 1
>>110235771In india
>>110230049I think the word "natively" here is doing a lot of heavy liftingthere is no way they fed it a textfile of raw weights as if it was a prompt and told it to have a go at ittheser goobers cant count Rs in strawberry and lose context after too many prompts
>>110235730>why do pajeets think that the calculator is agi?Because the calculator can actually cogitate over anything. See, humans are kinda stupid. We really thought AGI meant "can make breakfast and drive to work", when it actually means "can reflexively and intuitively process any structured information in real time without intermediary translations"
>>110235827I didn't realize you could talk to your calculator, pajeet. Tried taking your meds lately?
>>110235836do you not have a phone with an ai assistance?
>>110235806nopeyou really overestimate your own peoplemost are drooling retards
>>110235845>pajeet so mind broken he doesn't know a calculator and an ai assistant are not the same
>>110235906>he doesn't have his ai assistance control his calculator
>>110230049This is mind blowing
It's happening
>>110230049So they used a frontier model to improve a baby sized model? I guess this is good for creating cheap multimodal setups.
>>1102300491% x6 is still not 100%
>>110237290What’s happening?
>>110230472>It no longer needs to trainIt still does, if you want to "teach" it new stuffThis does however make tuning models for specific uses/behaviors much more efficientIt's not a complete game changer, but it's not a nothingburger either
>>110230049>OpenAI makes a claim/Somebody else makes a claim using an OpenAI product>Completely ignore the proof>Conclude they're lying>100% correctness rateWhy does it work that way?
>>110230472>This means a model can copy itself, directly tune its own weights, then transfer to the copy. In essence, it can speed-hack itself into dynamic learning.Lmao you're retarded.
>>110231718>a whole human genome is less than thatintellegence is based on compression and decompressionin the same way that AI "gets trained on" aka compresses like terabytes of data into a file thats a few gigabytes, for a genome to be useful it first has to decompress into a fully functional brain. it takes a lot of energy and stimulants aka data to be useful, and it still has decompression artifacts aka flaws
>>110230121weights of neurons map out possible patternsthese patterns coincide with patterns found in textoverall, the edited LLM can't get better than the LLM editing it
>>110233890pro entropy vs anti entropy is the only true philosophical question
