https://www.youtube.com/watch?v=21EYKqUsPfgIndian podcaster Dwarkesh Patel debates the 'father of reinforcement learning' Prof. Richard Sutton on whether LLMs are a dead-end.Sutton believes so, while Dwarkesh Patel proclaims LLMs are the foundation on which experiential learning can happen, citing recent LLM models' success at high-school math problems. Patel claims humans all learn by imitation too.The following exchange summarizes the debate:Patel: But there are phases of learning where there's the programming in your biology early on, you're not that useful. And then kind of why you exist is to understand the world and learn how to interact with it. It seems like a training phase [in LLMs].Sutton: There's NOTHING where you have training of what you should do; there's nothing. You see things that happen, you're not told what to do. Don't be difficult, I mean, this is obvious.Patel: I-I mean you're literally taught what to do, this is where the word training comes from, from humans.Sutton: I don't think learning is really about training, I think learning is about learning, an active process. The child tries things and sees what happens. We don't think about training when we think of an infant growing up
It's like witnessing a p-zombie talk with a human
>>107080123if you need 1 gorillion examples to learn a thing as a human like an LLM would need to "learn" then you must be pretty slow.
Who the fuck is this Patel guy? How is he getting Sutton on his podcast when he's a complete idiot?
The thing is, learning aside the human brain(singular) is, while not optimized for many 'computing' tasks(outside autistic savants maybe) worth every single datacenter GPU in the world and more when it comes to self-correction through inferences and coming up with not just solutions, but new problems to be solved which drive progress.Chatbots are really just good at storing vast amounts of data and presenting it in a filtered form.
>>107080260All you have to do nowadays is grift about AI on Xitter and create an AI startup
>>107080260>Who the fuck is this Patel guy?An idiot who's useful to billionaires
>>107080260>Dwarkesh Patel>Patel only came to America when he was around nine years old, spending his early life in India. His father, a doctor, sought an H1-B visa and was often assigned positions in rural America, where there was demand for quality doctors. Through Patel’s teens, the family shuffled around states like North Dakota, West Virginia, Maryland, and later, Texas, where Patel eventually attended college.
>>107080123>Patel: I-I mean you're literally taught what to do, this is where the word training comes from, from humans.how are these "people" even real? Who teaches humans how to reproduce? Was he taught how to do that as a child? The fuck is that kind of sentence?
>>107080213This. And you always need humans to give the LLM shit to learn in the first place. It's got nothing to do with intelligence so far and retards insisting on LLMs are unironically preventing progress
>>107080123LLMs can mimic intelligence, but they can't mimic super-intelligence because you can't mimic something that doesn't exist unless it's an abstract concept I guess, but that's the only exception. so I think they will become as intelligent as the smartest human, but it's a brick ceiling after that.
>>107080440>Who teaches humans how to reproduce?early humans learned it from watching animals, same way LLMs learn from us.
>>107080513>I think they will become as intelligent as the smartest human, but it's a brick ceiling after that.desu I wouldn't even say that. Generally, it will appear as intelligent as the smartest human and in some contexts this may actually be indistinguishable from real intelligence. But as Sutton puts it, LLMs aren't capable of learning on-the-fly and humans/many animals are genuinely able to do much more than just imitation. When you observe something, you aren't being taught a thing directly, you're not being told anything. It's the intelligence here that turns this observation into something useful/worthy learning.
I truly believe that to make any real artificial intelligence you need to first unlock the human brain.If your design doesn't have inspiration from the human brain you're doomed for failure.You can squeeze a surprisingly good amount of value from neural nets but ultimately they are probabilistic and the weights are static.As it stands they will never replicate intelligence
>>107080738not to mention neurons themselves are unironically much more complex than neural nets. We've basically just pushed really simple architecture to its limits
https://www.youtube.com/watch?v=u3HBJVjpXuw
>>107080812Yup. We have bruteforced our way into this and got pretty far. This AI provides value and will stick around but it's not real intelligence.The human brain uses 20-100W Better architectures exist we simply don't understand them.
>>107080123It's incredible how gormless Dwarkesh is in this. Large language models are still unusable for fields that aren't massively overrepresented on Reddit.On multiple occasions, I've tried queried the different chatbots on some niche engineering field and no matter how I prompt them their answers default to the much more conventional and better known alternative to whatever I'm asking about
>>107080440>Who teaches humans how to reproduce? Was he taught how to do that as a child?I mean sex education has been a part of most school curriculums around the world for quite some years now>then you hear stories of Americans who fuck ears/give a blowjob and are afraid they might get pregnant
>>107081506brother, humanity has existed for far longer than sex ed has. Also, animals reproduce just fine without any of that. Point is there's something beyond just imitation
>>107080123What is it with Indians and AI? It’s like Indians are all excited that they’re finally going to have AI to bridge the gap between Indian and human intelligence
>>107081208he's playing devil's advocate. he thinks agi is many years away
>>107081933They recognise a good scam
>>107080123Who the fuck would choose to listen to a jeet talk?
>>107080738You don't know anything about AI, logic design, or neuroscience do you?
>>107082512I know I'm right and I know all AI bros immediately kvetch when facing this argument
>>107082512seethe
>>107082347kek
>>107080186>humancalling the old guy human is pushing it pal
>>107080123>a toddler calms down when breastfed because other humans trained him to, he copies people around himPajeet my lad i dont think you're that smart
>>107080397I can smell that backstory through my screen... Fuck you, anon.
