https://www.youtube.com/watch?v=21EYKqUsPfg [Embed]Indian podcaster Dwarkesh Patel debates the 'father of reinforcement learning' Prof. Richard Sutton on whether LLMs are a dead-end.Sutton believes so, while Dwarkesh Patel proclaims LLMs are the foundation on which experiential learning can happen, citing recent LLM models' success at high-school math problems. Patel claims humans all learn by imitation too.The following exchange summarizes the debate:>Patel: But there are phases of learning where there's the programming in your biology early on, you're not that useful. And then kind of why you exist is to understand the world and learn how to interact with it. It seems like a training phase [in LLMs].>Sutton: There's NOTHING where you have training of what you should do; there's nothing. You see things that happen, you're not told what to do. Don't be difficult, I mean, this is obvious.>Patel: I-I mean you're literally taught what to do, this is where the word training comes from, from humans.>Sutton: I don't think learning is really about training, I think learning is about learning, an active process. The child tries things and sees what happens. We don't think about training when we think of an infant growing upWhat's /sci/'s verdict? Who won?Is Dwarkesh Patel right in saying humans are literally taught what to do the moment they're born? Is all learning just imitation?
>>16834075>IndianStopped reading.
>>16834075>>Patel: I-I mean you're literally taught what to do, this is where the word training comes from, from humans.this guy is unbelievably retarded. The core component of intelligence is not imitation. It's exactly as Sutton puts it: you're not told what to do, intelligence is about observing the environment and exhibiting novel, goal-oriented behavior based on it.
>>16834075It's both, isn't it? A child with someone helping them is usually going to advance a lot faster than if nobody helped them. And some things, like a child learning English to be able to interact with other humans, can't really be done by just trying things out on their own, especially if they're alone
>>16834101yeah however the question is what makes an animal intelligent and most would agree that mindlessly imitating things isn't it but for that to not be mindless in the first place you need some kind of intelligence. Not to mention, human brains are about a million times quicker and a billion times less energy-consuming than LLMs, and LLMs can't learn on-the-job, so for Patel to say LLMs are the foundation on which experiential learning can happen is just wrong. Clearly, the architecture of animal brains is much more intricate and effective and should be studied to make any kind of progress towards AGI
I think I take a young optimist Indian over a super grumpy super old CSler
>>16834075>We don't think about training when we think of an infant growing upHas this man ever raised a child? Obviously not. He either left it to a woman or he's a childless gay.
>>16834075>[Embed] [Embed]retard>>16834138I watched this video a couple of weeks ago, and IIRC, his point isn't that you "train" the kid, it's that the kid is able to learn by himself and, most importantly, to act on what he learned. you don't train the kid, he can learn himself, (sorta kinda) reflect on it and use the learning and knowledge as new tools.
>>16834138>stay-at-home chud to train the childrenNow this is a novel approach
I found this guy through his interview with Richard Rhodes a couple years ago and he seemed to play the "good interviewer" pretty well, which consists in my opinion of:>get a good guest>ask reasonably well-researched questions occasionally to keep them talking>otherwise shut the fuck upWhen he (or any interviewer) tries to make it a real-time debate I usually find it ends up just being a mess. That said I haven't seen this specific interview, maybe it's good.
>>16834075i dont think this poo understands that LLMs are just recycled logic machines it cant invent anything the government can make it happen through some ways but they wont release it to the public ideation LLMs are not going to come out lmao
>these models do seem to have very robust world models>SEEMjeet could have stopped there.LLMs have dazzled retards so much they are like a reverse Turing test
>>16834075I'm just glad he was finally explicit in saying all the llm stuff has nothing to do with what he meant in the Bitter Lesson - none of it is le bitter lesson-pilled. 2+ years of hearing that. It's a short essay and not hard to understand at all. Techdrones don't read so good.
>>16834075Stupid debate. Old guy says new thing not as good as old thing. Young guy says new thing good, because new. Why not just get old ham radio enthusiast to debate a Gen Z on the benefits of smart phones. That might actually be better.
>>16834075notice how the white person is the inventor and knows how ML works at the core RL specifically and the poo loo gets excited over mundane bullshit thinking its some revolutionary piece of tech lmaoits a google search api on steroids thats it too bad my 2019 search engine was controlled
>>16834075>>Sutton: I don't think learning is really about training, I think learning is about learning, an active process. The child tries things and sees what happens. We don't think about training when we think of an infant growing up>Sutton: I don't think learning is really about training, I think learning is about learning, an active process. The child tries things and sees what happens. We don't think about training when we think of an infant growing uptheres an element of innate learning to humans that a markov model or token bin distributed model cant replicate i guess we need RL generative activation functions or some bullshit agi pure math invention for activation functions otherwise we're stuck at this lmao
>>16834814>Old guy says new thing not as good as old thing.Old guy is an expert in the field. Young guy is a tard. Hope this helps.
>>16834138>>16834814>>16834132t. p zombie
>>16834083>The core component of intelligence is not imitation. It's exactly as Sutton puts it: you're not told what to do, intelligence is about observing the environment and exhibiting novel, goal-oriented behavior based on it.You're correct, obviously. But Indians don't see that, the average Indian really does just imitate what everyone around them does. That 'jeet thinks this is intelligence because that's exactly how the 'jeet mond works.
>>16834075Dwarkesh Patel is subhuman, he cannot comprehend the idea of figuring things out for himself
>>16834814you're never getting anywhere with that ignorant attitude.
It's amazing someone as stupid as Dwarkesh is surrounded by intellectuals such as Paine and Sutton.
>>16834075>Is all learning just imitation?A lot of it instinctual and hard-coded into our DNA most likely. You never learn breathe by imitation, for example.
>>16834075>>16835294https://www.youtube.com/watch?v=u3HBJVjpXuw
>>16835591not interested in hearing any more of this retard's musings. There also was no "misunderstanding" here, Dwarkesh's position just isn't based in reality and fell apart on the slightest pushback by Sutton
>named dwarf pajeet>iq as high as a midget Who wrote this shit?
learning isn't just imitationlearning is when you change your reaction to future stimuli based on past stimuli that's itfor you to imitate irl you have to have some kind of planning and abstract thinking and whatever its obvious animals don't do that and neither do babiessome stuff is built in through genetics and everything else is either learned reaction to stimuli or some shit to do with planning and abstract thinking and whatever which requires a model of the world learned through aforementioned stimuli anyways
>>16834075i swear these tech people like patel have got to be llms themselves
>>16834105Mindless imitation is how we create a mind, then superstructure in the mind is formed after lot of mindless imitation. A baby's mind isn't doesnt have full consciousness, they're similar to animal as they have not evolved an inner mind.
>>16834075For once I'm with the jeet on this one. Learning is training. We have a reward system in our brain that is analogous to ML scoring functions.
>>16834786The government is retarded and can only do a fraction of what the private sector can. It can absolutely invent things just like you can. Inventing things is just combining things that already exist. It has the same primitives that you have, and can combine them in new ways if given appropriate stimuli, just like you or any other human.
>>16834956That’s how hindpoofags are they lack actual intelligence like us aryans
