I changed my mind about LLMs. I think it can be a pathway to AGI. At the end of the day creativity and innovation is really about rearranging and connecting concepts and ideas.
>>108950427Linear regression is creativity?
AGI will never come true because it's a marketing concept and thus an eternally moving goalpost. If Claude Opus from nowadays (hell, even Gemma 4 31B) had suddenly been available in 2022 it would have been hailed superintelligence.
>>108950427I don't think LLMs will ever be good, but we will invent things that are better than LLMs, and these will be good.
>>108950427based, you are becoming smarter.don’t let the luddites tell you what to do bro
>>108950552They exist already, but not available to general public. Nothing new, just a lot of hardware and data. Imagine you have an actual index of the Internet, where nothing is censored and no regard for copyright etc. When you look for data, it does two dozens of different passes using several types of expensive rerankers, instead of one pass and the cheapest reranker for general public, not to mention complete lack of aids and shilling. Same with RAG (vector DB search), same with context size (probably around 10 million or more) and so on.Just current tech, but several magnitudes better. And just as expensive as you can imagine it is. Who can afford it? Whoever uses tax payers money. And Israel probably.
>>108950427yes, people who say LLMs can never lead to AGI are retarded. They're like cavemen who say Computers of the 1970s will never lead to LLMs. Just because one is not a direct descendent of the other does not mean it wasn't instrumental in its creation. Every bit of technology throughout human history uses previous inventions as stepping stones along the path.
>>108950451LLM's are non linear.And yes, a sufficiently sophisticated regression on, say, the writings of the most brilliant scientists will functionally emulate the process that wrote that text.
>>108950594you think llm trainers give a fuck about data?eleutherai made a multi TB torrent of all books they could get their hand on (libgen etc.) and it's the shittiest llm in existence.modern AI is about synthetic data. the bulk of all human made data has already been consumed.
>>108950427The real ball knowledge is that there are multiple pathways to AGI are we are going to take all of them eventually.
>>108950427>I changed my mind about LLMs. I think it can be a pathway to AGIYour first instinct was correct, LLMs are a deadend for AGI.
>>108950427LLMs is what Steve envisioned with Siri. Tim couldn't build on it
>>108950427the anticipation was that overlaying enough rulesets for AI in particular would be sufficient for comprehending and interpreting human language. the task has been too complex for any number of people to implement. AGI is a long ways off from LLMs, and tangentially I've wanted to implement an AI to selectively compile e.g. Linux kernel against given syscalls from a higher level language e.g. C for minimal distribution release engineering. I digress, the point is even something along those lines would require something like a neural network and not traditional AI because programming every single syscall on each platform is not only a moving target like human language is, that there's just too many to design or update, especially considering that there's ABI details involved when working between different architectures. ones I don't know just as nobody knows every single word.
Formal verification for code and math + neurosymbolic
>>108950427> a pathway to AGIDefine "AGI" first. A God Illusion?
