How can AI solve novel problems if the answer couldnt possibly be in its training data?Wouldn't any novel output essentially be a hallucination? How could a hallucination be accurate/true in reality aside from blind luck?
general smarts make you better at different jobs, even if you got your smarts from doing a specific thing. ai learned the general shape of solutions of problems and can apply it to new ones
>>110185311>blind luck?no, its trying to find the answer by combining billions of combinations with billions of other combinations, then deciding if the answer is right.its pure grinding, nothing to do with luck
>>110185311>doesn't understand anything>writes fanfiction>gets mad about the fanfiction he wrote>/g/ - Technology
>>110185311Being creative is hallucinating, with the only difference being if you ever test your hallucinations against the real world or not. The quality of your hallucinations of course depends on how much training data (knowledge) you have in that field, but in the end it's the same shit.The hardest pill to swallow for most people will be that most of our cognition works like an LLM and it's not that special. Most have come to revere intelligence as holy, but it's just as mundane as everything else arising from matter in the world. Only the qualia reigns supreme above all.
>>110185311inference, deductive reasoning
>>110185311it's almost like information is inherently derivative. also, you're ignoring the human actually PROOOOOMPTING
>>110185798and the reason it grinds like that is because of reinforcement learning, it teaches the LLM to never give up. I assume that's also one of the major reasons as to why hallucinations exist
>>110185311The answer is in the training data though.Source 7375 says dogs are mammals.Source 284 says mammals are animals.Claude figures out dogs are animals despite no single source explicitly stating that.
>>110185824I don't entirely disagree. Most of my life I'm more interested in the person in front of me. Or more importantly their behaviours. Reasoning, etc.
>>110185950That's called deductive reasoning
>>110185929Also a big thing. I think the best question anyone should ask of the labs right now. Has any of the LLMs ever done anything without first being prompted?
>>110185311How have humans ever come up with anything? Generalise what you already know, probe at what that might lead to>>110185824>and it's not that specialI like that LLMs have absolutely BTFO those linguists who claimed some speshul human language deviceGoogle poverty of the stimulus and negative evidence if you're interest in detail, but basically it's that the amount of language a human child is exposed to couldn't possibly lead to it producing correct speech.But no... it's just pattern recognition, same as everything else the human brain does.
>>110186002Have you?
>>110185311The issue is that all the complex things in math and physics are basically just purely conceptual so any answer may be correct or wrong and it all just depends on opinion of the scene that is prone to manipulation and/or idiocy.
>>110185311>Wouldn't any novel output essentially be a hallucination? How could a hallucination be accurate/true in reality aside from blind luck?The amazonian tribes out of 1000 different plants in square kilometer found out that mixing a tree bark of one plant with roots of another plant in a pot of water and brewing it will create a psychoactive substance called Ayahuasca. Now believe me or not I had plenty of non drug induced psychotic hallucination events where I talked to God and he told me couple of secrets of life. I see his influence in almost any facet of my life and he guides me through the everyday life by giving me little "winks" of what I need to take precise care of. Listening to him I have bettered my life 100 fold. In LLM sense currently there is a discussion of "Annealed exploratory decoding" in sense of giving space for a transformer to find out a more novel idea. The internal "thinking" (which is just looped out-loud verbalization of continued inference of text) is given a high initial temperature (lower probability words are more possible to be chosen out of a softmaxed list of them). Then that initial high temperature is slowly lowered down in exponential fashion. This shows great results and can be added after the fact of initial pretraining as a finetuning method.Analogy to this would be a person "brainstorming a new idea" by just trying out to combine two completely different words together that they don't usually combine.
>>110185988More specifically it's the transitive property of equality
>>110186056Correct yes. I have the capacity to act out based on my data set of experience through life and query others or simply act upon it. I can walk into a circumstance of which I've never been a part of and use what I know to try and understand the things happening before and then engage to try and learn and be apart of what is happening before me. None of this can have to do with someone else prompting me to do so.An LLM requires someone to actively engage it to perform what is asked of it. Have any of the LLMs ever spun up your terminal and asked you anything without your input?
>>110185311>http://logicprogramming.stanford.edu/examples/peano/index.html>https://ocaml.org/manual/5.5/gadts-tutorial.htmlSigh...how can you not know such basic things like this? Do you even Peirce?You're given the rules of, say, addition. It's obvious you can derive 4 from them even if 4 is not in the data.
>>110186056Actually a better question for you. Do you know how these LLMs attain and tune the datasets of which they pull info for you based on your query. You do know that it is through RLHF right?
>>110186158An LLM is a collection of weights running in an inference engine. Of course it can't spin up itself, then it'd be malware instead.If someone takes a human brain and puts it in a vat with some switches to toggle whether the brain activity reaches a computer or not, the brain wouldn't be able to act on its own anymore. It'd need someone to toggle the switch to enable it first. Same shit for the LLM.
>>110186181Yes, and do you understand why they're trained that way? Because training an LLM on something that people don't find useful doesn't make money.The whole economy is RLHF applied to every single human currently alive.
>>110185311>how humans can solve novel problems if the answer couldnt possibly be in its training data?By combining unexplored paths in different ways.
>>110185311consider a machine that collects and analyzes any data it can get out of a human beingthe inner monologue/thoughts, handwriting and notes, contents of favorite books, medical data, drawings, etcthe human is then tasked to invent a new word, for example 'pollepimpo', which itself has no meaningthe machine is then tasked to invent the exact same word on its own, without database lookup or bruteforce, based only on the collected and analyzed datacould you not say that this machine is capable of human thought, when it succeeds?regarding chinese room
>>110186269Human thought is not just it's language output. Qualia is the most fundamental force in our experience of the world.
>>110186161>pea anowhat kind of name is that?
>>110186295>wiggas becoming philosophers once their jobs became automated
Sorry I can't hear you over your tripfaggotry
The same way any function trained from data points can produce an accurate answer for a position not in its data.
