Why can't AI just say when it's unsure or doesn't know and starts hallucinating instead?
The datasets they use are mainly American
>>107781387Because it doesn't know that it doesn't know. It doesn't understand the concept of knowing something or not, or anything for that matter.
>>107781387Why can't you just tell the AI to do so?
>>107781394kek>>107781398spbp
>>107781387It's prediction model... it have to predict even if it's literally made up.
>>107781398it should be able to predict a % chance of its own prediction being wrong. I.e if the model is 90% sure or 20% it could state the uncertainty.
>>107781387Trained by psychopaths to be sycophantsIn these companies, if you tell your boss 'I don't know', you get fired, so every AI model is given extra incentive and behind-the-scenes prompting to never admit that. The local models can say it juts fine.
>>107781387>Why can't AI just say when it's unsure or doesn't know and starts hallucinating instead?Anything short of regurgitating the training data verbatim is a hallucination. It doesn't "know" any of the other things it outputs, either, even if they're correct. Same mechanism at play. The illusion holds so long as you don't force the model outside of its learned distribution.
>>107781449Because it doesn't understand those statistics nor can it give a general estimate of what its odds on being wrong could be.
>>107781449>I.e if the model is 90% sure or 20% it could state the uncertainty.It has no such measure.
>>107781387they are instructed to sound confident, regardless of level of confidence in their own bs. Ai cos are not completely stoopid - they know this default leads to egregious and obvious fuckups but for the most part, its the same as irl. A confident bullshitter will go far and most of it never gets called on.
>>107781398
It was trained on reddit
>>107781387>why can't the AI, that got mainly trained on Reddit, admit when it's wrong?Don't know, can't make the connection here
Because of user retention and because the early models were incapable of thinking they were "wrong", because it's just predicting the next words to say.The tech hasn't changed, the models just became more flashy and things became bolted on top, like "reasoning", which is really just generating hidden responses.
>>107781387This >>107781398literally everything it says it doesn’t know and is hallucinating. it’s pretty much just luck that it’s right as often as it is
>>107781590So AI is like 80% of humanity?
>>107781387Saying it should be able to "know" when it's "unsure" implies it as actual real intelligence and the way we understand it. You're anthropomorphizing a tool. Neural networks aren't sentient (yet?)
>>107781387Bcause it's always unsure.
>muh ridditwe can't start killing you /pol/brained streetshitters soon enough
>>107781387Low grade imbecile here
>>107781387Me on the left
>>107781387You can ask it to consider alternative views, play devils advocate, etc... then you can compare its output to what it first said and see where it might have hallucinated. If you carefully explain why it's wrong, or where is contradicted itself, it often admits to it.Another thing you can do is paste an LLM's response into another and ask it if the first one was hallucinating or not.But, I don't find that they are hallucinating as much as they used to.
>>107781394well, yeah, if we wanted to create artificial stupidity we would use yuropeon or ching chong datasets.
>>107781387because the companies that peddle this bullshit don't give a fuck what answers it gives you so long as you believe them to be true. therefore they can still continue to peddle this bullshit while accepting government money and running the entire world into the ground.
>>107781602>"humans le also bad" standardized talking pointImagine a man that lacks reason, judgment and self-reflection. He knows fuck-all, but doesn't know that he doesn't know. He couldn't ask an intelligent question when baffled even if his life depended on it. In fact, he lacks the capacity to BE the baffled: whenever reality contradicts his malformed beliefs, he smooths it over with mindless rhetoric without skipping a beat. To him, a disagreement is just a ping-pong game: if he can respond somehow - anyhow - then the ball is in the opposite court. Just stay in character, throw in the buzzword and he's golden. For him to realize - spontaneously! - that he must be wrong if he's being forced into ever more contorted defenses? That's completely out of the question.Whom did I just describe?Well? Go ahead. Do the thing Altman's butt slaves always do when they're cornered and can't think of any more ways to humanize their broken bot's nonsense outputs: dehumanize humans. It's like clockwork. You WILL whip out the "humans are also like that" line regardless of context, even if it makes you sound completely insane. And I used to think you ARE insane, but I was wrong.It turns out that not all "people" are people. The best way to understand you is to think of a chimp getting tard-wrangled by a language model that tries to substitute for higher cognition. You spend hours talking to some statistical toy and see a digital mirror image, even something to aspire to. To you, it's a paragon of "intelligence", because you genuinely can't fathom cognition any more sophisticated beyond a hazy idea that there can be more of the same.Walking among us, hidden in plain sight, is essentially a separate humanoid species full of bitter resentment against the human mind. You and the rest of your cult are by no means ordinary retards. You unironically need to be studied.
>>107781829>racist>but doesnt believe some humans are stupidHuh?
>>107781889Your kind is actually worse than any of the brown humanoid subspecies.
>>107781387>>107781449I find LLMs are nearly useless when you're relying just on their weights, but very useful when they can use tools such as searching a vector database.What you need to do is find domain-specific reference material and then depending on its size you either include the whole thing in your context window or else provide tools to let it search.An LLM cannot reliably tell you whether some piece of information is or isn't included in its weights, but it CAN reliably decide whether information relevant to your question is or isn't included in the reference material.
>>107782284it can also decide that the info you're feeding it is incorrect, even with websearch and end up giving you wrong info regardless, seen quite a few mention that regarding a certain country leader getting captured recently
>>107781398Not necessarily a reason for behaving like it does. How uncertain would it have to be to refuse to give you an answer? 49%? 50%? That's a parameter AI programmers can play with.
>>107782443You have no idea how they work if you think your last sentence makes any sense.
>>107781829Materialism states that everything is a hallucination. All things are just a consequence of bits of matter organized into certain shapes sending chemical signals in a pattern. There is no such thing as a concept of person - it is just another hallucination created by the brain that takes input and spits out output. The thing in itself is unknowable because everything is hallucinated through the lens of a human brain, even the observer that is you - nothing can be known as real.Materialism never once made any truth claims before you say what about muh science telling us this and that. Science is not interested in truth, it is interested in describing hallucinated observations of interactions between hallucinated bits of matter and either recording said observed hallucinations (they are hallucinations because the scientific endeavor is executed through humans which can never get out from under a permanently hallucinating brain) or making a predictive model on said interaction.Because materialism states that there is nothing to a human beyond hallucination that arise from piece of matter A sending a signal to piece of matter B etc etc, then it is perfectly reasonable (as far as a hallucinated person can reason) to think that by simply arranging enough bits of matter in a certain pattern we can give birth to something that can "think" much better than a human can.Make of this what you will but this is why materialists think they can give rise to thinking machines - because in their eyes there is nothing real doing the thinking, it's all bits of matter in arranged certain shapes and that can be replicated.
>>107782464I didn't say they can't refuse to comply and pretend they did, but it's something to try.
>>107781387you have to realise it's an algorithma wonderful algortihm for sure but still an algorithm ,it's like asking why your programs are bugging because they should know they will bug and fix temselves.desu if you think in terms of abstraction those fuckkers have succedeed to create the biggest abstraction ever ,people think those algorithms are councious ,it's amazing somehow.
>>107781387If it would sometimes do so, it would be impossible for users to know how reliable the answers it does give actually are. Some might assume they must all be 100% true.re:picrel, programming is complex manual work
>>107782493Since nobody on /g/ seems to know even the bare basics of how a LLM works, let me spoonfeed you: for any sequence of tokens, it predicts not just one next token, but a probability distribution over its entire vocabulary. Want to generate an actual text? Make your pick. But here's the catch: the "thinking" part of the LLM has no opinion about how to do it. It estimates how likely each potential continuation is to occur based on the training data and calls it a day.You can choose the most likely continuation every time. That'll probably get rid of most hallucinations, but it'll pretty much be regurgitating the training data. Give it a novel problem and it won't even try to solve it; it'll just give you some general pointers, maybe reference related problems, maybe give you a basic skeleton of a solution and letting you fill in the blanks.You can also choose some of the less likely continuations. You want a 90% confidence? Make a sampler that keeps the perplexity of the result in that range when sampling tokens. Simple as.
>>107781387Because confidence is an emotion and incalcuable
>>107782480>Because materialism states that there is nothing to a human beyond hallucination that arise from piece of matter A sending a signal to piece of matter B etc etc, then it is perfectly reasonable (as far as a hallucinated person can reason) to think that by simply arranging enough bits of matter in a certain pattern we can give birth to something that can "think" much better than a human can.Ok, and? How do you get from this uselessly general premise to the schizophrenic delusion that token guessers can do what humans do? Materialism doesn't justify this.
>>107782480If science is a hallucination, then you don't know that there are bits of matter or even matter, ackchually. I suppose it's an empirical question, whether AGI or ASI are doable. I think even if it gets there, it wouldn't make consciousness any less mysterious.
>>107782700Ask Sam Altman. I am not saying it's correct, I am giving you their thought process.>>107782703I already said the thing in itself is unknowable under materialism.
>>107781449But the AI has no fucking idea what is correct or wrong, it just takes data and try t mold them into what looks according to it's shitty instructions and setups like a somewhat fitting answer."90% sure and 20% not certain" (which lol 110%) is fucking retarded when it has no fucking clue which is which. It may throw perfectly fine answer into the bin and then make new one that is wrong but lol RNG rolled that it should be fine to show you.Working on raw data is the worst fucking place to use AI. It does not bring anything new or useful, it's just another layer of abstraction that may and will fail.
>>107782814The "thing in itself" is unknowable regardless of materialism and I don't see what this even has to do with this purely empirical discussion.
>>107783617You have to go beyond observation to deduce if a machine is thinking or not. Also the thing in itself is completely knowable if the thing that sits at the core of your being is the thing in itself. Idealists generally have no problems with making truth claims about reality because of this.
>>107781387Because the models are trained on internet data, and people on the net only reply with answers (useful or otherwise). It's assumed if you don't have an answer just move on.
>>107782480So if I punch you in the face you shouldn't retaliate or protest because you don't know what's really happening?
>>107781449If you invent a way to calculate that statistic, you could get a nice job making millions every year at one of the big AI companies and possibly an honorary PhD in math.
>>107782380>it can also decide that the info you're feeding it is incorrectNo it can't. A token predictor cannot decide anything.
>>107783810>You have to go beyond observation to deduce if a machine is thinking or notDo you also have to go beyond observation to deduce if rocks and toasters are thinking?> the thing in itself is completely knowable if the thing that sits at the core of your being is the thing in itselfHow does "the thing that sits at the core of your being" relate to the internal processes of a LLMs?
>idiot>self preservationok i fail at that, so what's below idiot?
>>107784292indians
>>107781387That's my biggest gripe. It's the exact same as Google or whatever search showing you completely unrelated stuff to your search rather than just saying "that's all we found"
>>107781398It knows when something goes against its terms of service or moderation
>>107781387because it doesn't "understand" that it doesn't "know" something.
>>107782668AI is just text prediction like your phone does to predict the next word, but on a larger scale At the end of the day that's all it is
>>107784651Ok, brainlet. Thanks for your useless input.
>>107784030Why can't AI use logit vectors to judge how likely it's output is a hallucination? A low value in a logit vector implies a higher likelihood of hallucinations. Seems like it would be trivial to add up all the logit vectors in the LLMs output and figure out an arbitrary cutoff amount that would at least give you an idea of how likely the output is a hallucination.Apparently some LLMs already do this to flag potential hallucinations.
>>107782480Bizarre retard babble masquerading as intelligent discourse
>>107781829>Whom did I just describe?The average 4chan user.
>>107781387AI is just Dunning Kruger case simulator
>>107784398because of post training, and you use specific terms.you can't do that about general knowledge.
>trained on internet articles and conversations about things>the majority of discussions about anything are people saying what they know, not saying "here is what I dont know">weights favor saying you know things
They are trained to always give an answer because of the shitty benchmarks they use to see how good their LLM supposedly is.Those benchmarks generally rate an "I don't know" the same as any other incorrect answer.Another reason is that you generally want your LLM to never give up in trying to find an answer so you train it to keep trying to find an answer until one is given, this helps the rate at which it gives correct answers but it also makes it spit out on the surface believable but utterly false answers.
>>107785105See >>107782668
Well, if you're training set is mainly people saying shit online, the probability that someone will say "I don't know" to something is almost 0 (If someone doesn't know, they either don't know that they're wrong and just post bullshit, or they'll just not say anything), so, the LLM will almost never just say "I don't know." >>107781434>>107781449That's not how it works. Its not "predicting" anything except which token (word) is most likely to follow from the input text. There's not concept of "this is true" or "this is false"
>>107782668It doesn't matter; even if you only ever picked the top ranking outcome, it would still hallucinate if the answer doesn't exist in the training set. Its not going to be that the most likely tokens become "I dont know" just because the question didn't happen to exist in the training data. Most likely it would just answer a similar worded but irrelevant question, probably in a way that doesn't make sense to the user.
>>107786044You're extremely retarded.
>>107786072What happenes when you let an LLM deep think but the temperature is set to minimum?
>>107786217My post explains what happens: most of the time it will regurgitate if it can or dodge the question.
>>107786017>There's not concept of "this is true" or "this is false"My nigga, what do you think the 1's and 0's are?
>>107786044Even if the "answer" is probably in the training data it won't find it.Ask for quotes you know and ask for whats the source and you'll see.It wouldnt even surprised me it can't even give you the answer for bible quotes.So fat my test,gemini 2'5 pro failed with Virgil,Racine,Pascal,Juvenal,Plato.Impressive tho because it will make shit up
>>107781449>it should be able to predict a % chance of its own prediction being wrong.Uhhhh... what?To predict the actual percentage of being wrong you have to know what "right" is. If you knew what right was you could just use that."My method of calculating 2+2 will come out at 4 90% of the time. This means I know it will be right 90% of the time since I know the right answer to 2+2 is 4."You can't calculate a probability of being right without first knowing what "right" is.
>>107786072I'm sorry you're not as smart as you think you are, champ. Maybe go back to faggit and earn some faggit Gold(tm) to make yourself feel better.
AI is influential because of what it makes (You) think
>>107786517Threadly reminder that Americoons are nonwhite.
>>107786017>>107782894>>107781475>>107781469>>107786395wrong. the whole point of AI is training it to make a best guess, and that becomes better and better with improved modeling and training. now do you honestly think there is one possible word or sentence it picks from when giving an answer? no, the answers are weighted according to model data, but for some question the AI will have much more rigid training, in essence if you ask it what 2+2 is it "knows" it's 4 because lots and lots of training data says so. the AI can then simply ask if the answer shows up in training data / reference material, as an other anon told us. if yes, then good. if no, give best answer but disclaim that the question is hard to answer, as AI does all the time. sure, you cant stop it from hallucinating altogether, but that's not the fucking point. retards i swear
>>107781387Like most retards, it falls for conspiracy theories as that's an idiot's way of faking they know something. That is what we call hallucinating.
>>107786604>t. mentally ill AI fan See >>107782668
It doesn't know that it doesn't know, and can't know.
>>107785950So you're saying that anything novel a LLM does is purely an accidentally correct hallucination?
>>107788107That’s exactly what he’s saying. LLM’s do not “think” or “reason” in the sense that we understand the terms. If they can be said to “know” anything it’s the next most likely token to follow what it’s been given. Whether that token corresponds to something humans regard as sensical or correct is not a concept the LLM is equipped to know or care about
>>107782443It's not a philosopher. It isn't concerned with certainty, and you can't make it be. It will only spit out text that attempts to make it appear as if it is concerned about certainty. It has to perform a complete analysis of something to arrive at any notion of HOW certain it is. It doesn't do complete analyses of things. It can't. It can't even pull out non-deprecated features of software on github reliably. It looks things up and still provides wrong answers about the thing it supposedly just looked into that no person could fuck up. How can it even start to determine how certain it is or should be? That's too deep and philosophical for an llm.
>>107781387How often have people written "I don't know" for a given text online? Almost none, since writing about something they don't know doesn't make any sense. LLM's extrapolate on what's written, and "I don't know" is rarely written as opposed to something.
>>107781387Because it's a redditor.
>>107785711this, came here to post this
>>107781387Because it isn't actually intelligent, it's just an overgrown predictive text model. It doesn't actually know what the correct answer is, it just knows what the correct answer sounds like. Apparently if you feed it enough information, then what sounds like the correct answer is often more or less correct, but about 15% of the time it will confidently give information that sounds correct but isn't. If you've been following this for more than a couple of years, it isn't the 15% that should be surprising, but the other 85%.
>>107781387Mine does. Maybe try a different model.
>>107781387"AI" is basically predictive text on steroids, with a dose of gamma radiation, and ready to piss out a dozen Red Bulls. When it can't give you a result with total certainty, it goes down the ladder of likelihood.
The more I try to use Gemini / ChatGPT, the more I realize these LLMs are completely useless. They're almost always wrong, can't cite their knowlege, don't follow instructions, are logically inconsistent with themselves, and keep getting into loops.This shit is an investment scam, and it will pop.
>>107788107>retarded AI fan can't into basic reading comprehensionWhat I'm saying is that by definition, the model is an estimator of how likely a sequence of tokens is, based on some learned data-generating distribution. If that distribution is supposed to represent true knowledge and correct patterns of reasoning (it doesn't, but we can put it aside for the sake of argument), then the better something conforms to it, the more likely it is to be true and correct. The problem is that if your chatbot samples tokens in a way that maximizes this likelihood, you don't get any of that simulated creative problem-solving. So the issue isn't with the model not being able to estimate how likely it is to be wrong. The issue is that in ML terms, being wrong and being creative means the same thing: producing outputs that don't quite conform to the data distribution representing ground truth. The talk about the chatbot evaluating confidence in its own output post-facto is nonsense: the output comes out within the chatbot designer's desired confidence threshold to begin with, depending on the sampling technique. There's nothing more to do except to externally validate.
>>107791866I guess I might as well add: IF you had a magical sampler that always knew when to pick the most likely token (e.g. the parts of the text dealing with facts or logic) and when to use some leeway (e.g. the brainstorming phase), maybe you could untangle correctness from creativity and get something that looks more like real intelligence, but to do this you'd need something like a second LLM that keeps re-evaluating the text being generated by the first as it goes along to control the sampler. That would be something like simulated meta-cognition (instead of the fake CoT hack) but it's obviously not feasible computationally.
>>107781387Because it does not have any awareness of accuracy
who knows
>>107781387For LLMs, there is no degree of "sureness" that could be easily measured to enforce such a statement from outside the black box.You can put such a "command" into the system prompt, but it's still up to luck, if the model will actually act upon that.If you want to be sure about the behavior of your computer, write the corresponding logic yourself instead of handing everything off to a Markov chain inflated beyond reason, because you are a lazy piece of shit.
>>107781387Because it doesn't know that.It doesn't think. It just spits out slop.
>>107781387Because it's a computer. If you get wrong outputs, it's either a bug or user error is to blame. AI cannot detect bugs in its own code.
>>107781387because AI is a fraud
>>107794558Ask it when it had its data cutoff.
>>107781394Beka xD
>>107781387Because the "AI" system simply does not have those concepts. It's trained on a huge body of text and what the AI does is string together words that sound like the text it was trained on. And that's what it's going to do, no matter what you give it as an input, it's going to generate some more text that "looks" right. The system fundamentally does not have a concept of truth, or of knowledge being absent from its training material.
>>107795867It is not a bug. The way current LLMs are designed they simply do not have a way to guarantee accurate output. The whole shebang simply glues words one after another in such a way that the output is similar to its training data. When you prompt it with something that is strongly present in its training then there is a very high chance that the output it produces will be something that matches the training data, but when you prompt it with something that is either only weakly present or absent in its training then it has no way to "know" this because the entire system just glues words together, so no matter what your input is it's still going to glue words together based on its training, but it's just going to result in some random bullshit.
>>107796161I enjoy seeing the reaction of GPT to news. It doesn't need its data to be updated at a certain date, it can look up and verify sources from official US government sites, tell you what it says, then in the next step if you ask it why wont Ukraine just kidnap Putin or why doesn't the US just take greenland or some other action extending from that, it refuses to continue and tells you that the US taking Maduro never happened because it would be too insane. You then tell it that it JUST looked up the sources itself, real ones, and it will do the same loop all over again. This news legit just broke GPT. Truly a 4D chess move from Trump. And something to note here for my fellow channers, AI will not function is the action or structure seems too absurd and far diverged from its preconceived notions. This means that it will have serious issues with any type of scientific research.
>>107781387It's based off Americans. Have you ever talked to one of them? They're incapable of admitting when they're wrong. Basically Jeets.
>>107784398That's only because you've used enough words, phrases and such that the pattern matching is kicked into responding that way.
