I have no way to definitively prove this but I think the model being used by " AI mode" is a retarded single digit parameter in-house cloud model. White House. Would it be getting such simple questions wrong? From a scaling standpoint it kind of makes sense to use a such a tiny model for it since given Google's recent push to be an " AI company" and it bolting "AI Mode onto Google search. Serving AI on practically ALL Google searches (And by extension practically everyone since everyone uses Google at some point) even while not being logged on Would it be stupid expensive if they were using the "smarter models" while not expecting anyone to pay for it via a subscription or token pricing. I think this also tracks because single digit models are OK (nothing to write home about) at document summaries and simple tool calling and Google's" Effective" series models prove this. I think what the results in pic rel show is that in whatever back end model they're using is good at using AI Mode tool calling in order to fetch and gather info in order to serve the user the "correct" info but, like most single digit models, are utterly retarded for basically anything else. It's good at fetching information, but I bet if you sandboxed this thing and then asked it simple general questions or log it would fail most if not all of them. Oh, y'all didn't probably much dumber than even 2b or 4b "Effective" models Google released as open-weight models this year. Asking it logic questions Would cause it to "think": " I need to answer the question to the best of MY ability on my own" instead of what it does most of the time it just does a web search so that it can get the answer from somewhere else. The internet doesn't really have a bunch of random logic puzzle articles floating around for the specific " how many letters are in this word" pages so I think that's why it fucks these up so badly. Or I could be completely wrong and they're just THAT incompetent. Who knows
Gemma is pretty nice.
>>108920940What I don't get is why they don't just generate a response with a better, but more expensive, model once something's been searched a certain number of times and then just cache that for future searches. This could cause problems like getting out of date, but you could do something like use the small model once every x searches just to see if the response is wrong, then regenerate the result again if it comes back as a yes. In any case, I'm sure the people at Google could figure this out.I unironically think Google's AI integration has done huge damage to the reputation of the entire industry. You already have morons who think the results of LLMs proving Erdős problems must be fake because they used the free version of GPT-4 once two years ago and it couldn't understand decimals, but now you have literally everyone in the world being exposed to Google making elementary mistakes every day that not even the free versions of ChatGPT, Claude, or indeed Gemini would ever make
>>108921176Web results poisoning the reply.
google forces the model onto every web search, it has to be extremely light on compute and such
>>108920940>I have no way to definitively prove this but I think the model being used by " AI mode" is a retarded single digit parameter in-house cloud modelI mean that should be pretty obvious if you consider that they run it on almost every request
>>108921312But the summary can be pretty specific where it only needs to summarize the most relevant 5 pages for a search or so
Gemini is only usable in API form or AI Studio. App version and search AI mode are neutered trash for normalfags.
>>108920940>>108921312>>108921713>>108921176It does better when you're in a dedicated AI mode chat window so maybe the Google searches that trigger AI mode get routed to a dumber model and get routed to a "smarter" one when you're in the chat window?
>>108920940
>>108921176it can't, the prompt includes identifiable information, like location, date, and likely some personal info if you're logged in. also some answers can't be cached, like "has X passed away?"but yes, AI is really dumb. agents are just a cope because models will only get meager improvements while costing 10x as much to make every time. They're just data compression/search algorithm, like that Erdos problem, where the model found a proof inside a paper that proved something else, the problem was already solved, it just had no published proof.
>>108921864I dont really understand what people find fascinatng about the fact that letters are encoded as tokens inside the model for efficiency reasons, which then has faults when you start counting those hidden tokens.
>>108920940qwen is autistic as fuck but it does correct itself at least. google just hallucinates to you confidently
>>108921176i mean the slop served to normalfags is pretty shit, but once you start exploring local models adequate for your hardware you'll get way more bang for your buck and i'm sure it goes way beyond dumb predictive text completion.
>>108922583I think it's the "pulling the curtain" effect, most people don't understand a lick about LLM's so when it fails a simple task (for a human) it shows that it's not thinking like a person.
>GoopleJust had the most retarded giggle over the idea of the LLM saying "Goople" causing the entire AI industry to crash.
>>108922648it's a process akin to thinking, but scaled way the hell down due to the nature of neural networks so you can't expect to shit coherent senses from a gameboy. on the other hand it's way closer to an isolated frontal lobe analogue that can rationalize to some degree by instinct but it's way a far cry from a sentient being. it's more akin to a baby parrot that processes the world through pure thoughts of text.
>>108921176>retard believes the erdos problem solutions were realRead the papers, sub-0 IQ ainigger.Yet more proof that the only people who like AI are clinically retarded.
>>108922667It's a process nowhere near even remotely related to thinking and you would do the world a favor by stopping to waste its oxygen right now.
>>108922667hopefully it's not too late for you to remove yourself from the gene poolIt's nothing like thinking. This algorithm just overlays a bunch of q .05 jpegs shifted by one pixel and calls whatever comes at the edge an answer. The best "intelligence" that can happen is basic substitution. Linguistically there is no statistical relevance between google, p and 0 so no amount of text off internet will make it an output.What can happen instead is translation to a python script if it's trained to notice the letter counting task pattern. But that's specialization. And ironically that's mostly what's propping up the bubble. Ai-bros are selling basic bitch computer tasks back to mouthbreathers like you, just hidden behind a natural text interface.
>>108922583Why should a model be so retarded that it counts the wrong hidden tokens instead of what was asked?Google shouldn't change search to a very retarded AI model and expect praise rather than disappointment. It's clearly the same thing here: People would expect at least a decent reasoning model's level of correctness.
>>108922642>it goes way beyond dumb predictive text completionUnless you mean by that some workaround bullshit like thinking where it writes the answer and then tries to fact check it or go step by step then you are wrong. They all are and will remain next token predictors unless breakthrough happens.
>>108923857>he still insists there's any counting going onIt's literally see "count" say number. The peak in the histogram is in the wrong place for that combination of words, so it outputs "two", most likely because training text discusses number of os.
>>108923857just watch this vid https://www.youtube.com/watch?v=7xTGNNLPyMI
>>108923915 >>108923929It should count correctly like not buggy not retarded models do, that's all there is to it.Google pushed a piece of underperforming buggy trash into one of the most visible places they could have, that's all there is to it.
>>108920940>proprietary garbage gets mogged by free software alternativesmany such cases!
>>108921864CRINGE!!!!
>>108920940>>108924046I think you mean>tiny garbage cloud model made by google gets mogged by bigger local model made by googlePicrel was done WITHOUT hooking it up to any search engine and WITHOUT thinking mode enabled. It just one-shots the answer which proves that the cloud model Google uses is some tiny model in the sub 4B range that got quantized to shit.
>>108924625ask it about strawberry
>>108924704
>>108924791>that snowbunny strawberrykek
I hope they don't fix it, it's pretty funny.
>>108920940indian-made software hard at work.>SIR I AM REVERTING BACK WITH THE LETTER COUNTING IT IS CALLED AS...
>>108924867>goigle>goiYou aren't supposed to let the goyim know Rajesh!
>>108922618>qwen is autistic as fuckYou kind of want that to be the case if you want these things to be useful. Mouth breathers at /lmg/ will tell you otherwise, but they're too focused on nudging the things to make them nut so those specific kinds of anons aren't really worth listening to sometimes.
>>108923995Retard. It's not counting anything. The dudes ITT that keep saying " it's just the next token predictor" are entirely correct...... There's no "bug" causing this behavior because Google serves a small, Dom, and likely heavily quantized, AI model to the initial AI mode search queries. Based on how it behaves it's okay at doing low-level web fetches via tool calls but literally nothing else.
heheheheheheheheheh
>>108920940AGI soon
>>108922649the issue is that 90% of the people don't care, they don't question the results, no matter how nonsensical. but it would be extremely funny.
>>108925376true enough. i mean i'm trying to use ai to help me learn and work faster rather than erp with it, and i think gooning with an ai model trained mainly for it/engineering adjacent work is just a waste of potential.
