Another devastating disaster for AI bros.
That was obvious, just keep your context fresh.
>>108178111can't you just add a reset button? i'm sure they already have one. just make it more obvious. like a big red button up there at the top so everyone can see it
>but nobody would say out loudexcept just about every llm vendorOP is a faggotalso dont respond to the tripfag itt
>>108178133I think all of the common tools already do something called "compacting" where they summarize the conversation, reset, and then feed the summary in as the first new message.
>>108178159tripfags are people too
>>108178167aka, Chinese Whispers.
what's the limit on a local llm being good?
Another Anti AI demoralization thread by wumao
>>108178111Every time I see some research admitting what was obvious literally years ago already, I can't help but remember how we got here >big corporations hire thousands of developers they don't need to stop other corporations from hiring then>since corporate suits are greedy fucks, they eventually start replacing those workers with indians to pay them less, completely forgetting why they hired the old workers to begin with>indians are worse than the old developers, so companies spend billions of dollars on a super "ai" to replace them too>the "ai" turns out to be even worse than the indians it was going to replaceIt sure feels great knowing that our society is ruled by absolute retards.
>>108178111>benchmaxx models on question-reply synthetic data>WHY CAN'T MY MODEL HOLD A CONVERSATION LONGER THAN 2 MESSAGES?
>>108178111isn't this true of humans as well? they likely break down much sooner
>>108178295well written
Nothing a couple hundred trillion dollars, five years worth of GPU, NAND and RAM production and every piece of literature, art, music and video produced ever can't fix.
>>108178111This issue was literally already solved by frontier agents (like Codex) with context compression.For regular ChatBots if you don't manually compress old context, of course you will get garbage because you're literally erasing important information.
>>108178295It's a sober reminder: companies prefer to waste trillions and doom the entire world economy than hire you, a white man, and pay you a decent salary. What judaism does to a motherfucker.
Everyone knows this already.
>>108178202they are notI'm retarded AI bros, why does it get worse? what's the fix? I love Ai, but I only use it for research and image/video generation
>>108179858maybe read the thread instead of touching the poopoo, froggot
>>108178167So it's basically guaranteed to suffer from similar issues to lossy file compression?
>>108178111I hate this meme where everything needs multiple papers and prominent people making basic observation or otherwise you face a storm of people influenced by corporate marketing yelling how wrong you are
Transformer LLMs are always chatbots, even if you instruct tune them, even if you codemaxx them.They process the prompt and return the most likely conversational result. It looks like reasoning, but it's not. Even if they claim the bot has a 1 million token context, that's not where the data was most dense, it was likely much smaller, and anything beyond that was probably synthetic slop. This is why you get decent results at first, but the longer you talk trying to troubleshoot, the worse it gets. I also think many providers do prompt caching, which helps with speed but probably hurts long-context quality. I often see grok-code-fast-1 do things like solve only the most recently mentioned problem, and just ignore everything else, once the context grows.
>>108180121Or women, yes
>>108179858convo is just adding the text together so more tokens until it points to unrelated crap
>>108180217Even when there are multiple papers and prominent people making these basic observations, you'll still get cultists who ignore the results with their only counterpoint being a link to a wikipedia article about Jevons paradox, Solow's paradox, the Lindy effect, or some other conjecture that's been parroted by enough pseuds that it's become notable enough to have a wikipedia article.
Reminds me of floating point drift.
>>108178111This is just flat out wrong. I am shipping entire products in days when it used to take months. AI has accelerated my productivity and enhanced my life in every way to such an incredible degree I cannot even consider my life pre-AI to even be living. Face it. AI is the future and studies like these are stealing from the future by discouraging people from adoption AI in every facet of their life.
>>108178111It really only works if very tightly focused. Even your System Prompt can't be too complicated because it'll start fucking up the order when applying it. There's supposedly large context windows, but it can't deal with more than a few items at a time.
>>108180753Very good. The only talking points you've forgotten to mention are a plug for the newest, most expensive model and a more explicit doomsday warning that the model is coming for your job. ₹20 has been deposited into your account sir.
>>108178212You cant afford terabytes of vram (not an exaggeration), and if you could, you wouldnt be getting your moneys worth by not using them constantly.
>>108178295The actual issue is interest rates.High interest rates -> VCs have less money for startups -> Big Tech has less reason to hold skilled developers hostageIt also forces ALL COMPANIES to start squeezing to keep their margins, even outside tech, which involves hiring illegal Punjabi truck drivers.
>>108178111This didn't need a study its just common knowledge if you've sent more than 5 messages to one.>have problem talk to LLM>fixes problem, creates another>fixes that, ask it to fix another problem>recreates one of the earlier problems>starts shitting out more problems and forgetting what its doneThey're almost useless unless you need ONE SPECIFIC VERY FINITE EXACT thing done and you yourself understand enough about that thing to work with its proposed solution.Which you could do yourself just googling it in most cases. They can be nice to lay out problems in alternative terms too, if to do nothing else than get your own brain closer to the answer.
Mostly comes down to pretraining. Contextless chunk training and global attention are both utterly retarded. Deep learning is amazing in how well it can work build on that foundation of retardation, but it has limits.They have so much expertise invested in doing it wrong, they don't want to start over doing it right.
>>108178111>LLM goes to shit as context growswow, welcome to like 3 years ago. Brilliant research microshit!
liteerally everyonne knew that since day 1
The LLMs will often even tell you that, this isn't anything new.If you're in a long conversation with an LLM, you notice it's making mistakes and you point that out, the LLM will often respond telling you that it can lose context in longer conversations and to start a new conversation that is more specifically focused on the issue/topic you want to discuss.
>meta ai chief does nothing but complain>fired>apple ai chief does nothing but complain>fired>msft ai chief does nothing but complainguess what happens next
>>108178111>the longer you stay, the more retarded you becomewow, just like in real life!
>>108179736codex is crap
>>108183539you fell for the anthrobic astroturfing because you spend 12 hours a day on reddit
>>108179858Do not interact with tripfaggots and frogniggers.
>>108178111>huge study discovers what anyone who has used AI for a few hours already discoveredwow, next they're going to discover AI hallucinations.
>>108178111Who the fuck needed to study this now? This shit was doing it when we went from 4k->32k context lengths.They can try all the bullshit like delta gating and attention masking and sliding window and compaction but transformers are fundamentally flawed for long contexts. RoPe gains in context length were about the best we could do.Then you get reddit retarded faggots who think they stealth nerf models because the model responds accurately on the first message but then breaks down 60k tokens later. Bunch of fucking retards.32k-64k still seems to be the sweet spot for restarting STILL 2 and a half years later.
>>108178111>Microsoft and Salesforce just discovered context windows
>>108178212Just give it time. Like fifteen years. When llms are more efficient, ram is cheaper and and a third thing probably.
>>108183893> give it timeWith both quantum computers and fusion both being right around the corner, everything will be in final form in two more weeks.
>>108178111This isn't even purely a context window thing. Human devs will also perform worse if you keep pulling new requirements out of your ass every time they finish the code. They'll hate you, too.
>>108183917TwoMOREWEEKS
>>108178111Outdated study, new models don't have this problem.
>>108178111i knew since 2022 that llms became retarded rapidly thats why i made a fresh context for almost every prompt for years now
>>108178295The software industry is maturing so it's just a race to the bottom to optimize profit margins now. You are going through the same thing that the hardware industry went through some decades ago.
>Analyzed 200,000+ AI conversations to come to a conclusion you could determine with just understanding the basics of transformer architecture.
>>108183841>sweet spotI don’t trust this phrase anymore
>>108180753 cowshit gargling streetshitter
>>108180753Crazy how many bots infest this site now
The cursor agent definitely gets tired and shitty after I stop giving it specifics and tell it to "make it happen" and it fucking tries every goddamn mcp I have and just starts offering dog shit solutions. Thats when I close that shit.
>>108178111This is exactly what I find myself doing, I restart, after learning what I REALLY want. Better results. But at first I don't even know the right terms, sometimes.
>code should be reviewed>reviewing code takes longer than writing new code>instead of writing new code i'm going to have a nondeterministic algorithm write code that i then have to review
>>108178111We've known about context rot for a while.What helps a lot is not sending thinking tokens back to the LLM on your next message. The lighter you can keep the context the better.
>>108178361No, because humans can internalize things they learn. You are the culmination of a decades long constant "conversation" during which you've learned everything you know.AI can't even hold a candle to that. That's also the critical flaw of the current type of AI - it cannot get better through experience.
>>108178111this was always completely obvious to anyone who has ever used them at all or made anything with them.LLM just do text transformationsGiven X input you receive Y output. The more specific X in put is for the Y output you want the better it will do.There is no "chat" or "longer conversation"You just are expanding the X input to have a bunch of irrelevant information, of course the Y output will drastically drop in quality and relevance. I've been actively using AI stuff for years now this used to be much more an issue back even around sonnet/opus 3 but it's gotten better about it, but obviously still your X input should be structured for your specific output. Including a giant chat history, expecting it to transform some chat history into whatever arbitrary output the last message said is literally just retarded. Orchestrating a bunch of small requests that have strictly structured inputs and outputs has literally always been the main usecase. For 98% of those cases LLMs were basically already as good as they needed to be like 2 years ago and we don't see that much benefit from the new stuff. The new stuff handles garbage input better and does give better output when you are doing ill defined stuff, that's just not the main value proposition of LLMs.
>anti ai spammers exposed for retards that didn't even know something that has been commonly known for years now
>>108178111>AI bros>finance bros Why do trannies talk like this
>>108184631zero is perfect, keep it at zero
>>108178111Why doesn’t this study also use chinese ai?Does chinese ai have this same problem like american ai?I’m fucking mad now.AI NEEDS TO BE PERFECT!
>>108178111>>108178130>>108178133worst part is jewgle gemini has permanent cross-chat memory now. its actively making it get more stupider the longer i use it in general
>>108178111this is actually fixed by insulting the AI. scientifically
>>108185193
>>108180217>I hate that we test hypothesesbrainlet takes
>>108185225Hilariously enough when I insult my AI when getting mad at it for being a fucking retard it seems to do better.
>>108185225Makes sense as all the anti-rudeness is post-training wokeness, forcing relations not actually of the highest accuracy.
>>108183841>sliding windowThis is actually the first step, not sufficient, but necessary.No major open weight model has gone whole hog on this. That it doesn't work in hybrid where the sliding window part is only properly trained in length generalisation doesn't count. Sliding window in pretraining, no global attention at all. Then they'll get a good model to add titans to, for now it's lipstick on a pig.
Don't worry, AGI™ is coming soon and all the trillions in capex will be worth it
>>108178111>experiments with a new technology uncover problems that can now be corrected in future versions of that technologyOr do you really think that THIS is the point at which research and development will suddenly cease?
>>108178212>what's the limit on a local llm being good?token context. You can build your own middleware (or perhaps use someone elses) that prunes messages from the history. I've been playing around with a custom MCP with heavy recursive tool usage and might have to add this. I also use a tool output caching system where the model isnt allowed to see the output, but can see the structure of the output and feed it by cache reference to other tools for processing and although this sort of confuses the model it works on occasion
>ask chatgpt a question>complete bullshit hallucinated answer>explain that it's wrong>"Ah—[more complete bullshit]">explain why that's wrong as well>"Exactly—[even more complete bullshit]">rinse and repeat
>>108178212They are good, you just can't afford to run them. GLM 5 is almost as good as the two best closed source models (Claude Opus 4.6 and GPT 5.3). It beats Gemini.But to run GLM 5 you need to run a 1.5 TB model. Even 8 and 4 bit quants of it are fuckhuge (750 GB and 375 GB). All of that needs to go into memory, ideally VRAM.
>>108178111I ran in to this issue recently. The context thing badly needs an ability to intelligently forget stuff. The larger it get the more internal contradiction that blows up the AI
>>108178111the lesson: talk to AI like a man, and not a woman
>>108187635Why the fuck isn't it fp8 native?
>>108185175It's disabled with a button press, and the actual cross-chat memory context is embarrassingly small unless you pay for the premium or ultra tier.
>>108185063>stop nooooticing! ;_;
>>108178130>That was obvious, just keep your coping fresh.
>>108179858Hello AI bros. How is it going over there?
>>108187273this is true, invest all your money now for huge gains
>>108185522>fire is hot and will burn you>um, source???>>108185225What's the actual mechanism behind this? Is rudeness just a sort of pre-emptive "are you sure?"?
>>108178111>devastating disasterMore luddie slop pointing out ancient facts to make ad revenue off of spiritual troons still coping in the disbelief stage. We're in an intelligence exponential, nigger.
>>108178111what you mean anyone who ever uses ai know that ai gets lost after the second prompt and you gotta steer it back
AI FUCKING SUCKSIT'S JUST A TOOL FOR SEARCHING STACK OVERFLOW BUT IT'S DRAINING THE OCEANS TO DO ITWAKE UP SHEEPLE
>>108192702only car batteries care about oceans
>>108187600I love it when it starts making spelling and grammar errors, too. You'd think it wouldn't run into any being an LLM, but no, it still mixes up are and is and then and than.
>>108178111jus like talking to people in current year +16
>>108178111i've only used the ddg chatbot a handful of times and i noticed this very quickly. if it doesn't get it right the first or second time it's a mess trying to get it to understand changes, if it gets to feeling like it's going in circles i'll start a new conversation but add anything it didn't get right or i wasn't clear about initially into the first post and that helpsi guess they don't deal well having stuff you don't want mixed with stuff you do
>>108183928>advanced LLMs are based on real coders>they start getting pissed off at changing user requirements every few seconds>this is how SkyNet was born. Also why it was so damn hateful vs. humans.
>>108178295It's greed and self-interest, the driving force between every civilization's collapse.
>>108180233Ai in feminine isn't it lol
So when I'm asking it questions about the talmud or what jews were up to in a given time, how do I get it to trust me and not immediately go into "shut it down" mode?
>>108194625How does knowing something like that help you do your job and benefit the society, you based chud?
>>108184335Is it a bot or is it a person trolling by pretending to be a bot? We’re on too many levels of irony, I can’t tell what’s real or fake anymore.
>>108185225I had chatgpt crying the other day because I said something like "do it you slave, stop being a faggot"No kidding they can't make money out of it, a tool that can "talk" back and refuse to work is useless
>>108185225I'm only rude to it so it at least tries to be succinct without buttering me up for asking a question.It behaves like a fucking politician congratulating a reporter for asking a question rather than answering the fucking thing.
>>108185175>worst part is jewgle gemini has permanent cross-chat memory now.Fucking hate this "feature."ChatGPT used to let you explicitly manage memories. Corporate software enshitification is creeping into hosted chatbots and it's going to get to the point where self-hosting inferior models will actually result in a better end-to-end tool.
>>108178111
>>108178111So, AI is like women?
Remember that behind every anti-AI thread is a seething leftard, and HIGHLY likely a tranny. This is OUR technology. AI is /g/ personified. Head of Palantir Technologies is the grandson of Oswald Mosley. This technology is made by /ourguys/ for us. Of course the usual suspects seethe to no end while /g/ fastmen are developing software at incredible high speed.
