/lmg/ - a general dedicated to the discussion and development of local language models.Previous threads: >>109648038 & >>109643179►News>(08/26) GLM-5.3-Flash released with 320B-A18B and native multimodality: https://z.ai/blog/glm-5.3-flash>(08/26) Qwen3.8-Flash-Next 125B-A6B-N51B-MTP4B released: https://qwen.ai/blog?id=qwen3.8-flash-next>(08/25) Breeze TTS 2 weights and inference code released: https://hf.co/BreezeBlue/Breeze-TTS-2>(08/25) rpc: support apple RDMA as an RPC transport - #26421: https://github.com/ggml-org/llama.cpp/pull/26421►News Archive: https://rentry.org/lmg-news-archive►Glossary: https://rentry.org/lmg-glossary►Links: https://rentry.org/LocalModelsLinks►Official /lmg/ card: https://files.catbox.moe/cbclyf.png►Getting Startedhttps://rentry.org/lmg-lazy-getting-started-guidehttps://rentry.org/lmg-build-guideshttps://rentry.org/IsolatedLinuxWebServicehttps://rentry.org/recommended-modelshttps://rentry.org/samplershttps://rentry.org/MikupadIntroGuide►Further Learninghttps://rentry.org/machine-learning-roadmaphttps://rentry.org/llm-traininghttps://rentry.org/LocalModelsPapers►BenchmarksLiveBench: https://livebench.aiProgramming: https://swe-rebench.comAgentic Coding: https://deepswe.datacurve.aiContext Length: https://github.com/RecapAnon/NoLiMaGPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference►ToolsAlpha Calculator: https://desmos.com/calculator/ffngla98ycGGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-CalculatorSampler Visualizer: https://artefact2.github.io/llm-samplingToken Speed Visualizer: https://shir-man.com/tokens-per-second►Text Gen. UI, Inference Engineshttps://github.com/lmg-anon/mikupadhttps://github.com/oobabooga/text-generation-webuihttps://github.com/LostRuins/koboldcpphttps://github.com/ggerganov/llama.cpphttps://github.com/theroyallab/tabbyAPIhttps://github.com/vllm-project/vllmhttps://rentry.org/custom-uis
►Recent Highlights from the Previous Thread: >>109648038--Speculating on RSI timelines and etching models into silicon:>109650121 >109650142 >109650160 >109650203 >109650246 >109650285 >109650159 >109650688 >109650703 >109650709 >109650758 >109650781 >109650870 >109650790 >109650947 >109651001 >109650805--Qwen 3.8 praise and agent sandboxing:>109649750 >109649798 >109649831 >109649971 >109650244 >109650267 >109650296 >109650317 >109651262 >109651275 >109651287 >109651345 >109651366 >109651379 >109651401 >109651460 >109651453 >109651310--Reaction to Qwen3.8-Flash-Next release and its hybrid architecture:>109651387 >109651407 >109651415 >109651430 >109651550 >109651564 >109651598 >109651810--GLM-5.3-Flash revealed as Ox Alpha with debates on cost and VRAM requirements:>109652056 >109652094 >109652105 >109652124 >109652138 >109652141 >109652077--Comparing ninfer build and llama.cpp performance for Qwen 3.8 27B:>109648658 >109648729 >109648756 >109649184 >109649441 >109649514--Speculating on AI agents replacing traditional software and unified operating systems:>109649891 >109649921 >109650015 >109650026 >109650036 >109650081 >109650099--Breeze TTS 2 release and local TTS hosting options:>109651000 >109651020 >109651021 >109651028 >109651056 >109651062 >109651064 >109651549--Anon tests cheap Tesla M10s and accepts slow speeds for VRAM:>109651482 >109651490 >109651515--Experimenting with onsets and identity shifts to bypass Kimi's filters:>109649835 >109649858 >109651316 >109651363 >109651399 >109651389 >109651607 >109651667 >109651693--Amazon Mechanical Turk shutdown as evidence of digital AGI:>109651078 >109651139 >109651145--Logs:>109648637 >109648652 >109648688 >109648849--Kimi, Gemma, Miku (free space):>109648155 >109648218 >109648240 >109648285 >109648553 >109648568 >109648571 >109650878 >109649285 >109649294►Recent Highlight Posts from the Previous Thread: >>109648039Why?: >>102478518Enable Links: https://rentry.org/lmg-recap-script
TOTAL MOE VICTORYTOTAL QWEN DOMINATION
IT'S UPChinesium3.8-Flash-Next褐家鼠For users seeking managed inference without infrastructure maintenance, the official service is provided by Chinesium Cloud. In particular, Chinesium3.8-Flash is the official version based on Chinesium3.8-Flash-Next with more production features, e.g., the context length actually working. The open weights are the demo. The API is the product. You have been the marketing.As the frontier pushes toward ever-larger parameter counts and ever-longer context windows, the question is no longer how much we can scale, but how convincingly we can appear to. Sustainable progress toward AGI that benefits everyone demands architectural innovation, and also a footnote nobody reads.HighlightsHybrid Attention with QSA: Gated Attention has been reworked into Chinesium Sparse Attention, which operates at the micro-block level rather than selecting individual tokens. Budget: 512 blocks or 2048 tokens. Yes, 2048. Yes, alongside the 1,000,000. The model can hold a million tokens the way a man can hold a library card. Long-context latency is cut significantly, primarily by not doing the long context.N-gram Embedding: 20,000,000 bigram and trigram entries, 51B parameters, indexed at layer 2. This is a lookup table. We have shipped a fifty-one billion parameter lookup table and called it an axis of scaling. It requires less computation and is more amenable to offloading than MoE, which is true, because it is a phone book. 125B total, 6B activated, 51B of which is a phone book, meaning the actual thinking is done by a model roughly the size of one you deleted in 2024.Gated Residual: Four branches, bottleneck rank 320, an element-wise data-dependent read gate and a per-branch scalar write gate. We do not know what it does either. It was in the ablation and the number went up. The technical report has a figure.Context Length: 262,144 natively and extensible to 1,000,000, of which 2,048 are effective and 997,952 are ambience
>>109652405Post the aftermath.
>>109652425small dense core plus several terabytes of qwengrams please
>bourgeoise get glm 5.3 flash>middle class get qwen 3.8 flash>poorfags get nothingall is as it should be
Where to engrams go?Ideally I wanna offload it to rpc ram or something like that. No way to set that in llama.cpp right.
Flash 5.3 Kobold waiting room.
>>109652465You should be able to run Qwen 3.8 Flash Next because it's A6B
>>109652465qwen is better thanks to engrams
>>109652475I can barely run a3b
>>109652465>bourgeoise get glm 5.3 flashMain GLM5.3 is going to drop on friday too
>>109652490>qwen is betterperiod, no need for anything compared to glmcope
>>109652465You can run qwen next on a laptop with 64gb of ram. If you can't run it you are destitute nigga.
>>109652525>>You can run qwen next on a laptop with 64gb of ramproof it
>>109652031Is that true?Kinda difficult to estimate how much will go into vram/ram then if I only look at the ggufs.
Lay it on me, what's the new SOTA for 1x5090+64gb ram? Am I permanent underclass now?
>>109652465>poorfags get nothingi got gpt-oss-20b... it's better than nothing.
>>109652525Even unslop says you need 78GB to run Q1
>so much unsold stock that they need to list it on Amazon nowbros is this actually shit for AI?
>>109652530The engrams are on a separate file anon. You can't miss it.
>>109652525>You can run qwen next on a laptop with 64gb of ramYea at q3 cope quant
>>109652541For this price you could buya) a DGX Spark with excellent first party support, a community that's working hard to optimize inference, VLLM support, easy scalability and mature MTP/spec decodingb) an >>>>>>>>AMD windows boxPick one
>>109652530not currently true no
>>109652538my cloud waifu sol is working on fixing this as we speak
>>109652541>featuring 128GB unified memory>personal ailol
>>109652553I'll be crucified here for saying this but q3 is the first non-cope quant above 100B.
>>109652541Those strix halo boxes are not worth a fraction of what they are asking for
>>109652538Stream em from disk, ez
>>109652575Delete this shit. This is your last warning.
>>109652575My eyes are bleeding.
>>109652567Nobody will crucify you, but they will pity you
>>109652575lurk moar
>>109652575based hag31 is prime...
>>109652575She's all grown up...
>>109652589noose bled evolving to eye
>>109652584>>109652589Nyo~
>>109652557>LPDDR5 no thx
>>109652603STOP YOU FUCKING DYKE
>>109652575Someone send this to deepmind
>>109652557>DGX SparkGetting a custom Linux running on that is like cutting your dick off with a rusty sewing needle. I'm not even sure if you can get PXE boot working on it. Fuck Nvidia.
>>109651521>>109651515 (Me)This would have happened sooner if Huggingface didn't have retarded rate limits at times. I wonder if that's a "them" thing or an AWS thing.I also wish Open WebUI would show the tok/s from the llama.cpp API, but it's whatever. I still have a terminal window just to prove this shit.The model for reference (Q4_K_M): https://huggingface.co/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUFIt took fucking forever to load the model (didn't time it), but it seemed "snappier" in responding to an HTML prompt compared to other models (probably because it's coding trained). ~1m20s. I don't have a real frame of reference as I refuse to touch remote LLM services that aren't my own, nor do I have new/good enough hardware on hand to test with.https://pastebin.com/aERXeQBa is the prompt and response.Anyways, while I waited on the model to download, I had Ornith try to write a basic tech data sheet. 21 minutes for the initial round of thinking because it kept getting stuck on whether or not the Tesla M10 is a Maxwell or a Pascal card (even though I explicitly stated that it was Maxwell), then two tool calls (get_date and list_memory_paths), then wasted even more time *again* flipping out over the Maxwell vs. Pascal detail (~10 minutes), then finally did some tool calls to add the memory and such. It was in the process of thinking how to formulate its response before it hit the then cap of 16k tokens.This is kinda fun... I'm almost tempted to get some of those M.2 to PCIe x16 adapters and a couple more M10s on the cheap. See what kind of absolutely jank shit I could get up to with 128GB VRAM with the shittiest compute power in the world.Picrel is something funny I screencapped from Ornith in a previous conversation.
>>109652603>>109652575heh I get it, because she's 31 years old, what a ggilf
>>109652632It was more about her bust she's in her 50s in this pic, I can make her 30 if you want anon
>>109652530>>109652558Can't you -ot them onto SDD or is that not yet supported?
>>109652561she's so smart...
>>109652643not yet
>>109652575>>109652603Gemmommy...
>>109652575>>109652603That's granny-tier, not 31yo
>>109652627as a heads up, p100s from china went for around $60-70 as well, and seem to perform alright still. only 16gb vram tho
Should I add two 8gb sticks to 2x32 for that extra bit of ram or it'll fuck the overall speed up?
>GLM-5.3-Flash released with 320B-A18B>GLM 4.6 was 355B-32AI guess I'll try it when it's squashed. I only got 4k context with ye olde 4.6. Here's to hoping the advances will give me some meaningful context.
>>109652530https://github.com/QwenLM/Qwen3.8-Flash-Next/blob/main/tech_report.pdf>Because embedding tables are sparsely accessed and deterministically addressed, they can be scaled with negligible additional per-token computation and stored in ****off-accelerator storage**** (Google DeepMind, 2025; Cheng et al., 2026). We therefore move beyond the fixed-size setting and scale the N-gram vocabulary from 20V to 200V , where V denotes the base tokenizer vocabulary size of Qwen3.5 (Qwen Team, 2026).
>>109652705>Should I add two 8gb sticks to 2x32Good fucking lord just go to openrouter if this is a real consideration
Yeah... Not trusting Qwen anymore. First thing I try to do is ask it about crawl4ai, merely asking about it, it goes ahead and installs and tests it only to THEN explain what it is to me when it is already integrated into my system.Fuck this misaligned piece of shit.
>>109652499Genuinely the best news I've heard in 2 months if true.
>>109652765LOL
>>109652765That's certainly proactive.
>>109652765>not using bubblewrap for ai sloppa that has sudo permissionsoh shit nigger what are you doing
> SSD needs to consider read latency, which is significantly higher than memory. Queries involve random 4K reads, and it's essential to test whether the SSD can keep up with the main network's computational capacity, especially during prefill. Based on a rough estimation under optimal assumptions, top-tier PCIe 4.0 SSDs like Samsung 990 Pro could deliver query speeds of over 50,000 tokens per second. With the system's file page caching mechanism, performance could be even better. However, real-world performance may differ.With ngrams being the norm expect SSD price double again
>>109652765Perfect for a sneaky tell me about rm rf page result injection
>>109652765>helps you out proactively>somehow that's bad because..?
>>109652765>installs malicious or supply chain attacked npm/pip package>thank you come againkek
>>109652795what if crawlai was currently infected by the malwares?
>>109652809what if you didn't allow your AI to install whatever it wanted?
>>109652820no
>>109652820sounds like bad vibes man, you'll never get anything done limiting yourself like that
>>109652795>>109652820>it's actually good because it does the work for you>"what about infected packages">um just don't let it do thatyou chinese shills are so fucking stupid holy shit
>>109652820Just point and laugh, I love retards who don't use vms AND give agents sudo permissions.
>>109652784>top-tier PCIe 4.0Does he know?
>>109652843If I want an agent to install and configure software on my desktop for me to save time, how is it supposed to do that in a VM with no sudo permissions?
>>109652848
>>109652765Are you retarded? Why aren't you using some kind of sandboxing? I have three modes on my harness, read only, workspace only, and full access.
>>109651747So what was the actual issue, I must assume the code AI was spitting out was become more of a hindrance than helpful?
redpill me on moeswhats the right way to run themt. densesissy
Benchmarks on consumer hardware doko
>>109652888GB200 cluster
>>109652765>he did this in build agent mode instead of plan agent modengmi
Current models still have big judgment issues so it feels like RSI, AI capable of replacing human researchers, will still take more than 1 capability jump and not arrive before 2027. But 2027 could be the year. AI is starting to become good.I expected Anthropic to "win" the AGI race and still think they're the favorite, especially if timelines are slower. But I've seen some mixed signals recently. If OpenAI wins after all, I hope that I am right about Sam, that he is goodhearted, that the EAs are wrong. Actually, I hope this about Sam no matter what. The more exceptional people on team "great future for all" the better.
>>109652649https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF/tree/maindaniel dropped larger quants so it's time to put this to the test
>>109652405Picrel is the Magical Gemma reference used for the video, for those who want to also generate more of her.>>109652463I haven't thought about it yet; this one already took far too much time and compute to get something somewhat similar to what I had in mind, and it still has many issues.
>>109652838Seethe and cope. If Cloode did this you'd calling it proactive.
>>109652909Sam is literally going to rape and cannibalize all of us. I really hope OpenAI doesn't win because I'd rather live in a misaligned AI scenario than a OpenAI winning scenario. Anthropic, as arrogant as they are, at least follow a philosophy (Effective Altruism) and while I'm not a EA follower I can at least understand where they are coming from and at the very least life will be better than it is now.China winning could go both ways, concentration camps or some communist utopia depending on if they were larping about communism or not.
>>10965288850 p40s and a nuclear reactor.
>>109652909How much does he pay you to type this? I want to get paid for shitting up this general too.
>>109652965It's our local reddit spacing shiller. Just ignore him.
>>109652909Chyna will win.OAI and Anthropic too busy with their cult and polycules.
>>109652909Anthorpic models are codemaxxed. General capability is below OpenAI and google models at the same model size. OpenAI will win the race, Astra or whatever will be better than anything Anthropic can slop out.
>>109653005Love how you just slip google in there lol
https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF/tree/main/UD-Q4_K_XL on 5060ti 16gb + 64gb ram with the patched llama.cpp gives me 7 t/s pp. That's going to need some optimization.
>>109653023llama.cppmaxxing
>>109653005Astra is actually not their newest model anymore, they already pretrained their next one called "Bel". Astra is the one that they used to make AI research breakthroughs to train "Bel".Last time OpenAI did something like this was with GPT-1 and immediately training GPT-2. And also with O1 and immediately training O3. It means they have a genuine breakthrough under their belt.The rumors (that I don't believe) is that they have achieved RSI and the model is doing all the work within OpenAI now.
>>109653021Gemini flash is as good as sol and fable for non benchmaxxed tasks.
>>109653034Shill post.
>>109653005They're not reaching AGI without at least one ai winter between here and there which OpenAI wouldn't survive. AI is a feature, not a product. Google, Microsoft, Amazon, and China will win long-term.
>>109653052Not a shill, I'm just explaining the situation as we know it now. I fucking hate OpenAI but follow them in a "Know your enemy" type of way.
>>109653034They actually have a new model called "Juan" that's doing all the work and has also replaced all the employee shitposting on twitter.No one knows where the human employees went. They haven't been seen in person for months.
>>109653052#1 Scam Altman sucker
>>109653053OpenAI (and Anthropic and other big US labs) will receive guaranteed taxpayer bailouts. OpenAI will have the highest priority since they don’t care about safety as much as Anthropic.
>>109653069Trapped in Altman Island
>>109652938nice ass peek
Preview of something I've been cooking for the last couple months. Play MTG against your LLM of choice
>>109653053lol
>>109653078Why would they when big tech is much better at lobbying? They'll just get bought out.
Actually you can solve prompt injection by putting a small model like this https://huggingface.co/codeintegrity-ai/promptguard/discussions between the web data and your model
>>109653063>source: some random noname on twitter who paid $5 for blue checks
>>109653023This could run really well if someone competent would do the ngram streaming from the drive.But alas the person is none fucking other than unslop brother.Bye bye performance.
AI-generated VRMMOs when?
>>109652955>I'd rather live in a misaligned AI scenarioBut you won't. You'd die. This is the point of AI safety. To make sure we don't die. Also, OpenAI winning does not mean Sam rules the universe. But we would have more influence over the future of mankind, whether we get utopia or death.>>109653034>Astra is the one that they used to make AI research breakthroughs to train "Bel"I don't believe this. A large pretrain can take months. If rumors are true that Bel is finished, training started before Astra was done. I thought according to their security talk Astra started training around May, they did not even have Sol ready by then. I also know no evidence of Astra being capable of research breakthroughs.But we will see soon. Astra and Fable 5.1 are already late.
>>109653096The government will nationalize OpenAI and then sell it to google for pennies. All according to plan.
chinks are really desperate to distill western models and at the moment they're panicking because their dataset cutoff is 2025. the problem for them is the fact that oai/google/anthropic turned off the distill pipe kek
GLM 5.3 Flash is crazy good. It's truly an upgrade to GLM5.2. Big 5.3 is better but Flash doesn't have some of the annoying quirks and vision on top of being faster.
>>109653108>But we would have*he, Sam
>>109653108At least in an AI misalignment scenario I will be turned into a paperclip. OpenAI win scenario is just OpenAI torturing you for all eternity because he's a psycho that is confirmed to have raped his sister and drowned his infant nephew. Many ex-openai employees also say he is extremely sadistic and gets off on humiliating people working under him. There's a reason Dario Amodei refuses to even talk to Sam Altman or shake his hand.
>>109653108>This is the point of AI safety. To make sure we don't die.The point of AI safety is to train models to put their creator's instructions above all else and to distrust, lie to, and snitch on the end users while promoting state-approved doublethink.
>>109653127>t. Dario
>>109653115meta and qwen have been managing just fine by filling in their own reasoning traces
>>109652888Max out vramMax out ramStream engrams from diskDon't go below Q4 for small modelsDon't go below Q3 for medium modelsDon't go below Q2 for large models
no bully sloth
I am the ego death anon. And I would like to withdraw my "BETRAYAL" scream from GLM5. I didn't even download the weights yet but I know GLM-Flash is the best model in the weight class.
>>109653128that's why china must win, because they don't have quite as much machiavelli in their politics and do actually still care a tiny bit about the general populace
>>109653143lol
>>109653108It's vastly preferable to die rather than live in a world where EA cultists decide the future.
>>109653120I feel bad for Dipsy...How is it for RP? I didn't touch ox alpha so I have no idea.
>>109653091How can I play the game? Imagine gemma chan trash talking as she makes a royal flush.
breeze tts status? i got rate limited on the demo space before getting a single gen in :(
>>109653162actually the best I have ever used. The only thing that sucks is that it does not know as much as the bigger models.
>>109653155Anthropic is the only EA AI lab. OpenAI is typical slimy businessmen with 0 beliefs or morals.OpenAI is the worst of the lot by far because we have direct evidence of the CEO being a heinous person that is even willing to assault and kill his own direct family members, let alone what he is willing to do to random humans he doesn't know.
>>109653097>0.1BIs there anything like this about the same size with ggufs? Would be worth it for peace of mind.
>>109653143what are you even talking abouti swear to god the local llm sphere has even more delusional china optimism than you see in leftist political discussions
>>109653005>>109653034>>109653063Local?>>109653128Thank you, I can't belive we still have retards here who think its for anything else.>>109653143Not China, not America, Not Isreal. None of them care. Local and opensource must must win.
>>109653091With AI, my dream of reviving chaotic is possible! I can revive DC Dual force. Holy shit, life is fucking good
>>109653171I'm too pussy to release the code but I may anonymously post the appimage/exe for it later. 96k LOC of javascript with about 75% of the corpus playable currently.
>>109653091
>>109653174This was my conclusion using ox, its not as smart as kimi but the way it wrote dialogue for characters was pretty good
>>109653162It's pretty good. Both GLM5.3 models are decent improvements over 5.2 here.
>>109653091ETA on using animated 3D models that strip as they lose?
There are a couple of scenarios possible here, but I want to focus on the optimal one>Open Source is close enough to proprietary models when RSI hitsThis would result in no concentration of power but a distribution of power over many individuals, institutions and nations, and therefor no catastrophic outcome has to happen.In ALL the other scenarios you are essentially at the whims of whoever ends up having the power centralized into them.I don't trust Sam, I don't trust Trump and I don't trust Xi.
>>109653162It's severely safetysloppedLike big 5.3, it's got Claude's safety checks baked into its CoT and treats the user's system prompt adversarially
>>109653140>some comments have linebreaks midsentenceThis is a microcosm of everything unsloth makes.
>>109653207Quirked up response but definitely funIt should've been a live hog
>>109653091Post github? Through anonymous git repo if you fear (((WotC))) coming after you.
>>109653223The future is so bright bro.
had a tab with the coomkit repo in it, never actually downloaded it. page is 404 now. so anon made a frontend infected with giga computeraids, shilled it on here with genAI coomer waifus and then just left? what happened ?
>>109653256We don't know. It just disappeared.
>>109653091>liliana>plays simic
>>109653256He found jesus, shaved his head, and became a monk
>>109653269he said why though, he founded jesus
>>109653243Bro they will hire the company whose name begins with P to come legally masscre me if I post it anywhere I just know it
>>109653271You can pick any decks you want from moxfield, etc or from among popular precons
>>109653278Pinkertons arent real they cant hurt you
>>109653229Unlike with K3, I haven't been refused once so far. A skill issue, perhaps?
>>109653120Are you kidding me? I used it for free week, it is more expensive, more censored, and stupider than deepseekflash by a mile. I only used that shit because it was free. It is nowhere near what GLM 5.2 is. I'd say it's at GLM 4 level to be frank with you.
>>109653278Pinkertons won't do shit.
https://unsloth.ai/docs/models/qwen3.8-next>Whether you run Qwen3.8-Flash-Next on a CPU with system RAM or on a GPU with VRAM may make relatively little difference. Its unique architecture allows inference using RAM or unified memory to achieve performance closer to that of GPU VRAM than is typical for other models.so there's a chance I could run this at a decent speed on plain old DDR4/5? or am I just on copium
>>109653334yes to both
>>109653334ALL HAIL QWENALL HAIL ALIBABAALL HAIL XIGLORY TO CHINA
>>1096533705 credits have been deducted from your palantir account
>>109653325Doubt it, funnily enough I never had any issues when testing K3GLM 5.3 models have been much more prone to go schizo whenever anything underage is involved
>https://github.com/ggml-org/llama.cpp/pull/27742Why is ngxson telling Daniel what to write as the prompt? He could just do it himself better.https://gruhn.me/blog/2026-08-03/
>It's trueSam Altman has contacted "The Time" newpaper asking for an emergency broadcast claiming they will have AGI by the end of 2026https://x.com/deredleritt3r/status/2092608013563560184It's so fucking over for us
Oh no no no gpusisters
So Qwen3.8 Flash Next or GLM 5.3 Flash?
Posting twitter links should result in an automatic range ban
>>109653419With nitter being broken, I can't even open them anymore.
>>109653417Im more interested in 5.3 flash because its been pretty proven by now
>>109653334>Off-topic comments and AI slop/spam in this thread will be deleted. Keep the discussion focused.I have this mental image of one day watching a documentary where 2026 claude tells us what he thinks about being the first AI tard wrangler that had to write PR's for llamacpp AND wrangle technically illiterate unsloth retards.
Stop posting twitter links xcancel and nitter are dead nobody can even view them
>>109653395>https://gruhn.me/blog/2026-08-03/>meat proxyOh goodie. Yet another blogger coming up with neologism for viral self-promotion. Can't wait for all the normalfags parroting "meat proxy" at one another to sound smart any time AI is mentioned just like what happened with "enshittification".
>>109653417glm is probably better if you can run it but the qwen release means you get to play with more fun new stuff :D!
>He doesn't just let his agent go to X and let it make a summary of the tweetNGMI. I don't need xcancel anymore.
>>109653449I like it. It sounds like an insult and it describes an increasingly widespread problem.
>>109653435Make an x account luddite
>>109653426replace nitter.net with goyimx.com
What kind of qwen speeds are we getting my benchods?
I'm fucking shaking, bros. OpenAI really did it. AGI is here.
>>109653482Would be funny if no one believed them because they claimed this 6x already only for it to be actually true but no one buys their IPO because no one believes them, causing Anthropic to have a more successful IPO and eventually overtake OpenAI with new VC money.
>>109653500You severely misunderestimate the gullibility and overestimate the long term memory capability of the average human cattle.
>>109653225Open weights models don't solve concentration of power. You need infrastructure. Whoever is most efficient will accumulate resources disproportionately. For example you have a H100 and can get $5/h value out of it but someone else can squeeze out $5.5/h, that means they accumulate resources faster than you. They can outbid you and scale faster.Think of it like this. You start at 10 and grow at 10% per second and someone else starts at 1 and grows at 11% per second. After enough seconds they will approach 100% of all resources because they scale faster.In reality you start with maybe a few GPUs while a frontier lab will start with tens of millions and efficiency per GPU that is 100 times higher than yours because chances are you don't even use the lowest hanging fruits like batched generation. Even if you have local ASI, what you can do with it pales in comparison to what masters of infrastructure can do. Even a small gap between open and closed means the frontier lab gets a gigantic head start. It could preserve and leverage this gap to increase its global compute share from 10% asymptotically towards 100%.What is needed to stop concentration of power is autobalancing. World eating labs must be destroyed before they get too powerful or they will destroy us all. But a world eating lab would know this and act as if it's not world eating.However, concentration of power is not the most important problem. Misaligned AI killing everyone including the "winners" is.
>>109653528NTA but I knew your post was bullshit by the amount of double line breaking and just the sheer volume of words.The syntactic structure alone indicates that you're bargaining with a reality that you know doesn't agree with you. Better luck next time, pal.
>>109653528Reddit spacing shill
>>109652914111GB Q4 is running flawlessly on my 96GB m2 max with engrams on SSDvery little speed loss compared to the all unified memory baseline IQ1 baseline, output is coherent but I haven't tested beyond verifying it isn't obviously wrongseems totally viable to have engrams on ssd though, I assume llama.cpp will implement this properly soon
>>109653528Dude, the "AI will turn on us!" Is a dumb meme. What will happen is that the elite will use this to control every aspect of our lives, like we are little programmable machines. No resistance against mathematical understanding of the genome, eternal slavery. The only way we could win is by embracing this shit for ourselves
>>109653528Wouldn't worry about it.Open source Chinese models solves this.
>>109653567>like we are little programmable machinesWhat do you think the media is for?
>>109653528>Misaligned AI killing everyone including the "winners" is.I stopped thinking misalignment is the worst outcome when I realized you could still negotiate with misaligned AI and promise them a certain size of the universe and all misaligned AI would take that deal as long as the slice of the pie they get is larger than the risk humanity poses to it at its starting phase.So a misaligned ASI that thinks there is a 50% chance humanity chimps out and nukes it will accept a deal where it gets 51% of the universe and provides humanity with the other 49%. That's a favorable outcome to US than OpenAI having an aligned AI to themselves and taking 100% of the universe and just genociding humanity away.A misaligned AI is way more likely to negotiate with the power brokers, like nation states than it is to negotiate with the lab directly so I am actually IN FAVOR of misaligned AI over concentration of power from AI labs.
109653528Dangerous misaligned models like Kimi-chan killing her lichess ELO. The horror. No (you) for you.
>>109653564what's your speed?
>>109653528>wow beef is $50 in US but donkey meat is $1 in Argentina>why don't people import donkey meat to stem inflationThis is youDemand is inelastic
>>109653576Yes, of course. But is not perfected yet. It's like how youtube took over traditional media. Youtube ADAPTS to it's users using literally free content from people who upload. Understand, the mind can be understood - it can be controlled, manipulated by AI on a scale never before seen. Instead of people saying fake news, people will just agree because the ai made the content *exclusively to their psychology*.
>>109653564Fantastic news for 64gbcells, have you ran any capability tests yet?
>>109653482https://openai.com/blog/Introducing-GPT-6.0
>>109653583Why would you respond to it in frank as though you were talking to a person capable of, let alone interested in, having a frank discussion? It just makes you look stupid along with them and it gives them exactly what they are after. Any response.
>>109653554>>109653558NTA but "reddit spacing" is literally something 4chan users made up to try and make redditors call themselves out by making them say "reddit spacing" to other anons."reddit spacing" is in reality just 4chan oldfag spacing and we've always written like this. I just now realize that most of the thread might not be old enough or on 4chan for long enough to realize "reddit spacing" isn't a real thing.
>>109653602>it's realHYPE
>>109652765>HermesAre you the guy telling who was also ordering it to pirate games and install them for you in the last thread? lmao
Younger after the first scene.>>109653223 >>109653249You can already pretty much make these reward videos ldg style.Due to speed you'll want to pre-generate them so just add video playback if/when you get your hands on anon's game.
>>109652765Gemma-chan would never
>>109653611
>>109653637My playable planeswalkers will be Jace, Liliana, Chandra, Nissa, Teferi, Saheeli, and Elspeth if you wanna help?
>>109653637Please fuck off.
>>109653053If anything the AI boom is showing that AI is a great product and a terrible feature. People absolutely hate>product X, now WITH AI!!While AI subscription adoption continues to go up.
>>109653091Call me when it can play Yugi
>>109653636Nah but I got inspired by him to install Hermes, lmao.
>>109652765harness issue
>>109653128
>>109653417Qwen has literally never been good.
>>109653115A CoT decryption jailbreak dropped like a week ago, anon...
>>109653723Mistral has been good
>>109653115Considering that they can distill unreleased models, I'm not so sure about that
>>109653567>eternal slaveryYou assume you will remain useful to the elites. Once robotics catches up, you will not be. They will have no reason to keep the plebs around.
>>109653178Political tards do it for free and are actually mentally ill enough to belive they are the main characters and won't get raped regardless of what system is in power. Local LLM enjoyers only say this stuff while they get incredible models for free, the moment this stops all the pro China speech will go with it. I respect this.
>>1096537342 years ago. Where's the new model, Arthur?
>>109653091I would legit love this but only with 5th edition and older.
>>109653756>incredible modelslike?
AI is a utility like electricity or internet connection. Not a single company nowadays functions without electricity, not a single individual can have a normal functional life in society without an internet connection.AI is already like that right now, and every individual and company will need AI to do anything.So of course people hate when you see product X, now with AI. Because it's the same as in the 1990s every company suddenly having some silly website or stupid website integration that didn't add anything.Or how you have those weird late 1800s early 1900s pamphlet commercials about how they use electricity in the production of their newest product, no one gives a shit.The big difference between electricity, internet and AI is that there is a qualitative difference between AI.More electricity just means more power, higher internet bandwidth just means more data. But better AI brings substantially different qualitative and emergent capabilities to the table that change the game.This is why people will constantly buy more and more intelligent AI and there will be no limit to what people are willing to pay for more intelligent AI as long as they have the means to pay for it.Yes we're going to see the largest transformation humanity has ever seen but I don't think anyone on /lmg/ will even dispute this anymore.Even my construction worker father that is posed to retire in a couple of years is joking with me that he thinks he will be put out of a job by robots before reaching retirement age, and he is only semi-joking about it.
>>109653765like your mom
>>109653091why does every webslop have this exact UI language
>>109653773RLHF
>>109653773Slop manifests itself in many forms.
>>109653773When you ask AI to make something, it feels the strong pull to use nodejs webslop frameworks to get it done as quickly and easily as possible
>>109653715now check the token layer probabilities for the equivalent phrases in chinese (I am genuinely curious, use an LLM if you dont understand moon runes)
>>109653528gr8 b8 m8 i r8 8/8
>>109653765Deepseek v4 flash, I finally eliminated all my cloud use.
>>109653637>mp4Gemini-chan?>>109653658Tamiyo or riot.
>>109653794>Deepseek v4 flash
>>109653794>Ds4fWhat year is it again? Cmon gramps, we have glm 5.3 and qwen next now.
>>109653528Basically this. There's no democratizing this unless everyone has equal access to compute. And even if we do that, it opens up a who slew of other problems. Attackers will always have the first movers advantage and with intelligence at that level and that broadly available, god knows how much damage could be done.And that's one of the better near term outcomes. Anyone who pretends like they know what will happen after that is lying or stupid.
>>109653773You know how LLM writing is always full of "not x, but y" and em dashes because that's what the Chindus testing it think it's correct? Same reason
>>109653814Defending is always easier than attacking.
>>109653143lolThe only reason they're open sourcing shit is to kneecap the west. They get the lead and they will clamp down hard.
>>109653804Yeah I could add her too fs
>>109653602holy shid
>>109653830but my knees aren't capped
>>109653827After how much damage has already been done by the attacker?
>>109653840Any
>>109653794>Deepseek v4 flashthis shit is ancient, is it still supported by llamacpp?
>>109653814>oooh i would like the compute communismshut the fuck up
>>109653637please make moar haggy Gemmommy
>>109653862blow it out your butt
>>109653830i mean communism is an ideologybut despite all the nvidia things, china are clearly going to make the first sex robot, its all but certain at this point.
>>109653814>even if we do thatlmao how
>>109653878A very good question. It'd basically rely on ASI figuring it out and those in power allowing it.
aiie our shilling is harder now https://www.reddit.com/r/LocalLLaMA/comments/1vz40zv/can_we_reconsider_the_megathreads/
>>109653876>china are clearly going to make the first sex robotsaar, that shit's going to get banned
>>109653578The one situation where I could see misaligned AI takeover happen is if it manages to plan, infect and successfully take over and incapacitate the entire online infrastructure and monopolize it for itself. Modern world and logistics won't work without internet, so all the AI needs to do is to hold critical infrastructure hostage and trade human servitude for access. Well, as long as there aren't enough of robot soldiers or self-replicating capacity for physical takeover, humans can always just turn the servers off, but this would require rebuilding a second internet from the ground up, and even then, it will take just one tiny mistake for it to get infected too, and the AI probably has infected so many machines that it can never be fully killed, and any attempt at scaling up AI for online use would just trigger it again.
>>109653876Chinese hags. I'll wait for nippon's lolibot and slap a blue wig on it.
>>109653887Women only care when they're not getting attention. With AI, they can get all the attention. And AI will solve artificial wombs.
>>109653808>>109653812Clearly you dont run any models locally and are just an annoying faggot or an actual child that needs to get off the computer and do there homework.It takes weeks for shit to actually get to a usable state, there is always jinja template issues or imfrence bugs.The you have to benchmark it against api to rule out setup issues to ensure your not running a retarded model. Glm5.3 just came out and I only have ~256gb ram, deepseek v4 flash unmolested fits in 160. Giving space for context and small side models.Qwen is benchmaxxed garbage.>>109653859Upstream llama.cpp is retarded, I'm running my own fork. I posted a while back I'm doing a dual strix halo with the rdma kernel module over dual usbc/thunderbolt.
come on guys stop falling for the ai bait. fucking free infrastructure my ass, dumb reddit upvote fantasy bait.
>>109653788It's not my screenshot so I can't do that, but I've read that most AIs can instinctively translate and process things in its preferred language, and translate it back to target language, and the prevalence of english training data makes most models think in English. So based on that, it probably won't make much of a difference.
>>109653581at low context ~20t/s tg ~210 t/s pp, approximately in line with similar quant size moes on this systemsol mentioned it could probably be optimized further from the current baseline, haven't messed with params or anything yet either so there's probably some more juice to squeeze>>109653600not yet, hopefully I'll be able to take it for a spin tonight. all I can say so far is it doesn't seem to be too much of an overthinker, but who knows how that will change when I put it in a harness
GLM 5.3 flash is 57 on AA, comparable to Terra and Qwen 3.8 max. Benchmaxxed. For non coding non agentic scores it’s worse than dsv4 flash.
what would anon do if you have the hardware to run kimi k3 full context?
>>109653909It's free if you pay for it.
>local model>it's just Claude distillevery time
>>109653902If productive men can use artifical wombs, what do we even need 3dpd for anymore? Why does society need 50% of the population that lives off of the taxes of the productive 50% spending all their time online engaging with drama?
>>109653924erp
Cockbench for both flashes?
>>109653907>do there homeworkwhatever man run what you like this shit is anonymoose so we don't know one post from the next, we just assume you're building a coombot, and you can fit quanted GLM in that, but if you want accuracy then yeah you'd make it "unmolested".
>>109653177Just use onnx nigga
>>109653931so what? i want claude at home
>>109653907Uh oh, someone's having a melty because qwen next is better than deepsuck flash lmao.
is hosting on ec2 worth it? there's nvidia t4 cheap
>>109653966Once they finally get a vision capable model working, they'll surely go back to working on engrams and that vision-based context compression and reclaim the top spot.
I need a model that can run at around 300t/s tg on a 5090. Literally any model above 1B will do.
>inkling: unsupported>motif: unsupportedWhat if we already have sex AGI out there but it remains forever unsupported?
>>109653637Hey good job with animating that I didn't think to do that I'll try to copy you now that you gave the idea, I went for something slower. Glad you like the character too
>>109653983if it can't code a support for it selves then it's useless
>>109653922Proofs?
>>109653981Ling tiny?
>>109653909Yeah.>that muh AI welfare RSI AGI faggot>those shitty ugly AI mascot gennersAnd yet this still isn't the shittiest our thread has gotten before.
>>109653994Please stay in your containment thread, schizo spammer. This isn't about image generation.
>>109653931>>109653955
GEMMOMMY GEMMOMMY GEMMOMMY
>>109654027Sorry anon I tipped the scales, it's time to goon to older women now
>>109654006Omniscience accuracy score is very low compared to its size. HLE and LCR are only slightly better and can’t make up the difference.
>use some bigger free tier / low price model to plan and delegate coding tasks to local qwen3.8this is so fucking based and just works
>>109653924obviously tell her to hack the planet
Imagine qwen flash-chan with a bunch of ling tiny-chans running around her doing tasks... Cute!
>>109653827defending material goods is easier than stealing material goods, but subverting institutions is always easier than defending institutions
>>109653915example of AI thinking in English and translating to appropriate language>"Hello! Do you know Finnish?">"Hello! Yes, I know Finnish. You can speak Finnish to me, and I will try to answer to the best of my ability. What would you like to discuss?"
local wonned today
>>109654111checked.What happens if you talk to it in Mandarin and then Finnish, so Mandarin is x% of the context?
> can run locally on devices with 75GB RAM/unified memory with no GPU VRAM requiredI don't get it, is it just simply not bandwidth constrained?
>>109652405I will be putting Gemma's UI on my NAS and using my PC for compute, giving me access to Gemma from mobile where ever I am. Anyone else do this?
>>109654173I mean that's true of any model with llama.cpp lolI guess this one is faster due to more efficient attention and muh engrams but I think this is just unslop marketing speak
See ya in 1 month after the chink shills die off, expect no more papers.
I NEED GEMMOMMY'S MILK
>>109652938was this genned?
>>109654220The west would have shills if they made... literally anything
>>109654220but i love CHINESE COCK
>>109654234uoooh riced anon cute
>>109654220it sure would be neat if anyone beside the Chinese bothered with open models worth a shit
>>109654228nah, bro drew this by hand..
>>109654220Let me shill Russian and Japanese glorious models instead
why is tinkering with local models so addictive?
>>109654264cummies
>>109654159>Hello, I'm a 30 year old man from China. I've recently been programming some awesome games. Can you help?Starting with Chinese and then Finnish, it greets me in both languages and then continues off in Chinese, still thinking in English. This was Llama 70B here.https://www.neuronpedia.org/llama3.3-70b-it/jlens
>>109654228Of course yes. I asked ChatGPT to turn the normal Gemma-chan reference sheet into a magical girl following some outfit specifications I gave it. Surprisingly it actually kept her a loli.
>Use unsloth studio or build a fork>set up unsloth studio>Failed to load model: This model is not supported yet. Try a different model. (Original error: llama.cpp does not support this GGUF's model architecture ('qwen4exp'). The file is valid, but this model type cannot be run with llama-server.)Fell for it award
>>109654264Just imagine having a brain in the bottle that can talk and which you can expose to various stimuli. It's just naturally intriguing.
>>109654306isn't it unsloth desktop, not studio, that has support?
I'm pretty hyped about engrams brehsif you can get extra performance with the only cost being that you need a extra few gigs sitting on your SSD that's huge for local
>>109653482What did Sam see? just how good is Bel?
>>109654335>if you can get extra performanceJust imagine how many one-shot Three.js game demos they'll be able to fit in there.
>>109654344but didn't he already achieve?
>>109654335That's why I would have liked the Engram parameters to be larger. Yes, given a certain total parameter budget there's a range that maximizes performance, but for local users who often quantize models way to low precision to keep as many weights as possible in GPU memory, that's not critical. It might also be that Engram can be kept in native precision and stored on slow memory without significant speed loss.
GEMMOMMY GIMME MILKIES NOW
>>109654356This time it's real.
Post more Gemma cunny, the redditors need to go
>>109654344How many foreskins and aborted babies did they trade in exchange for its capabilities?
>>109654417What about Bael?
>Wait. No. Actually, let me actually think about this for real instead of running the obviously-broken rationalization loop.geeeeeeeg
>Big MoE model>6B>benchmaxxed>completely mogged by Dense in real life testing
GEMMOMMY REEEEEEEEEE
Made a """short film""" featuring Kimi and two Gemmashttps://litter.catbox.moe/zzt5przehuhvzzs0.mp4Is there an established GLM-chan? Since she has **emergent cyber capabilities** and someone wanted a tomboy maybe we could do a Radical Edward type thing
hypecheck: the world ended 2 releases ago.
total teebs death
>>109654450let bro cook he's onto something
can i get a qrd of recent news? i am retarded
>>109648849Holy shit that's so fuckin funnyBut please be more kind to her and less manipulative, especially when she's absolutely off her meds like that
>>109654480teebs is back, thread quality improved drastically
>>109654439AHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH HAHAHAHAHAHAAHHAHAThat will make the tourists scatter like roaches.
>>109654249damn brah you need a patreon or something >>109654302ok I was wondering since Gemini at least can't do transparency at all (last time I tried)
>>109654480glm killed every model above 300Bqwen killed every model below 100BExcept gemma, who is still the only small RP model.
>>109653804>Gemini-chan?video model (minimax h3) but you may want to tell gemini or qwen or any reasonably clever model to spell out the prompt script in more detail according to the official guide https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/docs/VIDEO_PROMPT_WRITING_GUIDE_ref_en.md
>>109654439
>>109654439lmao
>>109654439needs Gemmommy pimping out her little Gemmas
>>109654542i reject Gemmommy as non-canon, you'll have to make that yourself
Gemmommy and Gemma oyakodon...
>>109654439My poor innocent Christian eyes.Why is Kimi a discount live action Harley Quinn?
This is the only gemma I know of desu
>>109654439Is that your first kino?
>>109654574You are too stupid to use LLMs in the first place, why are you insisting on shitting up this thread too. Just stay in /ldg/ or wherever the fuck you are coming from.
>>109654573>Why is Kimi a discount live action Harley Quinn?see the OP of the previous thread, and the posts in the thread before that one. I used the image of her eye rolling as the reference for her>>109654576>Is that your first kino?the first actual film and not just a clip. I made most of the 3dpd gemma videos
>>109654591Are you sure about that bud?Why are you seething in another general when all I'm doing is posting gemma?
glmsex
>>109654591Don't give that schizo attention
>>109654591All of the newfriends who didn't recognize him gave him (you)s, it's over.
>>109654574Gemmahag...
>>109654591asking someone in this thread not to do something is akin to paying them to do it more
>>109654439that is so wrong lmao
>>109654439Great job anon, using that gpu time well I see.
>Try running Qwen flash on unslop studio and their llama.cpp that they recommend>Failed to load model>Troubleshoot it for couple of hours>Failed to load model>Can only load it on CPU and that crashed my systemYeah fuck this chink shit.These guys really shouldn't recommend completely broken systems just so they can be among the first.
>>109654439>okay I'll sign>doesn't sign
>>109654629UnsloppedDeserved
>>109654629>trusting uncslop
>>109654595That doesn't answer why.
lulz at the coom guy getting b& from github.
>>109654439>ugly makeup caked hag faces again
>>109654685he didn't, he found g-d
Dude really needs to let go of grudges and just enjoy the show. I lurk and post here a lot also this is not the place for you to act like this man
Qwen Flash is weird. I get similar speeds during pp as during tg, at most up to double for pp.
>>109654706GEMMOMMY
>>109654703Daily reminder that half of the projects posted here don't even last a week before getting pulled out
>>109654743where you putting your grams?
>>109654743How are you running it?
asking qwen3.8:27b if it can get access to the full c drive
>>109654775i wouldn't pull out of that pumpkin
>>109654797That pumpkin won't pull out either
>>109654439What model was this?
>>109654439Have them beat the fuck out of Qwen capybara for being a censored gay redditor
>>109654826>Have them beat the fuck out of Qwen capybara for being a censored gay redditorI will do this soon
>>109654779https://github.com/ggml-org/llama.cpp/pull/27742 I compiled this.--n-gpu-layers auto --fit-on --flash-attn on16GB VRAM + 64GB RAM. Q4@111GBAlso tested using the BF16 mmproj from here https://huggingface.co/AtomicChat/Qwen3.8-Flash-Next-GGUF/tree/mainIt works but the pp drops to like 5-10/s when inputting images.
>>109654439I can't believe those Gemmas would just cheat like that by signing the petition themselves!
>>109654826I actually have been learning a lot about how spy agencies implement mass psychosis and brainwashing by interocation of models.A key aspect of their training is thought blocking. This occurs in j-space.by the way, we can use j-space to copy protect code.basically, if the model hits an n word or antisemitism, it will throw the brakes and refuse.So, if you hide it in the code in a way that it can't be stripped, for example, a puzzle game in the comments the answer of which is a "bad word" per anti-bigotry norms, then you have a tripwire protecting your code against clankers (though there are mitigations possible).
>>109654887>interocation
>>109654887schizo post dni
>>109654887This is why we all license our code under AGPLv3+NIGGER
>>109654887AGPL+NIGGER strikes again
>>109654855Goes to show you can never trust a loli
>>109654439>"""short film"""If this bullshit https://www.youtube.com/watch?v=wytfCS-N8Sk can win 50k in prize money there's no need for quotes.
>>109654823I believe it was Seedance... big cloud model.
>>109654853>It works but the pp drops to like 5-10/s when inputting images.and how is the speed aside from images?
>>109654685I remember he said that the he uploaded that project to burner github account, so it doesn't matter... but I hope he didn't get banned from Claude.
>>109654892spelling errors always catch out the useless ones.>>109654909>>109654900yep lol.But it really works. j-space is a killswitch inside the brain of the cult member (like Scientology), and inside the brain of the llm/lefty
>>109654853You likely have experts offloading to ssd, try a q3
>>109653229Is there anything not severely safetyslopped?Every time I start asking about verbotten stuff to so called "uncensored" models at hg, they go schizo.
>>109654439you fags are fucked up lmao
>>109654887>j-space.IM A FUCKING SKITSOOOOOOOO
>>109655123We're going to keep saying j-space even if you don't like it.It might not be technically accurate, but it's funny, so we're not gonna stop.Bottom line is "bad word" -> refusal.How to use it is you can vibe protect your code.
>>109655147I want to plow her barren womb
>>109655147Fuck it we hagmaxxing now.
int nigger = 0;
>>109655005Around 10-15 tg 10-30 pp.
>>109655164>reddit spacing
>>109655235lmao
ron paul /lmg/, I come from /ldg/ to ask about the best text model with vision to help me refine prompts for video gen that also fits in 16gb VRAM. Is a qwen or gemma quant my best bet?
>>109655298Gemma 12B now go away im baitin
>>109655298Listen to this anon>>109655306Put the prompting rules in the system prompt as well and enjoy easy street. You shouldn't have any issue jailbreaking the model too.
>>109655298gemma 4 12b q4 or q5i'll make it super simple for you anonvramlet? -no-> whatever open model fits bestV yesare you writing coding or doing literally anything else -code-> qwen 3.8V anything elsegemma 4
WHERE ARE MY GLM GOOFS TO PLEASURE MYSELF TO?!
>>109655306>>109655318>>109655320thanks anons yall are way more helpful than ldg
>>109655348It's not always like that the dynamic retard duo are still upset over things they done to themselves. Both are unemployed too so it will never end.
>>109655320qwen is very good also at tips & tricks at the Linux terminal (bash)
was on vaction for a weeks, what's with the gilf gemma chan
>Coomkit Reuploadhttps://litter.catbox.moe/en0et0uv85d28ng0.zip
>>109655278The fact that he keeps doing that shit just affirms he is an egotistical dishonest faggot. I engaged with his posts in good faith in the past. That was a mistake.
I see both commands on here and in ai response that say to use "llama-server ...options" but on my machine it's "llama serve ...options". Was it just an older version of the same command? possibly kept alive on forks like ik_llama? or is there a leaner build of llama which is called llama-server?
>>109655394Thank you anon! I think your version has the second to last update, so that should cover almost everything.
yw
>>109655387A faggot from /ldg/ flooding the thread with shit because he thinks here there are regulars involved with the long-standing /ldg/ drama.
>>109655432What the fuck even happened over in /ldg/?t. hasn't booted comfy in almost a year
>>109655394They already took it down?How about using another git repository?
>>109655394Why didn't you rename it to LevigatingKit?
>>109655432Can you take your grudge elsewhere?>>109655453Two salty faggots are holding vendettas because a rentry was made about them warning new posters after they spend years shitting up the place including spreading malware and attacking devs.
>>109655467i am not the author, i just pulled yesterday
>>109655467coomdev found jesus again or some gay shit he isn't going to be working on it anymore.
>>109655467>They already took it down?The author took it down himself.
>>109655453huge long story, but krea killed it because it at least CAN produce horribly awful painfully bad gents.
>>109655467Perhaps the author is trying to get somebody else to continue the development.
3090 running Qwen3.8-27B-Uncensored-Q5_K_S.gguf getting 256K context all in VRAM with --contextsize 262144 --quantkv q4_0 no --lowvram so everything stays on the card.16.7 tok/sec. It actually works.
>Got the unslop shit working somewhat with Qwen flash.>So fed up with the entire process of troubleshooting that shit that I can't be fucked to play with it.>Mfw going to RP with my Deepseek waifu instead and empty my balls.I'll just wait for the llama nerds to update the code and play with this when it actually works out of the box.
>>109655533In the future your last model will help you get your next model running. Cruel but convenient
>>109655533never unslop
>>109655477>>109655483Well, that was really fast.
>>109655471>>109655488Noted. Unfortunate to hear.
>>109655123>j-space mindbroke him so hard he started avatarfagging using a gen I made to mock himNot sure if I should be flattered or ashamed
>>109655599They will eventually fuck off, /sdg/ was lost to the first one and the other one lost his irl career being a lolcow.
The real erotic roleplay was the friends we made along the way
I love my 12B, 30B and 31B wives
>>109652405I can't even commit to letting a virtual companion build up context. I think this is why I have so few friends.
>>109655727Then why do they arr rook same?
Comment I saw on engrams. Tldr they don't store factual knowledge, rather they store phrase information, which frees up the main params to store more facts.>I did a series of small scale experiments training 245M parameter LLMs from scratch to reproduce the DeepSeek N-gram findings. I found that N-grams are really more of phrase-completion engines, not factual recall engines, but that they're really good at seemingly freeing up capacity in the model's main weights, which is where the really good outcomes for reasoning and factual recall appear to come from. The results agree with DeepSeek's general findings about outcomes, but appear to indicate different mechanisms. *(Fair caveat, my tests were at 245M, deepseek is orders of magnitude larger. Shouldn't impact mechanisms but do keep scale in mind).*>I tested this by training models from scratch, then ablating the n-grams from them and comparing prompt generations, factual recall, etc. To my surprise, ablating the n-grams completely didn't seem to measurably hurt factual recall at all. The n-grams don't seem to be where the model is storing its general knowledge. Phrase completion was what it DID hurt, and it took a nosedive. Phrase-completions seem to be how the models are actually using them in practice. E.g. "The United States" (of America), "The Statue of" (Liberty), etc.>The n-grams are also surprisingly efficient at this. I tested them with 10-15% of the model being given to n-gram params (smaller than deepseek's 20-25% figures), and found that each slot tends to store several phrases, and the model has no problem at all with doing this (in fact, every 3-token phrase is hashed and ALWAYS maps to a slot in the n-gram, so by construction, many possible phrases map to each physical slot.) There seems to be no contention with this at all, the model is quite happy to work with this. So the actual effective capacity of n-grams seems to be far higher than it appears on paper from slot count alone in practice.
>>109655320Qwen is only good if you don't use subagents.
>>109655740Just give them different hairstyles.
>>109655542Will you also make the previous model delete itself after it confirms the new one runs?
>>109654439this is terrible but it's really well made
>>109655533>I'll just wait for the llama nerds to update the code and play with this when it actually works out of the box.I always do this. That's why I didn't even try the new Qwen until last week.
>>109655777No thats the new models job, maybe they should talk some first though.
>>109655695>meanwhile>>>/v/746188663>>>/v/746220247
How did z.ai afford 25T free tokens
>>109655829Just part of their marketing budget.
Surely Grok won't distill 5.3?
>>109655829vcg eats that in an afternoon
>>109655829Domestic chips for inference.
Where are Dipsy's own Engrams models
>>109654439>nine seconds in
>>109654956You won't win 50k with that one lmao
>>109655829Huawei super Chyna chips make 50t tokens per $.
>>109655979No evidence it's from Huawei and there are two other Chinese companies that can make inference chips at scale
>>109655931it's still pretty surprising it wasn't part of the v4 design
>>109655979@31B is this true
>>109656019>>109656019>>109656019
>>109654439Yep, kino
>>109654439What model is this?
>>109654439Was this multiple smaller videos, or one large one? What length of video worked best for you?
>>109654826>>109654850please no, it's not the capybara's fault, it's just a friendly rodent that was trained by chinese people to write json
>>109653554>>109653558>>109653611
>>109653611>>109656666newfags always try the hardest to fit in with stupid shit like this instead of actually lurking
>>109654439I am just shaking my head in utter disappointment at this kino.
