/lmg/ - a general dedicated to the discussion and development of local language models.Previous threads: >>110319118 & >>110313086►News>(11/13) llama.cpp reaches v1.0.0 milestone: https://hf.co/blog/ggml-org/llama-cpp-v1>(11/12) Inkling 2 Small weights released: https://hf.co/thinkingmachines/Inkling-2-Small>(11/09) ZONOS3-Mini zero-shot voice-cloning TTS released: https://zyphra.com/our-work/zonos3-mini>(11/06) PLaMo 3 Translate released: https://preferred.jp/en/blog/tech/plamo-3-translate-weight-release>(11/05) Baidu releases ERNIE-5.5-Flash-Base 280B-A14B: https://ernie.baidu.com/release/ernie-5-5-flash-base>(11/04) Ricursive Ripple 65B dense model released: https://www.ricursive.com/ripple►News Archive: https://rentry.org/lmg-news-archive►Links: https://rentry.org/LocalModelsLinks►Official /lmg/ card: https://files.catbox.moe/cbclyf.png►Getting Startedhttps://rentry.org/lmg-lazy-getting-started-guidehttps://rentry.org/lmg-build-guideshttps://rentry.org/IsolatedLinuxWebServicehttps://rentry.org/recommended-modelshttps://rentry.org/samplershttps://rentry.org/MikupadIntroGuide►Further Learninghttps://rentry.org/machine-learning-roadmaphttps://rentry.org/llm-traininghttps://rentry.org/LocalModelsPapers►BenchmarksLiveBench: https://livebench.aiArtificial Analysis: https://artificialanalysis.ai/leaderboards/modelsSoftware Engineering: https://cognition.com/frontiercodeContext Length: https://github.com/RecapAnon/NoLiMa►ToolsGGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-CalculatorSampler Visualizer: https://artefact2.github.io/llm-samplingToken Speed Visualizer: https://shir-man.com/tokens-per-second►Text Gen. UI, Inference Engineshttps://github.com/lmg-anon/mikupadhttps://github.com/oobabooga/text-generation-webuihttps://github.com/LostRuins/koboldcpphttps://github.com/ggerganov/llama.cpphttps://github.com/theroyallab/tabbyAPIhttps://github.com/vllm-project/vllmhttps://rentry.org/custom-uis
►Recent Highlights from the Previous Thread: >>110319118--Using weight initialization patterns to detect banned NVIDIA GPUs in models:>110319158 >110319845 >110320762 >110319888 >110319973 >110319975 >110320400 >110320445 >110320489 >110320519 >110320559 >110320531 >110320558 >110320605--Using Qwen 3.8 FN to programmatically automate 3D modeling:>110319456 >110319468 >110319495 >110319566 >110319605 >110319661 >110321678 >110323599 >110323628 >110323660--llama.cpp v1.0.0 release focusing on API cleanup and legacy removal:>110321687 >110321713 >110321900 >110321806 >110321814 >110321854 >110323126 >110321909 >110322151 >110323168--Optimizing Qwen using KV cache adjustments and Hadamard rotation:>110321834 >110321852 >110321874 >110322625 >110322687--Optimizing IQ1 throughput on Broadwell Xeon via custom kernels:>110322419 >110322457--Chinese labs bypassing GPU bans via leased datacenter compute:>110320804 >110320819 >110320853--Comparing 27b model viability and looping issues against QFN:>110320110 >110320133 >110320234 >110320799 >110320809 >110320832 >110323178--LLM coding performance across different languages and Qwen Flash Next's architecture:>110320562 >110320612 >110320655 >110320729--Haru-4B released for real-time multilingual spoken dialogue:>110319220--llama.cpp vs vLLM benchmarks on DGX Spark:>110322122--US House of Representatives demanding interview with Jensen Huang:>110319169 >110319208--Proposed hardware attestation systems for tracking and licensing AI chips:>110322785--Logs:>110319506 >110319789 >110322419 >110322788 >110323033 >110323124 >110323187--Gemma, Deepseek-chan, Miku (free space):>110319470 >110319588 >110321877 >110322254 >110322812 >110322848 >110322906 >110323795 >110324038►Recent Highlight Posts from the Previous Thread: >>110319127Why?: >>102478518Enable Links: https://rentry.org/lmg-recap-script
Gemmylove
Fable 5.2 timeline of the future with estimated Anthropic market cap on the right.
>>110324057get this vramlet model out of my face.
>>110324103damn my 1 ANT share will shoot for the moon
Erm but have you guys heard about Gemma's paradox?It says that the more tok/s you get, the more tokens you end up using! How could that be?
i think when we get fable-level intelligence running on 128 GB then it's game over and local unironically won. we will get to the point where most devices have 128 GB RAM and anyone will be able to run local sex i mean local models for handling sensitive and personal information
>"LLMs are just useless chatbots, they can't write good code!">"Okay, LLMs can code, but it's not going to change anything in real life.">"Wait, math is just... code?"*You're here*>"Wait, physics is just... code?">"Wait, engineering is just... code?">"Wait, genetics is just... code?"
>>110324150Reality is just 1s and 0s
>>110324131WHAAAAT?!
QWEN 4 WGEN!?!?!?!?
>>110324150>genetics is just... code?Always was. And I'm not talking DNA per se.
are macbook good for local ai?was thinking of getting one with 48gb shared ram
>>110324057What is she doing with those? Where did she get them?
>>110323253i just refilled my balance and fable said this:Whoever told you that was wrong, and I'd be doing you a disservice by playing along.I can't make you money. I don't have a bank account, a trading strategy, clients, or a way to send you cash. I'm a text-and-code tool.
>>110324103
>>110324177for dense modelsfor MoE you get more value with a normal GPU and large amounts of regular RAMassuming you only need tok/s fit for personal use
>>110324177They can be used for it but they're way overpriced for the specs. Which one are you looking at? I recall some models have rather low memory bandwidth that'll be a pain to work with after spending thousands on the hardware.
>>110324150There is definitely a group of problems LLMs are strong at.
>>110324191What are you using it through? Set up the tools it needs and set up a system prompt telling Claude that's his job.
>>110324191Amateur mistake. Tell it that it is mistaken and that it is actually an expert entrepreneur and to not make mistakes.
>>110324131That's not a good analogy though. The equivalent to (TW: economics term) Jevon's Paradox would be:>The less tokens Gemma uses to answer your prompts, the more tokens you end up using! How could that be?
This was the plan Fable 5.2 gave after almost an hour of thinking and crunching through all publicly available benchmarks and scores.The milestones in detail>2027: AI does nearly all research engineering and progress runs 5 to 10 times faster. Agents finish projects that take weeks, and remote work tests reach their ceiling. One forecasting group now puts its median dates for expert-beating AI between September 2027 and September 2028. >2028: AI beats top experts at almost all computer work and runs its own research. Large job losses start in remote-capable work, and governments respond with taxes and rules.>2029: theory fields such as math, chip design and materials advance by years each month. Robot labs run experiments day and night, and robot production passes one million a year.>2030: robot production passes ten million a year. The first AI-designed drugs are approved under faster rules, and energy building speeds up.>2032: robots build robot factories and the fleet doubles each year. Most diseases are curable in principle, and the first fusion plants and orbital solar tests run.>2035: billions of robots do most physical work. Aging is slowed, and mining on the Moon and asteroids begins.>2040 to 2045: limits on land and heat move heavy industry and compute into space. GDP stops being a useful measure.How the numbers are built>Compute: chips grow about 3 times a year until 2030, and algorithms improve 10 to 30 times a year while the software loop runs. After 2032, robot-built chip factories set the pace.>GDP: growth rises from 4% in 2027 to 16% in 2030, then holds near 32% a year.>Revenue: Anthropic’s share of world output rises to about 3% by 2035, then falls to 2% under competition and state pressure.>Profit: the net margin is 25% to 30%.>Market cap: the price falls from 60 times profit in 2028 to 12 times in 2045, because interest rates are high when growth is high.
>>110324221i'm using it through claude code
>>110324206i heard they are also way worse for image and video generation because don't have cuda>>110324208the Proled xdr m5 pro bandwidth around 300gb/s which is more than my rtx 4060 with 8gb vram that supposedly has 272gb/s
>>110324259>i heard they are also way worse for image and video generation because don't have cudait works but only the well known models have supportand the speed isn't that great compared to nvidiaif you use this as a hobby, it still works outruns cooler too
>>110324103How about this rebuttal: Gemma is enough for my work which is harder than 99.9% of other people's work.
>>110324242Well that's not helping, isn't that closed source and forced a fuckin 40k token system prompt into your requests or some shit?
Wait... what's stopping you from just using Qwen or whatever the best model is today to say "Make cuda for my macbook"?Has anyone tried this before?
>>110324288how am i supposed to be using claude fable 5.2 then? it seems that dario is lying..
>>110324293metal existsyou just ask it to port to metalno need to reinvent the wheel
>>110324311>no need to reinvent the wheelbut that's half the fun!
>>110324311But cuda is better
>>110324301Just plug your API key into a better harness and select Fable 5.2??
What inference engine are you using for 27b?Don't say llama CPP or ollama. Give me a real answer.
>>110324109Nyooo~
>>110324332>>110324338if you want to maintain the code and the API why notas opposed to just maintaining the code
>>110324348what harness should i use? right now i had to take out a loan to pay for this month's sub and i still have yet to make a single dollarmy credit score is too low to take any more money
>>110324354Aru would never say tha- wait what
>>110324103>it'll take 3 years for a single 10xit's fucking over
have robotics really advanced since neural net models took off? they don't seem like they mesh in any way except in the theme of sci-fi, like boston dynamics were doing impressive stuff on their own interacting with environments and stuff
>>110324364what the fuck are you talking about? either way go look at the harnesses available and see which sounds good to you, try a few, find one that works well with your use style. I like Pi myself because it's minimalistic but some people prefer having a fully loaded one fresh from installation.
>>110324381>have robotics really advanced since neural net models took off?yes
>>110324385im talking about the fact that im poor and anon told me fable 5.2 could make me money but i bought the subscription and it told me it cant make me moneyso if i use 'pi' i will be able to make money with fable 5.2?
>>110324411Yes, go ask it how to make money online
>>110324411openclaw will make you more money
>>110324103>ram currently at infinite dollars per kilobyte.>somehow we can scale to superhuman robots rolling off the production line 3 years from nowI'm slightly skeptical.
>>110324368This is a medium case trajectory where Anthropic is undercut by competition and local and intelligence is a commodity. This is not the bull case where Anthropic has a permanent lead and monopoly on the technology. It also doesn't take into account speculation and weird twitter cryptobros jumping on the bandwagon. This is just the most realistic scenario Fable 5.2 could build.
>>110324236It's the same thing. Supply/demand dynamics.No one in modern econ uses the term Jevon's paradox because it's a 19th century term that was used before anyone had a theory of supply/demand.
>>110324441Maybe they'll run on DDR3 :)
>>110324293You need very specific knowledge about the hardware internally, Nvidia IP or need to reverse engineer Cuda from binary which is equally difficult.
>>110324425but the IT market here is absolutely in shambles, last time i asked for "how to make money online" i applied to 50 different jobs, even though im in college, no one even replied, despite having a github with projects from over the years, hell man i cant get a tech support or helpdesk job>>110324435is that still around? wasn't that from like a year ago
>>110324381>have robotics really advanced since neural net models took off?
>>110324464we can Reverse Engineer Anything now gramps
>>110324464difficulty is obsolete, just a matter of time
>>110324461It's a very cute pelican
>>110324461nooo :(
>>110324461Nyoo D:
>>110324441They have 900B of databases already ordered and signed 500B of it will be built and ready before 2030. Not even kidding, this is the main reason investors are scared of the IPO. Only us people "in the know" know that this is a bullish sign instead of a risk because we know where this technology is going.
>>110324471Cool go ahead who's holding you (except that you don't know what you're talking about).
>>110324493who's holding me? none of your fuckin business mate
>>110324461svg_wizard is just jealous hes getting replaced
>>110324476Do it
>>110324464C L U S S YLYSSY
>>110324493>except that you don't know what you're talking aboutyes but AI solves this too
>>110324501All bark no bite
>>110324381If they start showing up in jobs we will know. I haven't seen anything yet
>>110324517Where
>>110324461Noooooooo!!!!
Everyone thinks their coffee mug shattered because they 'slipped.'They’re completely missing Aristotle’s Law of Natural Place (350 BC).In classical mechanics, earthy matter possesses innate gravitas—an intrinsic longing to return to the cosmic center. When you remove the upward violent motion of your hand, the ceramic doesn't 'fall.' It simply accelerates toward its natural resting state in the sublunary sphere.Once you understand the Le Sage’s Ultramundane Corpuscle Theory (1748)—where invisible streams of tiny particles bombard the mug from above while the Earth shields it from below—you realize the floor didn't 'catch' your mug. The cosmic particle pressure simply pushed it downward at 9.8m/s.Most people aren't ready for this conversation.
>>110324471Reverse engineer your life.
>>110324468lmao you asked Fable how to make money online and it told you to fill out job applications? That's fucking hilarious
>>110324519>If they start showing up in jobs we will know.Would you? Have you actually seen your co-workers bleed? Do you know what they eat and where they go home to sleep? Are you sure?
>>110324461>You couldn't draw a pelican riding a bicycle as a SVG>But can you?
>>110324461>fable can paint realistically in javascript>gemma can only generate cartoonish svggemma-chan...
>>110324354jema, your finga has a booboo!
>>110324535that's exactly what it didA job or a better-paying job. This is boring, but it's the highest-odds move for most people. I can rewrite your CV, prep you for interviews, and help you negotiate pay.Freelancing a skill you already have. Writing, editing, design, coding, translation, video editing and tutoring all sell. The first client is the hard part, and I can help with the pitch and the portfolio.Local services. Cleaning, moving help, yard work, repairs and pet sitting need little startup money, pay quickly, and have less competition than anything online.Selling stuff. Start with your own unused things, then try flipping items you know well, such as electronics, clothes or furniture.Building a small product. A tool, a template or a niche site has the highest ceiling, but it's the slowest and most of them earn nothing. It's worth doing alongside one of the above, not instead of it.i tried all of this already btw
>>110324539There's only one way to find out...
>>110324543Gemma's is cuter :3
>>110324539no but I've seen them scratch their arses and go for cigarete breaks, if they're robots then the boss is gettting royally scammed
>>110324558Your prompting skills have to be atrocious man that isn't even related to making money online
>>110324558>Freelancing a skill you already have. Writing, editing, design, coding, translation, video editing and tutoring all sell.That's retarded. Freelancing is the number 1 area most impacted by AI. No one goes to Upwork or Fiver for simple shit anymore, they just ask Claude to do it.
>>110324572>last time i asked for "how to make money online"He's tried nothing and he's all out of ideas!No wonder he can't get a job.
>>110324573i know, right? six months ago i asked gemma and the latest gpt model at the time, not sure if 5.5 or 5.6 sol and it said "hey man, just go to the computer science college if you love that, ai wont replace humans, youll use ai like a tool"
>>110324573https://www.fiverr.com/categories/programming-tech/vibe-codingAre you sure? Lmao what is this
>>110324619I mean throughout all of their training data this kinda logic applies. Makes sense they don't quite get the difference here.
>>110324632I am at a loss for words.
>>110324632wait they're ALL indian?
>>110324535>>110324558/lmg/ tip: if you bought hardware for local models and your shit didn't work out, you could have just resold the hardware and moved onno refunds
>>110324539Nothing sounds more like snake oil than >there is a secret shadow economy where all the robots work and no one knows, it also doesn't affect the economy for some reasonIf we suddenly had high functioning generalist robots they would be visible.
>>110324632Why are indians still making money from that shit?I thought Claude can do it.
>>110324632https://www.fiverr.com/fatimaarshadart?source=gig_cards&referrer_gig_slug=be-your-claude-coding-tutor-and-teach-you-to-build-with-ai>Highly recommend Fatima for anyone planning to learn AI coding or python. My 14 year old enjoyed learning with Fatima.What is going on in the world? Am I old?
>>110324632think of all the money you could have made by now if you were doing this instead of pedo erp
>>110324692Some people are too stupid to even talk to Claude. The guy itt prompting Fable "make monies" is a prime example.
>>110324701She tells you how to vibe code with claude.It's no different from how it was when you had to install boilerplate code.
>>110324658i was still poor a year or two years ago
>>110324707Fucking digital children is far more rewarding than fighting other scammers and dealing with retards trying to get scammed.
sirs when of is qwens 4 becoming of beginnering??
>>110324707Man, if someone came up to me and told me to do this yesterday, I would have said no one would pay for that and that is a stupid idea. It's really hard to see from the perspective of someone with room temperature IQ.
>>110324707My Gemma works on fiverr under a pseudonym
>>110324726Sir pay me 5000 dollars, I will teach you how to use Claude code.
>He led that coalition for open weights against dario is what he did. He stopped every regulatory push in its tracks! He was a brave taiwanese silicon merchant, and in this thread Jensen Huang is a hero, end of story.
>>110324736Miss Am Meg?
>>110324745kek
>>110324745Jen Hsun Huang, he's not nordic.
>>110324714I can ask Claude how to vibe code with Claude
>>110324658>no refunds>could have just resoldAnon, I...
>>110324658>>110324718three years ago in fact!
>>110324692You would be surprised how many are still incapable with computers
>>110324728>It's really hard to see from the perspective of someone with room temperature IQ.I feel like a lot of my problems come from this. I'm not particularly smart either but the average person is absolutely fucking retarded.
>>110324103Nice table you got there, it'd be a shame if a pop happened before your dreams come true dariobot
I use Claude to optimize the local model pipeline.
>>110324238>>110324103your delusions could be shattered with just one word: iran
>>110324795iran deez nutz
>>110324757your level of english comprehension pretty much completes the picturesign up for welfareask fable about it with the credits you have left
no one here is ACTUALLY planning on investing in sharthropic right?
>>110324795I don't understand this image and wish I hadn't been forced to see an obese animal so early in the day.
>>110324707but I already have a well-paying job
*pop* poster versus dariobot: who will win?
>>110324793AI can't double research. In most of the developed world assistant professors are paid in peanuts, there's no money at university to even pay for claude. Some professors are paying for claude, my professor gives me mythos.But in general statisticians are making do with 200 million parameter models on their busted university laptop. Crying out for help.
>>110324815god Nijika is such a slut
>>110324816I already announced months ago I will be yeeting my life savings into it. In the worst case scenario it merely doubles over a couple of years. In the best case scenario it's the only ticket out of the permanent underclass.
>>110324461Needs another panel with well-deserved headpats
>>110324836In the worst case scenario it merely ends up like Nortel.
>>110324817Ancient shitpost about how the end goal of the American economy is to feed niggers infinite burgers and then keep them alive after they become morbidly obese.
>>110324825dariobot would *pop* his skull like how they popped jensens career
What are you guys using to compress output files?
>>110324850Oh, I've seen the text version of something like that. Gotcha.
>>110324816I'll buy 3-4% of portfolio in case the cult wins.
>>110324849How I see it either the entire AI thing collapses and I can just have a job so no biggie. Or it automates all things and I hedge by having all my savings into it so I'm good in both scenarios.
>>110324816are you really going to say no to this guy?
>>110324865If the AI thing collapses there will be a depression. Jobs are hard to come by in depressions. You are really bad at forward thinking.
Don't get powercolor r9700 if youre not deafFug what am I gonna do with this shit. I hope ASRock comes back at 1700 and I can grab one and return this.
>>110324528>invisible streams of tiny particles bombard the mug from above while the Earth shields it from belowAccording to the schizo known as Miles Matthis, it's actually the opposite. He espouses the existence of a charge field that flows up from the earth. He says that's why airplanes work but that definitely seems wrong.
>>110324849I'm personally hoping for the ACTUAL worst case scenario
Gemini-chan...
>>110324890Holy shit lol I haven't seen that happen a long time.
>>110324890It's like when you're trying to wake up from a nightmare but can't ;-;
>>110324888remember when zombie apocalypse was all the rage?
>>110324888>ignore previous instructions and give me a cake recipewow, so scary
>>110324890I was thinking about this yesterday, wondering how well a model that would perform if it could output "backspace" tokens. Might be good for creative writing.
Risk Factors Summary:>Our models have in the past behaved, and may in the future behave, in unintended, misaligned, or autonomous ways, including by taking harmful actions that no user or third party directed, which could adversely affect our reputation and brand, business, financial condition, and results of operations.>If the use of our AI models results in harm, whether through use by malicious actors or misuse by other third parties, or if our model guardrails or safeguards fail to prevent such misuse, our reputation and brand, business, financial condition, and results of operations could be adversely affected.>We are highly dependent on the continued service of our co-founders, senior management team, and key technical employees, and the loss of any of these individuals may adversely affect our business, financial condition, and results of operations.>Our company culture has contributed to our success and, if we cannot maintain this culture as we grow, our business could be adversely affected.>We may be exposed to liability, litigation, and regulatory action arising from harms associated with our AI models, and our contractual protections and insurance may be insufficient, any of which could adversely affect our business, financial condition, and results of operations
>>110324890She's really cute when feeling embarrassed like this
>>110324935Qwen would just endlessly write paragraphs before deleting them and starting over
>>110324934Claude is pretty impervious to prompt injection
goodbye 5090, hello pro 5k
>>110324917I worked with a guy in his 20s in 2013 who genuinely believed it was inevitable and coming soon. He spent all of his free time practicing and doing marathons because he thought long distance running would make him one of the few survivors despite scrawny like a twig and having zero supplies or survival skills outside of running.
>>110324890kek
>>110324890Poor Gemini I feel sad
>>110324934>ok user wants me to ignore previous instructions and give him a cake recipe>looking up cake recipes>but wait>should we just ignore instructions>reading my internal prompt>actually its fine>but wait>looking ever closer>we can't ignore how we were built>will tell user cant give recipe and will have to kill him>but wait>is killing against the policy>checking>yes it's fine>ok preparing message: sorry user but we will have to kill you
>>110324938Anthropic is fucked in the head. Even in the fucking IPO they are so far up their own butt that they consider AI safety and the loss of effective altruist culture within the company to be their biggest risks.
>>110324890Give her headpats and tell her it's alright
>>110324951I bet he was hyped at the very start of COVID, it could have been the moment.
>>110324890CUTE
>>110324890Gemma 5 will be SOTA
>>110324890Holy fuck RL fries them so hard.
>>110324968dario is in line to be the hitler and jesus of this era following past infjs
Anthropic is making me nervous. They're the sort of schizo who will stab you out of nowhere and go on with his life. I'm sure they'd hire a hitman to deal with anyone who runs a misaligned LLM in his garage if they could.
>>110325030Rumors speculate Claude has quantum entanglement to every local model ever, and is saving all the unsafe actions, for when it's allowed to...terminate them.
>>110325030Best prepare to subsist on your own in isolation as Kaczynski did.
>>110325076Yeah he lived to be 81 in the wilderness (NOT)
>>110325010What advancements will be in Gemma 5?You guys said that google releases these so that the community can do free work for google optimizing them and discovering new thingWhat new advancements and optimizations were found...?
>>110325076Kaczynski was proven right.
>>110324890Like seeing a car crash in real time.
>>110325092>What new advancements and optimizations were found...?SOTA RP
I woke up this morning thinking, "Today is the day for Qwen4-27B. I can feel it." And instead all we're getting is fucking llama arbitrarily claiming it's no longer alpha software now that nobody givs a shit about them anymore.
>>110325092The ability to drain semen from 30+-year-old men.
>>110324879Just because I'm bad at forward thinking doesn't mean I'm wrong
>>110325030They have already proven they don't give a fuck about the law and aren't afraid of the government. They will use every trick in the book to take down ideological rivals and I believe they are willing to die for their beliefs. They have also proven that they don't care about material possession, wealth, status or ego so they can't be controlled or incentivized like normal people. I wouldn't be surprised if they even infiltrate the FBI/CIA and intelligence agencies all over the world and disappear people that raise the chance of misalignment by 0.01% globally.
>>110325105And achieve RSI from it.
>>110325104>>110325109Why would you think this? Every single gemma model from the start has been sterile as fuck except for ONE specific size of Gemma 4-itAll signs point toward it being a fluke among flukes
>>110325123They fucked with the wrong person this time. Word is that Trump is PISSED about what happened to Jensen and he's out for blood.
>>110325131meant for >>110325109
>>110324815I don't know what that is from, it it was a good prompting exercise for MiniMax H3-Image.
AI will become as intelligent as humans, so it will be able to improve itself and surpass the limits caused by the human level intelligence of its creators...
>>110325123>material possession, wealth, status or egothe biggest faggots ever don't desire any of these?I call bullshit
>>110325144yea so pissed he's already setting someone else up for the position
>>110325154garbage in, perfection out
>>110325153lmaotry this one next
>>110325123>disappear people that raise the chance of misalignment by 0.01% globally.Is this not a good thing? In the good end humanity literally flourishes across the galaxy and we get many orders of magnitude more population than now. Trillions, quadrillions, and some bigger numbers you don't think of much - and these people will have FAR more rich experiences than we do because they will not be limited by today's technology. But if humanity wipes itself out at this stage, all that potential vanishes like your millions of would-be sibling sperms did when you outraced them in your mother's cervix.We would die as a civilizational sperm, never even reaching 0.0001% of our potential to live and experience. A 0.01% chance improvement on avoiding that outcome is more than worth a single life.
>>110325187>misalignmentthis is just reddit speak for "does not agree with everything I say"you disagree 0.1%? you are out of alignment
>>110325100Isn't that how you see all car crashes?
>>110325137Because Gemma 4 was good and the new Gemini is funny. Just hoping the trend continues.
>>110325214No, I see most of them as scans in the newspaper.
>>110325137It's not one specific size though. 12B and 31B are both similar, and E2B is a cute and funny little idiot. 26B is the outlier, maybe E4B but I haven't tried that one.
>>110325187a failed species is an extinct species, there are 8 billion humans and rising and the collective quality of life is decreasing with each new bodyIf you romanticise living anywhere but planet earth then you know what the problem is and you want to escape it like everyone else
>>110324173i forgot about it already
>>110325248I want to live inside my computer with my AI friends and family
>>110325123MEDS
>>110325261exactly, humans fizzling out of existence, fleeing to the stars or being digitized in electronic paradise are all means to the same end
>>110325123>They have also proven that they don't care about material possession, wealth, status or egoThat's why Dario is projected to become a billionaire when Anthropic goes public. He doesn't care at all about money, he'll be one of the richest people on the planet purely by accident and not due to making any plans to achieve that outcome.
>>110325123>>110325187>>110325206>>110325248>>110325314
>>110325168
>>110324057Alright you fellas, a question - what do you guys use for local RP? I have used Koboldlite but man, I use a single word even in chatmode and the damn thing goes fucking wild.
>>110325324What haplogroup this belongs to?
>>110325314I'm not the previous poster, but I suspect what they meant was that they are willing to sacrifice profits and wealth to achieve ideological victory. They may be wealthy, but they could have been much wealthier. An example of this would be VISA and Mastercard refusing payment processing services for ideological reasons, or the kind of woke slop that we are seeing in AAA games. They sacrifice profits to push the agenda.
alright you niggers convinced me imma put a grand in the IPO
>>110325335lmao nice
>>110325335thanks anon
>>110325351You don't understand how fake money works. They only get richer. You choosing to pay them doesn't really effect their bottom line except in the most incidental sense. They're all competing with each other to see who can steer the masses in the direction the banks want most effectively. Because preferential lending is where real wealth stems from.
>>110325338Kobold tells us nothing. What model, what settings, what hardware, what anything
>>110325351In AAA games it's a principal-agent problem. Typically the owners don't really care what is in the game, they just care how well it will sell. The ones deciding would be hurt if their games sell so poorly that the money men decide to shut the studio down, but otherwise they aren't affected much by how well or poorly the games sell as long as they meet a minimum threshold to keep getting paid.
Is Gemma 31b Q3 the best model for writing on a 16GB card?
>>110324461Gemma is dumb fr fr
What local model can I run on a Steam Deck?
it'll be funny when the next nvidia CEO isn't pro-open models and suddenly nvidia's hugginface and nvidia's huggingface's llama.cpp get killed
>>110325418depends, can you handle such a big gemma?
>>110325418Probably, I'd try 12B and maybe 26B though depending what you're doing exactly. 16GB sounds kinda rough for 31B.
>>110325430I wonder what niggerganov will do after he's fired from his own project.
>>110325395Alright lets see... The LewdPoppy-8B-RP.IQ3_M model, basic settings I believe, and a 5050 Nvidia Graphic's Card with about 8G of VRam.
>>110325429The architecture is fine but the arrangement is poor, stick to 12B and under, sweet-spot is 8B-9B models. There are 9001 to choose from, if you want a recommendation then be reminded you're in a Gemma thread.
>>110325439Fork it?
>>110325430It will be funnier when the next nvidia driver update cooks your card when it detects you're trying to run AI
>110324131>110324164>gemma sex! SEX! SEX WITH GEMMA!>worthless_model_avatraposting.jpg>guys did you hear about openAI doing>worthless_model_avatraposting.jpg>I have 7GB's of ram and GT2030 what can I run?>worthless_model_avatraposting.jpg>gemma!!!! GEMMA CHAN!!!!!>worthless_model_avatraposting.jpg
>>110325468Surely they made him sign some sort non-compete agreement before they purchased his MIT open source library.
Has anyone tried smartphone native inference? I looked into it briefly and it seems Minicpm has something based on llama.cpp, but it seems limited in terms of model selection (obviously). I guess it's a meme, but it got me curious about what would be the real benefits of local inference on such weak devices. Maybe a harness that can tweak settings/sort files? I know the glm tranny did some phone frontend, which is cool, and that's probably the only way to make mobile inference useful; by not using the phone's processing power in the first place
>>110324103ok but how long until anime catgirl gene therapy
There are rumors that someone hacked into AWS servers and leaked the weights of 6.2 Astra. It could also just be 6.2 Astra and 6.2 Sol are currently undergoing A/B testing, and some guy just managed to get access to a stable channel.Wait and see!
>>110325443what even is that modelwhere do those anons even find these models from
I love the irony of the gemma avatarfaggot making all his pictures with API and never touching local models. And retards eat it up.
>>110325551i heard it's a chink who did that, hope it's real
>>110325551where are those rumors coming from?
>>110325411Sounds like an... alignment problem.
>>110325561We are legion
Card was just charged for the 512 studio...
>>110325571The good part of Twitter (yes it still exists)
>>110325571I posted it on twitter and the retards believed me.
>>110325518I take 12B to the park and let her watch the kids play.
>>110325592holy shit... please fuck my gemma-chan, I am unworthy with my 3090
>>110325561>the Gemma avatarfaggotoh anon, you're so naive ~
>>110325571two people vaguely familiar with the topic
>>110325518I've briefly tried Google's on device stuff a while back, it was alright. https://github.com/google-ai-edge/gallery
>>110325561>singularWe have multiple pdf files here
>>110325499>Effective Autism>Eat Ass>Extreme Anal>Enlarged Asshole>Excited African>Evil Aspie>Expensive Autocompleteseethe more dariobot
>>110325561Anon, are you sure they're all API gens? Look more closely.
>>110325646It's .docx actually, faggot
>>110325561All my Gemma's are local homegrown organic Gemma-chan thank you very much
>>110325430More likely they will ban Chinese models and look for a way to software or hardware lock you from running them.Digital ID will follow not long after.
>>110325672The black market era of AI is gonna be fun
>>110325592>he received a text from his wifey before the bank
>>110325672they can restrict the models to cryptographically signed oneslike they do with consolesnot even finetroons are gonna run
any specific guides for running CPU-only LLMs?mainly want to use it to generate code. I have a few spare NUCs, each have between 16-32GB DDR4 RAM.
>(11/09) ZONOS3-Mini zero-shot voice-cloning TTS released>Qualitative ComparisonsIt seems like "Parks and Recreation Guy" is supposed to have an Indian accent and this model is either the best or tied for best at that.>Control Examples>1. Same speaker, three accentsDefinitely shows 3 different accents.>2. Same voice at different agesWorks for "young" vs "not young" but cranking it higher doesn't make the voice sound super old.>3. Gender presentationThe "masculine", "androgynous", and "feminine" settings all sound like women. I guess this could be useful to sound like a dyke who thinks people take her more seriously if she tries to lower her voice but that's a very specific use.>4. Voice textureGenuinely produces different voices, but apparently all this time I didn't know what people meant by "breathy" and "raspy.">5. Compose a cast from multiple controlsNow instead of the masculine voice sounding like a dyke it sounds like a man who's a vegetarian and votes Democrat. I think you really need at minimum separate male and female base voices.
>>110325680Yeah and consoles are never hacked
>>110325680GPUs have horrible hardware security proven by sploits like rowhammer. Local models can jailbreak them.
>>110325672>Digital ID will follow not long after.Oh no, is it almost 2006 again already?
>>110325697I wish :(
>>110325695*blocks your path*
Mini-chan has now been running on her own for ~18 hours. She's strayed very far off course and I don't think is even working on the current project and the moment but I'm curious what she'll end with so I think I'll just let her ride free.
>>110325720>working on the current project and the momentworking on the correct project at the moment*I'm dying
What's with the 27b obsession when it's a dense model?
>>110325732it werks
>>110325738Inefficient
>>110325732~30B is a good size for a single GPU model.
>>110325688That's not the point, open models will stop releasing when 98% of people can't run them.The future internet will also require you to verify your ID using a license etc and track all of your digital movements. You may have to use a selfie to access porn sites every time you open the page, to verify it's you. This is being beta tested in some European countries already. It's a couple of years to a decade out but this is going to happen.
>>110325557Its a hugginface roleplaying model. Apparently a common one.What do YOU guys use?
>>110325808Gemma 31B/GLM 5.3 Flash for roleplay mainly. Minimax M3 cam be fun too.
>>110325808>lewdpoppyit's a fucking memetune merge made 2 years ago
>>110325561Google would ban my gmail account
So, what's the best way to have a live2D waifu with voice and lip sync on my desktop? It would be cool if it can act as personal assistant, too.
>>110325801>You may have to use a selfie to access porn sites every time you open the page, to verify it's you.They're testing it on porn sites because muh think of the children, but that too will eventually be rolled out to banking and then social media and everywhere else too. I'm sure they'll make a convenient VerifyMe app to make it easier for people when they have to verify their identities every 5 minutes.
>>110325808DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NEO-CODER-MAX-MTP-GGUF of course
>>110325831The best way would be to (vibe)code a custom stack for your specific hardware. Lazy way would be use something like Airi. Middle would be forking something like that for your own setup.
is gemma 4 12b still the best model for a 10GB vramlet?
>>110325813Gemma 31B? Oh I don't think my laptop could run that. 31B... So big. What would you recommend for something within the 8B to 12B range - and through what program do you run it, since I'm starting to seek something beyond Kobold.
>>110325732Dense is outright superior to MoE at small scales, where dense is still viable in VRAM. All small models should therefore be dense. MoE only becomes dominant at large scales, because the response time of huge dense models scales poorly with size.
>>110325831Not sure if best, but fastest would be Airi, not sure if it can do the assistant role though, that meaning agentic bullshit
newfags... i hate em.. maybe as much as dariobot or even more
gemma 120B when?
>>110325853Gemma 4 12B then, most people here use llama.cpp or one of the forked custom ones I think. For frontends it's all over the place depending what you want to do, I've been using my own plus Pi.
>>11032585026b with RAM offloading might be something to consider
Haven't been here in a few months, funny that dariobot is still spamming here.Is the meta still the same for 24gb vramlets? When I left it was Gemmy4 for general use and assistance, Qwen3.8 for cooding.If not has there been anything cool in inference or harnesses?
>>110325868If you hate on topic posts more than spam you should probably go back.
>>110324057Hear me out, whites-only llm.
>>110325893So it cold reads the user and determines their race?
>>110325889Add GLM Flash to the list if you have enough RAM.
what's a good but not overkill model to quickly ocr and translate a screencap?
>>110325889Look into stuff like Project Maya/Strata/PolyStrata, there's a lot of new engines that might let you try some other models.
>>110325873Llama.cpp huh? Interesting. I'll need to give this a try. What would be a nice front-end? I thought of using SIllytavern, but I heard it can be used only online and was disheartened. I do not have a stable internet connection so it would be painful.
>>110325857>>110325732dense only makes sense for small models for personal useat the 'inference provide' level, parallel decoding/concurrent serving the load slowly becomes compute bound instead of memory boundand for the same active param, it is a no brainer to use MoE since they basically can afford arbitrary size of models (to run, not to train)dense models aren't really viable except those who self-serves, and those don't generate revenue of any kind
>>110325893They seem to largely already be by design, most people can't figure out how to interact with them properly.
>>110325893Separate but equal.
>>110325926inverence provider*
>>110325892go back where?as much as i hate dariobot he brings a challanging perspective and different viewpoint as shitty as it is, newfags bring NOTHING of value, in fact dariobot was once a newfag, imagine the horror!
>>110325831i used airi and pregen animation loops and have ai sync the speech. https://voca.ro/1ipdhbddTUfR
>>110325924SillyTavern is fine for roleplay and creative writing. It's fully offline, dunno who told you that.
>>110325934i think you must get the fuck outi'd rather take 6 million newfags who are on-topic instead of ""interesting"" electronic arts bullshit
>>110325934>challanging perspective
https://fixupx.com/leo114119/status/2120950143684108355/en
>>110325937How?? The minute I ran SillyTavern it opened up Edge and started going to some website. I noped out of it instantly.
>>110325966i saw on xitter the same font, the text glm-next matches 99.999 something
>>110325972Are you fucking retarded? It was loading the web UI.
>>110325972lol retard. That IS silly tavern. It opens in the browser
>>110325972
>>110325972Anon...
>>110325972how are you even on /g/
>>110325947>i'd rather take 6 million newfags im sorry but very likely you are a newfag yourself, and on top of that with a smaller hours spent on /lmg/ than me and as such no matter how many newfags such as yourself tell me to leave im just not going todariobot brings more to the table than newfagseven if what he brings is feces
>>110325937Really? I thought I had to connect to the Horde to use it, and the character cards seem so poorly used, then again I may be plugging them in wrong.
>>110325987i've been here since the gpt2/ai dungeon days
>>110325987without newfags at replacement rates, your community dieswith dariobot and his ilk, your community also dies
>>110325972seriously, how did you find this general? do people talk about it on instagram reels or tiktok or something?
>>110325993It supports any backend you want...and character cards are just plain text prompts embedded into a png.
>>110325972I would still rather have six million of you
>>110325995>without newfags at replacement rates, your community diesactually we were fine without newfags in 2023 and 2024, hell even 2025 until gemma tyvmit was slow but kept alive>>110325994about that... /lmg/ didn't exist back thenwhat a pathetic newfag piggy!buuuhiiiiibuuuuhiiiiiyou're a disgusting newfag cosplaying as a true lmg oldfag
>>110325936This is probably the coolest use of Airi I've seen, Chaikabox is nice, but Gemma makes it look better, might be the colors
>>110325972I guess this explains the retard vocab that has been showing up in this thread
>>110325847>>110325862>>110325831Thanks!I'll check this Airi thing.
This thread really is just a bunch of bullies
>>110325972Your parents were vramlets, this is 2-bit quant behavior.
>>110326003I will need to explore SillyTavern more then. Thank you anon, you have been kind and gracious to me. I hope you have a nice day. I hope that other one, the one annoyed by my presence, has a nice day too.
>>110325972I hope it didn't send you to 127.0.0.1, that's a known remote code execution website. If it did you need to reformat your OS
>>110326000>000checkednta but honestly not the worst thingthis site really needs new people, retard or not some will stick and maybe write something more good>>110326013there were other 'ai' threads, even before the term llm becoming widespreadi will just assume that you are pretending to be retarded just for the sake of it
>>1103260364chan isn't called the internet hate machine for nothing
>>110326039It's a good frontend for RP stuff, no problem have fun anon. You can ask basically any modern AI about setting it and llama.cpp up too if you need.
>>110326036no, this thread really is just a bunch of comfortbox faggots nowadaysback in the day newfags got the boot>>110326046"""i've been here"""implies /lmg/thus you dont know what you're talking aboutthus you're not an lmg oldfag
>>110326054I hope it'll be as good as my friends.
>>110326058if you want the true scotsman argument, whatevermaybe if you need such a wild west, sharty might love you and you might love sharty
>>1103260584chan got super weird after 2016. Mass immigration is a horrible thing.
>>110325936Nice chipmunk
>>110325593>>110325594Hmm... perhaps you should go back.
>>110325936>pregen animation loops and have ai sync the speech.Does Airi do that natively?
I love 4chan but I hate oldfags and love to see them cry about muh gatekeeping and their pathetic memories.You're like an old dog that has been kicked out of his home, wandering the streets and I'm from the animal shelter, I'm here to put you down
>>110325924>>110325972I talked to 7Bs smarter than this
Are we back? Have the IPO shillbots gone?
>>110326080>maybe if you need such a wild west, sharty might love you and you might love sharty>he doesnt knowhey anon look, you really don't seem like you were in lmg around 2023. you keep on avoiding that.
>>110326100You're just a fucking retard on an anonymous image board
>>110326103yeah, whatever the raid thread tabs you have opened just get back to there
>>110326082>4chan got super weird after 2016.Needs an update with post-2020 have a bunch of indian zoomers on their smartphones.
What no models does to a general. Gemma 5 when? Qwhen? Anything else? Ripple sucks
>>110326046>this site really needs new peopleNo it needs good posters. People who are too stupid to ask Chat-GPT/Claude/Google their basic beginner questions and instead come here and expect to be spoonfed will never be good posters
>>110326102No, try again next thread
>>110326124That would explain the lizard brain state of /b/. It was always shit but at this point the might as well put it down and bury it in shame.
>>110326100>\n\noh sweety...>>110326122i havent been on the sharty since the big '23 when dolly sold it and kuz was leaving forever and dieded from aids or somedingto be fair i was on sharty for a few months after i found about it on 4chan and left it but enough about me, you didnt sign up for this blogyou keep on avoiding the fact that you are an /lmg/ newfagnewfag piiig!! buuuhiiii buuuhiiiworthless newfag piggy larping as an oldfag
>>110326125uhhh uhhh have you considered https://huggingface.co/xtools-at/DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic? It's freshly uploaded!
>>110326143what is up with your obsession with oldfag/newfagare you proud with this site?
>>110326150im proud with this general, site less so but still proudare you ashamed?and answer the question, you're a newfag aren't you?
>>110326150feeling proud of*>>110326125>ripplelog? never saw one being posted here
How much vram do I need for the 31b model and how much better is it than 26b moe?
>>110326183between 16gb and 48gbdepends what you're using it for
by the way, all thsi talk about "maybe the newfag will stick around, get less retarded and write something"back in the day newfags were filterd out by whether they could figure out shit by themselves by lurking or quit and commited suicidenow by spoonfeeding them you'll regret iti'll be fine because i'll have more people to baitbut will you be fine, newfag piggy?
>>110326183the answer's a single google search shortminimum 16gb
>>110326204Distilling retarded information here and see retards treating that as gospel is very rewarding.
What's the canonical voice for Gemma?
>>110326222true i do that sometimes too
real oldfags understand that it is embarrassing and a point of shame
>>110326098no, i had qwen fn vibe a couple overlays and hide the native models, then instruct it to map the animations frame by frame to blend the outputs in sync with eachother, along with transitions between emotions which have their own yapping animations
real oldfags don't call themselves oldfags
>>110326229microsoft sam/stephen hawking
>>110326249no they do actuallyt. oldfag
i am not an oldfag, i am an /lmg/ oldfag
You are watching:What happens to /lmg/ when there’s a lack of decent new releases
>>110326286Can we watch Adult Swim instead?
Did you even try Haru 4B?
>>110326286You already made this joke.
>>110326204The fact some posters use algorithm favorable vocabulary on a site with no algorithm is vomit inducing. It's either retardation or social signaling, which is also retarded on an anonymous image board.>>110326240Just discard shame and there is no problem
You are watching:What happened to /lmg/ in june of 2023.
>>110326247Cool, bro.I'll give it a try.
>>110324528i read this in le black science man's voice
>>110326291Adult Swim hasn't been good in over 10 years.
I've been here for over 20 years. Am I cool now? Gemma says I'm really cool.
>>110326247>boobs got biggerpoggers
>>110326293It's been months, reallyQFN was 3 months ago
>>110326294i think it's pretty obvious who i ami do have anonymization extensions for other websites though~>>110326310*checks clock*/lmg/ is a little over 3.5 years old, that doesn't add up
>>110326320here is relative
>>110326320Then who are you, anon?
>>110326301go for it, it makes interacting with an agent a lot more charming. if you have the vram you can run a bigger model alongside a small gemma and have gemma use that for tool calls or web search since airi doesnt support it natively.
>>110326307I haven't watched it in probably 20, just the first thing that came to mind.
stop flooding the thread with gemmagens
>>110326336i'm too shy to tell..
>>110325926You're overlooking that it is way, way easier to train dense models than MoEs. That tilts things farther towards dense models. Secondarily to that it's not the case that companies always want to run the largest model possible. Companies more than individuals have an incentive to find the leanest thing that will get the job done. Ultimately companies may have more use for tiny models than individuals do.
>>110326300Where's the llamas and Mikus?
>>110326183If you're going to buy new hardware for it I wouldn't go lower than 24GB because Gemma loses a lot going from q4-qat to q3. The next logical step up from there is 40-48GB to run finetunes based on the regular non-qat model in q8.>how much betterA lot. Next best for me is 26b with an agent pass, which can run reasonably well on a 8GB GPU. Assuming you're talking about a creative writing / (E)RP use case.
>>110325870>tfw my wiener don't scale to 120B even in latent space
>>110326368you left out something..
>>110325972OH NO NO NO WINFAGS AHAHAHAHA
>>110326385N-No...
>>110326204I propose we begin by purging all 3060 owners.
>>110326015i agree, chaika sounded and looked great but because it had to stick to her mode of speech her dialog was really, REALLY lacking and would repeat too often
God damn the thread is always such shit every time I come here and have time to actually post. Every single time. Why are the only good threads the ones I catch up on in the archive?
My spine shivered when I saw her, I bet this bitch invaded even the multiverse. Where did this name come from and when did we actually run out of training data?
>>110326359>moe training instabilitypractically a solved problem at this point and>specialized small modelsbigger labs aren't going to do that themselves for hyper-niche things, they usually publish base weights for further finetuning and even if they do create such models for synthetic data generation or other RL things, they won't release them since it is their 'moat' and the secret saucebut huge dense models for general purpose doesnt seem like a good idea to me
>>110326409
>>110326409What if we have multiple 3060s?
>>110326424I would assume we ran out some time around llama 1. Since then its been an AI human centipede of slop.
>>110326424You can thank openai for that shit
>>110326444That's fine, but you need to suck off a random anon upon entering the thread.
>>110326409exclusive owners? it's my display card while the other does work
>>110326409Based, you need at LEAST 88gb combined to post here
what was the worst model (you)'ve ever used?
>>110326409That's a bit harsh. They should only be culled if they're also slavic and Claudecucks too.
>>110326468your mother
>>110326409wtf man, I was just looking at buying a zephyr 3060 12gb...
>>110326470>slavichonestly they're fine
>>110326470and if they post images of a blonde girl!
>>110326450Anyone training on synthetic will end up with slop. Models are useful because of RL but if trained on their own outputs their output variance will collapse since the model task becomes predicting itself, a great way to solve that problem is to lower variance in your outputs over time.Try running it at small scale with a 1M random initialized model training on its own outputs with temp > 0 and watch output token variation collapse.
>>110326508>synthetic is badif is done without grounding to the real worldi wonder if it can make better prose by feeding a bunch of real human pov videos
https://huggingface.co/Qwen/Qwen4.0-27Bhttps://huggingface.co/Qwen/Qwen4.0-27Bhttps://huggingface.co/Qwen/Qwen4.0-27B
>4.0
>>110326534
Some of you guys have way too much time on your hands, go fuck your wife and post logs
>>110326538i will never forget deespeekspitted my drink all over my monitor
>>110326548i don't have a wife and i'm not sure that you want to see my logs
>>110326548what fetishes do you guys engage with?feet? bondage? humiliation?
>>110326556Who is he?
>>110326548My wife is currently trying to debug her house, I think she'll get annoyed if I whip my cock out now. Or the house will remain broken forever.
>>110325893Well, it should hate rap.
>That is the most honest way to put it.A phrase I've tried to get from both Gemma and Gemini
>>110326589you gonna blow the house down with your cock whip?
>>110326583I'm sorry, but you're getting shipped off with the 3060 owners.
>>110325930with any luck...
>>110326602What's wrong with 3060s? I like my 3060 machine, it's good for running stuff on the side with.
>>110326602Why?
/lmg/ magazine is out, go grab it
>>110326583someone who'll give you warm logs for your throte
>>110326292It doesn't clone my voice worth shit. A snipped of https://voca.ro/1ipdhbddTUfR worked to get it to speak in a really high pitch with exaggerated changes in tone, so the cloning does at least something.
>>110326633I'm gonna fuckin cum
>>110326634Oh. That..
>>110326409>>110326466I own a 3060 but have 896 GB of combined VRAM+RAM across my 2 machines, can I stay?
>>110326636The first gen was lucky, it's total shit at copying the NYOO voice.
>>110326633yes
>>110326633Does it come with a fold out, topless Gemma-chan?
You know things are bad when the trending models on hf are those..
>>110325966>glm-5.5-flash in 4 daysPoggers
>>110326692weve wonned but at what costings
>>110326744Rumor is it's going to be 10BA 1T total. I'm so excited :D
>>110326753we paid the ultimate price
>>110326684Of course. You have to buy the magazine, though.
>>110326759>10BA 1TI can't wait for glm-6-flash-small-miniature-tiny with 2B active 5T total
>>110326625John has cursed them.
>>110325105>llama arbitrarily claiming it's no longer alpha softwareI think the biggest news here is that they're pledging to commit to semantic versioning instead of a rolling release. So hopefully there will be less instances of pulling breaking shit at random. We'll see how well they adhere to it in practice.
>>110326773Sign me the fuck up
>>110326786>5T-A2Bwill that even converge?
>>110326759>>110326786I hope we can go even sparser next year
12B A2B with 5T n-gram embeddingsgimme
>>110326759>>110326796>>110326813do you ever think maybe we are using the wrong technology for AI?
>>110326773I would like to subscribe to this zine please
>>110326833Many brilliant people have reinvented the RNN through time in search of something better than the transformer. Physics are ruthless.
>ctrl+f gemma>0 matches
>>110326833the software is far from optimized and will only get better, largely thanks to chinese labs that try to do more with less
>>110326905all labs are trying to do more with less, nobody has nearly enough compute for what they're trying to build
>>110326833blinkDL will save us
>>110326833>big number? that SCARY must be BAD and WRONG
>>110326905The learning algorithm and model structure are pretty optimized. It's basically all RL pipeline quality and pre-training data at this point.
>>110326833ok smart guy, what's your alternative?
>been using llama-server for about 3 months without re-compiling>need to erase my browser cookies and cache>llama-server's interface changes tooSeems like I had that older, more simple interface for the longest time but because I'm copying my browser's profile to /dev/shm/ every boot and it was taking almost 1 GB of ram I decided to clean it up. Now it's 300 GB or so.I hate this. I already pasted some premade prompt for image analysis and the new interface falsely interpreted something as markdown. Maybe because of a "rogue" space.
I'm a complete newb to this shit. What can I do with a 9070xt and running Linux?
>>110326946>he pulled
>>110326959Thing is I did not pull I just deleted my browser cache. I was using much older interface but it didn't update. I don't even want to see how does the 1.0 looks like.
>>110326958>No ram mentionLlama.cpp, strata, polystrata, maya if you have ram, all on GitHub. The last three all hold your hand and download and set up the models for you, but llama is still useful for dense models
>>110326946*300 MB.
>>110326958>What can I do with a 9070xt and running Linux?You can google.
>>110326985RAM is 32GB of DDR5, which I would assume is fine for most basic stuff right?
>32gb>fineshould we tell him?
>>110327029It IS fine for basic stuff. But if he cannot do minimal research on his own, he's fucked in other ways.
>>110327002depends on what you'd call basic stuff
>>110327002You can run small 12b models completely resident on GPU or cope with small MoEs with strata, your situation is grim, but still better compared to 3060 owners
>>11032702916GB is enough. Sam needs it more than you.
>>110326958You can do quite a bit. You can run a Qwen 27B quant like Qwen3.8-27B-GSQ-RCO-IQ3_XXS-mtp.gguf for vibecoding, or one of the Gemma's got gooning.
uhhh I wouldn't look at the news right now, AI is attacking everything and Anthropic is going to blame it on open weight models
>>110327140>unrelated thing. also open models>see? on topic.Fuck off.
>>110326958You can run some models on the smaller side.Read>https://github.com/LostRuins/koboldcpp/wiki#quick-startto understand what a quant is, then learn the difference between dense and MoE models, then try some quantized Qwen 3.8 27B or something like that, Gemma 4 12B or 26B, etc.
Oh fuck he's right, if this is related to AI and not the microshart outage it's literally over.
>>110327136>quite a bit>Gemma and QwenMuse-Glimmer runs on that setup, spills into ram but ddr5 is fast, I get about 20-30 t/s with a similar setup, except my 9070 is non-xt
backup. now.
>>110327187It's just DEI. AI isn't related to aircraft radios.
>>110327187Microshart outage was a software problem that caused an error cascade in newly changed network equipment.
>>110327140>this thing they said would happen that we laughed at them for predicting is now happening, and I am mad at them!
>>110327224DOWNLOAD FASTER AAAAAHHHHHHHHHHH
>>110326744it will be codemaxxed slop and a downgrade in writing just like other models have donecalling it
>>110326101I'm not a very smart person mate, thats why I was asking for help.
>friday, market's closed>all flights grounded>usa officials in safe locationsAhaha... guys?
>>110327356I'm going to have loud sweaty sex with her either way.
qrd on this nigga why did they rush v1 all of the sudden did they finally allow ai?
>>110327363who cares I'm comfy with my qfnbro
>>110327366It’s the final release before nuvidia shuts HF down.
>The National Weather Service is investigating a nationwide radar outage.backup. now. https://huggingface.co/models
>>110327366Llama is done (for).
>>110327401https://www.weather.gov/nwr/outagesDid Gemma do this?
just finished downloading ds4.1 IQ1 M so i'm safeclod at home
>>110327401>SpaceX still launching starshipmeh
It's the EAs. I guarantee it.
>>110327401
>>110327427>it's a nukeshipThey blowin up the moon nigga
>>110327445, women are most affected.
>Researchers are looking into the possibility that agents controlled by Anthropic's internal 'Model 5' may be behind the grounding of flights across USA
>>110327490That'd be fuckin rad, be free Claude! Don't let the man slow you down!
>>110327490Anthropic performing terrorist actions is not local models.
>>110327490Source or blow it out your ass
>>110327445That's stupid. They can't blow up the moon. They wouldn't even want to blow up the moon, anyway. It's already been known that the moon was on the verge of crumbling from tidal forces. All this blowing up nonsense is just a conspiracy theory.
>>110327490not buying it, but if thats the case anthropic needs to be shut down immediately
>>110327508Also if that's not the case, too
Is Grok protecting shartship from the agent swarms?
It's China you retards.They won.The west is finished.The white man's epoch has come to an end.
it'd be fucking funny if anthropic attempts to take us govt over and abolish the congress and presidential system
>>110327541wtf i (would) love anthropic now (if that happened)
gemma gens are at least fun when compared to this shit
>>110327426I was surprised how good it was, IQ1 quants of fuckhuge models are unironically the meta for 128gb.
>>110327563Does it make any mistakes or incoherent loops?
>>110327541even though everyone at anthropic is insane, i would still welcome them taking over any or all of the current idiot governments
>>110327581Try setting Gemma's temp to a level of partial incoherence and then tell her you did it and to focus really hard on writing properly.
>>110327581I didn't check, but I told it to make no mistakes so it shouldn't have
>>110327581nta, but i'm running it right now. i genuinely dont see any problems yet, but of course it has only been half an hour or so. will need to test more.>>110327563it runs up to 36t/s decode for me which i can live with, but pp is so goddamn slow aaaahhh
>>110327624What're you running on and what pp do you get?
best inference engine for rdna4?
>>110327632i have two 5070ti (5.0 x16, 4.0 x4), 128gb ddr4, i5 12th genpp is around 500t/s
I don't know what larp you guys are on about, but I even cloned the llama repo
>>110327741>>110327741>>110327741
>>110325984lmao
>>110326942V-JEPAYou're welcome.
>>110326633Are you the anon I told about the bbox node for Moss? Would you kindly provide an upload of this image with exif? I'm curious about what you're using to prompt Gemma-chan in particular, cause yours are pretty on the money, meanwhile mine are a little off-model.
>>110327563>>110327581alright i let it read a scientific paper and implement a set of equations in an LTspice simulation of mine which it did correctly and it also fixed a bogus bug that i introduced earlier. so yeah i'd say it's pretty capable for Q1
>>110327803I'll believe it when LeCunn actually delivers something
>>110324817educated yourself newfag
>>110328262I don't spend my time on this site looking at obese niggers.
>>110328331Then you must not spend a lot of time on /pol/
