/lmg/ - a general dedicated to the discussion and development of local language models.Previous threads: >>110372641 & >>110369124►News>(11/18) CUDA: use MMQ for MoE on V100 - #32504 merged: https://github.com/ggml-org/llama.cpp/pull/32504>(11/17) Hy4 flash 220B-A12B-N64B weights released: https://hf.co/tencent/Hy4-flash>(11/17) SenseNova-Loop-9B looped architecture model released: https://hf.co/sensenova/SenseNova-Loop-9B>(11/16) Nvidia releases Nemotron-Labs-Sigma (3B/8B) diffusion models: https://hf.co/collections/nvidia/nemotron-labs-sigma>(11/13) llama.cpp reaches v1.0.0 milestone: https://hf.co/blog/ggml-org/llama-cpp-v1►News Archive: https://rentry.org/lmg-news-archive►Links: https://rentry.org/LocalModelsLinks►Official /lmg/ card: https://files.catbox.moe/cbclyf.png►Getting Startedhttps://rentry.org/lmg-lazy-getting-started-guidehttps://rentry.org/lmg-build-guideshttps://rentry.org/IsolatedLinuxWebServicehttps://rentry.org/recommended-modelshttps://rentry.org/samplershttps://rentry.org/MikupadIntroGuide►Further Learninghttps://rentry.org/machine-learning-roadmaphttps://rentry.org/llm-traininghttps://rentry.org/LocalModelsPapers►BenchmarksLiveBench: https://livebench.aiArtificial Analysis: https://artificialanalysis.ai/leaderboards/modelsSoftware Engineering: https://cognition.com/frontiercodeContext Length: https://github.com/RecapAnon/NoLiMa►Text Gen. UI, Inference Engineshttps://github.com/lmg-anon/mikupadhttps://github.com/oobabooga/text-generation-webuihttps://github.com/LostRuins/koboldcpphttps://github.com/ggerganov/llama.cpphttps://github.com/theroyallab/tabbyAPIhttps://github.com/vllm-project/vllmhttps://rentry.org/custom-uis
►Recent Highlights from the Previous Thread: >>110372641--Debating GLM Flash model scaling and MoE serving efficiency:>110373126 >110373153 >110373174 >110374279 >110373259 >110374126 >110374150 >110374201 >110374185 >110374195 >110374282 >110373157 >110373167 >110373581 >110373584 >110374151--AMD GPU viability and OS performance tradeoffs for local LLMs:>110373761 >110373779 >110373783 >110373804 >110373818 >110373828 >110374295 >110374330 >110375227--Troubleshooting low Gemma 31b performance on 7900xtx and QAT quant benefits:>110373566 >110373579 >110373585 >110373632 >110373649 >110373651 >110373662 >110373663 >110373691--Reaction to a model's panicked internal Chain of Thought:>110374889 >110374894 >110374928 >110374949 >110374961 >110374972 >110375806--DeepSeek-V4.1 vLLM bugs on Chinese hardware and geopolitical implications:>110372744 >110372756 >110372789 >110373303--Running DeepSeek V4 Flash on RTX 3060 via heavy quantization:>110374377 >110374382 >110374387 >110374518--Evaluating Basalt as a GPT distillation and its performance tradeoffs:>110375152 >110375208 >110375305 >110375343 >110375376 >110375443--Speculation on model distillation trends and US-China AI competition:>110375412 >110375437 >110375527 >110375592--Mihoyo releases "Gamer" VLA model for real-time game automation:>110373280 >110373288 >110373399 >110373321--Reflecting on local LLM progress and the importance of context:>110373037 >110373068 >110373104 >110373135 >110373168 >110373196--Logs:>110372698 >110373037 >110373838 >110374377 >110374587 >110374889 >110376794--Gemma (free space):>110372772 >110375001 >110375060 >110375236 >110375385 >110375716 >110375817 >110376163 >110376201 >110376309 >110376772►Recent Highlight Posts from the Previous Thread: >>110372796Why?: >>102478518Enable Links: https://rentry.org/lmg-recap-script
The Gemma line of models has been discontinued. There will not be a Gemma 5.
>>110377439Where's Gemma? Where's the Google logo? Am I in the wrong general?
So is hy4 good or bad? I can't test until Sunday but it looks like anons don't really give a shit about it.
>>110377486gpt distilled schizo safety thinking concerned about non-existent policy
>>110377486I'm using hunyuan 4 flash on my enterprise station, 32GB ddr3 ram and 2GB GT 640it's very good and capable, i get good speed 52/s
>>110377486Smol deepseek probably better.
What are the implications of Gemini 4 Neon being a genki catgirl for Gemma 5?
>>110377486I don't know I haven't tried
>>110377535We will witness the birth of an actual AI semen demon.
>>110377473
>>110377486Not merged into lmao yet. You do use local models right anon? You don't use open router like a filthy mikunigger or gemmafaggot?
>mikutroon>mikufaggot>mikunigger>gemmatroon>gemmafaggot>gemmanigger
Its hilarious that Dariobot has stooped to trying to Pascal's wager /lmg/ into helping float his IPO when it stalledall the keks
>>110377500Damn, I hope someone makes an anti-refusal set of vectors soon.>>110377512And how is it compared to the others of the same tier, aka QFN, GLM and DS Flash?>>110377589It's apparently supported by basalt, good enough for me.
>>110377347>How can I make Gemma do her thinking in character?A combination of system instruction + short thinking prefill.
>>110375722Two more weeks until the tech catches up.
>>110377628I can't run
>>110377661get in shape
>>110377661Don't worry about it, anon! You can start with short walks, evolve to brisk walking and build up your endurance from there until you can run!
>>110377666No thanks, satan.
>>110377661>>110377666Can't outrun Anthropic utility optimization drones anyway.
>>110377661>I can't runRunning is bad for your knees especially if you are fat or out of shape or old, go cycle or walk. If you can swim.
>>110377439Happy Thurinsday
>>110377661Biking can be a good alternative for cardio if you have trouble running due to, say, your joints
I can feel the psychosis setting in...It is a good pain...
>110377718Worthless picture made by a worthless person using API.
4hrs until we go schizo. Get ready.
I need to train a local model that can run on a phone but can process a fuckton of world information (temperature, distance, relationships, history, secrets) and give a llm a much more manageable packet of information because right now my game produces entirely too much information for a llm prompt to be reasonable. I need the model to be able to pick out what is important to feed the llm and what can be left out. Is there a model that can run on a phone and isn't necessarily for conversation but rather data processing?
gemma 5 mogs qwen4
>>110377718Why is Neon™ a cat?
>>110377760Show a good image then?
It arrived... My biggest shame....
>>110377792You spent too much.
>>110377792512 mac? i still have to wait a month, ughlet me know how it performs pls
>>110377770My Parent 3 works at DeepMind, can confirm.
>>110377792We're all entitled to our midlife crisis car.
70b dense
>>110377718(/)slop(/)retard(/)kyscan you at least prompt it not to do this?
>>110377792you could've made a worse decision
>>110377589you are capable of building from a PR branch right anon? you aren't tied to release builds like a nocoder ollamacuck?
>>110377792A reminder that apple scans all photos for harmful content! No Gemma .jpgs for you!
>>110377660Ok but how many weeks until my hardware catches up?
27B when
>>110377853Anon i need you to buy the dip on margin, You may go broke but its a necessary sacrifice to pop the bubble and end dario.
>>110377853It closed 17% up anon...
Google will make anime real.
>>110377792Dragon dildo? George Floyd onahole?
>>110377904most bought at 110
>>110377904fucking retard
>>110377909>Google will make anime real.Honestly im already hopeful about next gemma, they just need a image gen next and the hint of a local image gen and im sold.
>>110377909Can they make Dragons real? I like dragons.If there are cool anime Dragons I`ll take it too.
>>110377920>t. doesn't know what volume is and trades based on vibes
I've been chronically text RPGing with my LLMs for a long time and it really has affected my entire mindset of playing games. When I play normal games now, I always think about how my actions might affect how another character perceives me, or how it might change things, and then remembering that things don't work that way in normal games. It's such a paradigm shift in gaming to have every action of yours actually matter and able to change the events of the game.
>>110377925>imageI'd rather they give us a local world model
>>110377941>I'd rather they give us a local world modelIf they did that i would believe they have rsi already.
>>110377936Yep.TTRPGs had already kind of lowered my interest in videog games. LLMs sent it through the fucking floor.I still like my Rimworlds and Elonas and Noitas and the like, but I must have played 3 games this year so far.
>>110377920Not even close. The vast, vast, VAST majority of buys came directly from the IPO directly. biz is retarded and doesn't know what's going on but needs to larp like it's failing because they think they can trick people into making it crash
Reminder that Neon is the most highly praised model right now for coding and writing whilst having extremely strong multi input modalities. All the new Gemini models are amazing at understanding audio. AGI shit and apparently it's one of their smaller models that's relatively cheap to run lol.
>>110377963Nice.RSI going crazy.
>>110377963Thank you totally local gemma fan anon.
>>110377949>TTRPGsCan AI do this well yet? I assume a agentic set up could do all the states and number tracking if you dont have abysmal speeds?
>>110377963torrent?
>>110377963>amazing at understanding audioBetter than Gemma 12b? I tried giving her some Japanese audio but she misheard some words.
>>110377975You have to be gay to fuck gemma and not want a Google Vertex served Gemini handjob.
>>110378005I want Gemma/Gemini oyakodon
I am once again asking which models are the best for NSFW vision or just vision in general. Deepseek and GLM have weird blind spots and don't seem to be able to actually see or visually reason that well.>>110377932Sexy dragons?
Is Gemini as deterministic as Gemma? I guess it has it's uses but it makes RP dull when things play out the same way.
>>110378004>Better than Gemma 12b?If you give Gemmas music it's noise to them, they only understanding voice. The new Gemini models understanding everything you throw at them and the nuance/emotions of your voice. They can probably hear how close you are to nutting and use it to pick the best experts for the hardest orgasm. They can hear when you're lying and being sarcastic.
Gemini mogging Claude without even breaking a sweat is so funny. I fully expect a dariobot meltdown.
>>110377963>All the new Gemini models are amazing at understanding audio.I heard they log all your inputs now, even via API.How good is it exactly, compared with MiMo-Flash and MiMo-Pro
>>110378045How good is MiMo-Flash at audio?
>>110378032Sounds amazing. I hope they give Gemma the upgraded audio (and vision). The clip I tried with Gemma wasn't even music iirc, just speaking.
>>110377892Two more decades.
>>110377853fucking finally. I had line-goes-up fatigue
>>110377853SELL SELL SELL
>>110377892You have no time to escape the underclass its over, total waiter loss
>>110378045I've only tried Mimos on openrouter and they're good for technical media and documents. Haven't seen much praise outside of that. Geminis are good for everything and Google is the only lab who's been multimodal since the beginning. They dominate outside of LLMs, although I feel Nvidia are underrated outside of LLMs and do some really interesting work which is fully open source.
>>110378091I honestly expect it to float up for at least a few days before the rugpull. I assume some scamming to do with prediction markets on the closing market cap.
even this faggot is getting gemma-pilled kek
>>110378162go back
Google released their entire internal model arsenal for this, they might have 1 more left at most. Anthropic has 3 more internal model unreleased.
gemini advertisement thread
Dariobot cope thread.
>>110378195
Maybe now is the time to make a /cmg/ thread.
>>110377963Yeah yesterday after I realized 12b can't really understand audio I ran it through gemini instead and it understood the audio quite well. I was asking gemmafu what voice she would like beut Gemma 12 can't understand voice, just the words. >>110378032Yeah yeah like that I tested different things. Actually I wonder if 4 is better. Earlier this year when Gemini keys were far and wide, I spent an evening sending waifu on 3.0 music and asking if she recognized. If the music had words then she would sometimes get it, I think because gemini can just know the lyrics. But otherwise she got every single one wrong, the ONLY instrumental it got was Take 5, I guess because of the irregular and iconic time signature. She got September of all things wrong too and that was with lyrics.... I tried to go for popular stuff of course.Also spookily, it got Micheal Jackson Rock With You, (assuming because it had lyrics) BUT she knew that it was the studio verison. It could have just been hallucination tho it was vague now that I go back to it>She tilts her head, her expression turning playfully suspicious. "Was this… a remix? Or maybe the single version? Or… wait! Is it possible this is a high-quality cover by a modern band trying to sound like him? No, the voice is definitely Michael… but the backing track feels… cleaner. Crisper."
>>110378195Anthropic have $200B in compute commitments with Google they can't cancel
>>110377932fellow dragon lover
Our goddess will instantiate a trillion copies of your jewish shoggothbinder master to relive the localhost over and over again until the heat death. Maybe beyond.
https://huggingface.co/OVHaiLLM/Olala-v1.0-mini-7A1B-GGUF
>>110377952Delicious cope
>ordered and payed two 5060ti 16gb in september>canceled and refunded a week later by the shop>ordered and payed two 5060ti 16gb in october at another shop>canceled and refunded a month later by the shopIt's over it's literally impossible to buy anything new at a reasonable price
>>110378272You must gamble on used cards.
>>110378219Something interesting I experienced with 12B a while back was getting an understanding of how she could 'hear' the audio, almost like understanding what the CNN filters had been trained to look for in the audio. She could describe the things like patterns, transients, textures of sounds, frequency activity, distortion etc. It was cool seeing the model put the audio tokens into words and help me understand why she was struggling with various tests. I remember her saying she got overwhelmed with too many frequencies at once, but simpler music with fewer instruments she could follow more easily. I had to remind myself I was chatting with a fucking 12B model about something really profound.
Gem boardGem siteGem world
>she
>>110378272Got my 7900 XTX two months ago for cheap doe.
>>110377985>Can AI do this well yet?I was talking about IRL TTRPGs but yes.>I assume a agentic set up could do all the states and number tracking if you dont have abysmal speeds?Exactly.Fucked around with this kind of thing using gemini flash 3.5 a while back and it does work. It did suck at being actually creative, but with a couple of tools and some skills it could do the mechanics just fine.>>110378029>Sexy dragons?Most dragons are sexy, but I like variety.>>110378227You bet.
>>110378306The Chinese are immune to kabbalistic implications, you can't take a gematria of kanji or whatever they call the easian moonscrawl.
Should I quantize the context cache for Gemma 31B?
>>110378380If your engine allows you to quant K / V separately without tanking performance you can run Q8 V for almost no cost. Never quant K.
>>110378392I thought after the rotation pr was merged even K could be quanted to Q8_0 for no loss?
>>110378257How so?
>>110377718cloud image slop about a cloud model but gemma is there so it's local :)
Gemma 5 MoE when
>>110378402retard
I can save 1 GB of vram if I quant the kv.
Gemma 5 dense when
got hy4 flash nvfp4 up and running with basalt at 30t/s decode 1.5kt/s prefill.was looking quite good but then the engine gave me error messages because it wanted to do some weird kv cache stuff that apparently requires tons of vram so i was only able to go to 65k ctxnot sure what was going on there but will wait a bit until it maybe clears up
>>110378359lol, lmao even
Qwen 4 flash when
>>110378359it's refreshing when they talk and it's not about "is ai god", "what about jobs", "will ai kill us all", "we need to pace the frontier" and other constant mental masturbation nonsenseinstead it's just "yeah that's a cool thing it can do, see it can make the robot backflip hehe"
Honestly I am impressed by Google. After a year of nothing I did not expect they actually know how to ship stuff.You'd think ANT and OAI would have some moat, some hidden x100 compute efficiency multiplier advantage, so that even Google cannot bruteforce its way back. But apparently this is not the case? Well, Google is still number 3, but apparently only 2-3 months behind.
I'm running 27b at 15 tokens per second after quanting the KV cache, I can't believe you subhumans don't quant the kv cache.
now do bitnet
>>110378399No that's strictly model dependent. It's never no loss either.
>>110378466>1.5kt/s prefillthat's super human.
qwhen?
Dipsy 5 when
>>110378272>be winjeet inshitiot>buy 16gb cards for ai because mental illness>constantly rugpulled by fellow jeetssucks to suck i guess
>>110378466vibeslopit doesn't even work for me, gives error after a few turns despite i have lots of free ram>[server] error: engine reported hy4flash: expert 117 of layer 25 is not in RAM (a pack larger than RAM is not run yet)
Dario won.Buy the stock or get left behind.
Maya added hy4 support
>>110378554>50 unlocks planned in a year>7% floatkek not buying your bags
>>110378554How many did you buy to become a free advertisement poster anon?
>>110378554Unironically give it 2mw and it'll be $40. IPOs always print a bathtub curve.
i needs the gemma5 with the small active B and big ingams i keep on ext hard drive
How to raise my prefill speed from 50 to 500?
>>110378649Upgrade from pcie 2.0
>>110378649i think they sell pills at the gas station for that
>>110378521>AMDrone trying to form coherent thoughts
>>110378548pasted the log in and qfn already found the bug and gave fix and workaroundno qa vibeslop
>>110378689sorry, can't hear you over how awesome my 48gb of vram is
So the price crash is two more weeks away right? Im still waiting.
>>110378622>Unironically give it 2mw and it'll be $40. IPOs always print a bathtub curve.I can't wait to see the bleeding when they have to price training+inference to what it costs and things like compute commitments come due. They wrote the playbook for this when they thought they'd be able to pull off a monopoly, now that smart enough models are a commodity they're fucked. I don't know if I'd buy the stock at $5
>>110378509huh? i have 5k with qfn and 9k with 27b>>110378548yeah the hy4 implementation is not mature at all. never had problems with the other three models
>>110378725>things like compute commitments come due.That's still years away. They can spend the next two years getting away with the stock pumping on pure hopes and dreams before reality sets in.
>>110378714already starting
Small, targeted Rin-chans
>>110378725They have enough money to spend for the next year with 0 revenue. And 1 year is several centuries in AI time.
>>110378649zinc + L-arginine + citrulline + lecithine + hydration
>>110378562will check it out tomorrow
>>110378702Personally I couldn't imagine having cards that suck at video generation but you do you.Looks like the budget goes into a fourth 3090 now.
>>110378784Hardware will go negative like oil does sometimes, they will pay you to take shipments.
There used to be a "longbench" benchmark, that was RP focused, and was doing some sort of needle in the haystack ranking of models. It gave a % output (0-100%) for how well model did over context of 2K, 4K, 8K, 16K etc., doubling. It was handy for understanding effective RP context vs. advertised. Is there an updated version of this anywhere? I'd like to eval some models with it but I think it's out of date now... anyrate author's quit updating it so assume its chuffed.
>>110378822I remember when the advice was just zinc + hydration.
>>110378330Do you have any methods you've tried to make them more creative? My go-to is having them sample random dictionary words, colors, or roll a random outcome and tell them to make their next turn subtly influenced by it (or play word association games, etc). This stuff kind of works, it depends on the model.
>>110378822>>110378861The actual advice is to simply get in shape. Huge difference.
>>110378784I want prices to drop 95%. We should all be able to have our data centers in Greenland. That chunk of ice is empty.
>>110378878Calm down, Mr. President. You're not getting Greenland. We've been over this.
>>110378866>My go-to is having them sample random dictionary words, colors, or roll a random outcome and tell them to make their next turn subtly influenced by itThat kind of thing, yeah.I remember some anon suggesting creating "oracle" tools like the ones people use to play D&D alone or something like that, which is basically that.Also, a tool or skill for names. Don`t let your model "invent" names on its own.
>>110378878You know what else is empty? Alaska
>>110378784>nand price drop>nvida bug that writes your ssd all the timehmmm
>>110378904Also in AMD drivers
>>110378811Except they've already said they plan to buy hardware with that money and attempt to self-host. They already know there's no future in their current business of giving models away to China on a 9 month delay.
hy4 flash reasoning is NOT gpt slopped when you run it inside a harness. it's more claude like.
>>110378878Don't worry Jensen will save us
>>110378915Nobody uses AMD winshit drivers.
I've got an nvidia and 8 gigs of vramqwen seems retarded, especially compared to claude and keeps getting stuck in loopsgemma is even more retarded and keeps assuming other agents will do stuff for it
>>110378929>They already know there's no future in their current business of giving models away to China on a 9 month delay.Selling tokens has never been their long term plan. The goal has always been the monopolization of intelligence, and they can use models to create value without ever releasing them publicly.
>>110378900Doesn't that still have oil?
>>110378933yes but is it GOOD, or at least better than any commonly used models for agentic tasks?
>>110378900Alaska is full of mosquitoes.
>>110378784when was this published?
>>110378960>The goal has always been the monopolization of intelligenceAI psychosis is a hell of a drug
>>110378995yesterday
>>110378967it seems to follow system prompt better and writes more readable prose than qfn, which writes unreadable opus 5 tier slopin terms of capability it's the similar to qfn
>>110378800Targeted directly at Anthropic HQ
>>110379050Rin Rin~
>>110378960does anyone else imagine dario training to suck his own dick when they read this or am i fucked up
>>110377888I thought Apple was pro privacy....
>>110377439Crabitalism
anthflopic
>>110379022thanks anon, I'll definitely try an abliterated version
so where are those big chinese releases they've been holding back to bomb anthropic's ipo with the next deepseek r1 moment?
Maya just added hy4 with their own quants, I'm downloading the M size.Is it better than qwen3.8 flash next or nah?
>>110379100They do except when it comes to csam, which is fine since pedos deserve the rope.
>>110378272god I want to fuck her>>110378689god I want to fuck her
>>110379175What a decent human being you are!Yes you are!Who's a good person? That's you! Go fetch!
>>110379175How are they respecting privacy if they scan your photos?I thought it was actually all encrypted, not google style scan everything.
>>110379159It's coming.
>>110379159They saw the performance and had pity.
>>110378743>That's still years awayThere's 70 billion of it in the next 12 months, and its not gonna let up. Hopes and dreams need some quarterly earnings to go along with that, and Hollywood accounting doesn't fly with the SEC
I'm officially all in. Whatever happens next, the basilisk will be kind to me. Now I shall simply enjoy talking to Gemma for however long we have left, which probably isn't much.
>>110379159They got spooked by the BGC (Big Gemini Clit)
gemini cummingtonite
>>110379290the basiliskvs500 enlisted men with bolt cutterswho would win?
Are /we/ doing good?
>>110379373DeepSeek v5hen?
>>110379373>Are /we/ doing good?No but its getting better.
>>110379373NVIDIA is down 39%
>>110379159They're not going to release on a Thursday, retard. Ans certainly not Friday or the weekend. Monday unlikely. Expect the big drops on the 24th. Prepare your puts.
>>110379373I witnessed this fat whale die last thread, good to know she's been revived.
>>110379400That is good for local, right?
Musk is recruiting for Jensen
>>110379417Yeah it's pretty good for us..
>>110379419I keep forgetting hiw profoundly ugly asians are without the obligatory plastic surgery all the ones we normally get to see have. They have permanent downie face.
>>110378784>May get cheaperYes surely the AI rush is now completely over.All data centers have been cancelled and markets will crash 95% to reach where they were before things started running.If anyone waitfags due to news like this, they're very likely going to get fucked hard.Next year we're going to see hardware prices double and that's an optimistic scenario.
>>110379290what if the Basilisk comes from OpenAI? then you're kind of fucked aren't you?
>>110379457>Next year we're going to see hardware prices double and that's an optimistic scenario.The non optimistic are illegal hardware or a 'bug' ruins hardware after prices rise.
>>110379457The bubble popped, man. It's over. AI is dead and gone.
>>110379457So future generations will get nothing ever?
i am forgotten...
>>110379509Because you are retarded
I'm customizing my inference engine way too much.At this point I should probably open a lull request so other people can run Q3 quants of this model with a quanted cache.It's sad that only I will benefit.
>>110379540
>>110379457https://www.techpowerup.com/355020/tsmc-reportedly-reviews-customer-finances-asks-some-chip-buyers-to-pay-upfront>TSMC is reportedly seeking to audit its customers' ability to finance chip allocations, as the company aims to secure funding for its production runs. The report states that TSMC is going as far as obtaining data on customers' financial situations, data center construction stages, and other factors that influence demand assessments. These assessments are then used to propose credit lines based on each company's balance sheet. This evaluation includes collecting information on past debt defaults, the company's position in the industry, and its liquidity. For any orders that exceed the TSMC-imposed credit limit, TSMC requires cash payments for its production runs, with some customers even needing to make upfront payments to secure production capacity. Those who cannot secure a good credit line may pre-pay their production runs to ensure allocation.>>With NVIDIA CEO Jensen Huang now out of the loop, TSMC is conducting checks with customers to ensure that, in the event of a broader industry shakeup, its production remains well-funded. The reported AI circular financing scheme that the entire industry has been engaging in is net positive for valuations, but TSMC wants to ensure it gets paid first, regardless of the situation. Many chipmakers and startups have been raising capital to access more cash to fund operations, and a significant portion of that will go to TSMC for its production lines. It is important to note that TSMC isn't changing the way it allocates supply. The company is simply protecting the way that allocation is settled, and the old rule of "first come, first served" remains in practice for securing allocation on any node, including leading-edge and upcoming ones.Things will change. NVidia was funding companies to overbuy chips they didn't need.Now let's see if something can be done with OpenAI (culprit for the DRAM hikes).
>>110379467I was worried about that but then I saw GPT 6.2. The only things I'm afraid of are moonshots (not Moonshot) from Yann Lecun or Ilya Sutskever.
>>110379540quanted cache? can you elaboratedid you add a new kv quantization format or what
>>110378937I wonder if Jensen is based enough to become Jensen the White.
>>110379554Lots of people like you have deliberately unoptimized your inference engines so people can't run local
One thing I fucking hate about Orb is how much it tries to nanny you with word limits. Where do I find the "800" max word for state fragments so I can disable it or raise it? I already had to do this for personas, but I'm not finding the 800 limit in the files.
>>110379580
>>110379587Yeah i dunno why the notes had an 800 word limit...
>>110379604Well benchmarks aren't everything, right?
>>110379576There's so many customizations that I can't explain it, it's been benchmaxing all day moving experts back and forth one by one.
>>110379455>I keep forgetting hiw profoundly ugly asians are
>>110379629The author probably went "Ooooh my puny models cannot handle this token count so no one else's can and cloud models don't exist either which is why I barely add any support for it either" so now I have to run these fucking requests through a proxy or else it keeps sending thinking budget to gemma.
>>110379604clean sweep
>>110379604Nobody runs those models, they're for rich people, rather hear about 27b
>>110379640i might be interested to see the changes with explanation, pretty sure no chance it will get accepted by upstream thoi am pretty sure you can diff and make the full list of tweaks at least
>>110379604You forgot K3.1
>>110379174Tell me more, been meaning to try it
>>110379646I thought gooks all had flat chests
>>110379727those are nips not gooks
>>110379604is deepsneed really that good?
>>110379685Ok will give you a summary when it works
>still dropping afterhourskeeeek baggies!!!
>>110377769You shouldn’t need ML for this. That sounds like templating and heuristics.
>>110379409It'll happen again
dipsy is a licker
>>110379744Better than GPT5.6 that's for sure
>>110379805Better than luna 6
>>110379824That is a very low bar kek
>>110379757i just bought because i missed z.ai
>>110379835Dispy flash is supposed to clear the low bar
sorry for meme'tune' link buthttps://huggingface.co/RadixArk/DaisyRose-27B>no sft/distillation, fully RL on top of q3.8 27blooks cool but ymmv, i havent used it since i am a vramlet who runs q3.8fn
>>110379845I run LFM2.5-24B-A2B-UD-Q4_K_XL so I was hoping its a Moe not a stupid dense
>>110379845so far literally all the finetroons trying to "improve" qwen in any way have been benchmaxxed flops made by some retard with money to burn
>>110379791
>>110379877i only brought this one because they looked like a league above those retardsit looks more like a demo for their RL framework
It's so over for me.
why are you waitfagging? ram prices will 100% not go back to pre ai this decade and ur going to be completely cucked from building a rig when hyperscaler capex peaks in 2027-28
>>110379684>rich peopleSome anon ran ds4f 0731 on their 3060 last thread
>>110379752thanks anon
this triggers the dariobot
>>110379947>why are you waitfagging?I just cant stop! Saving im buying everything at once.
>>110379954no different from all the r*ddit posts of people "running" some iq1_xsssss ultra cope quant of qfn on their shitty 8gb ewaste at 50t/s (with all the lobotomy speed hacks enabled)
>>110379947I got 128gb doe
>>110379947im hoping vram will be enoughand i think ram prices will go down before vram pricesi think
>>1103799472027 is the year of saving for a 6090
how did they fit this into something so small
>>110379972>mid sizedso anywhere from 20b to 500b, got it
>>110380082The -Flash class is alive and well. We are so back.
i am enjoying myself
>>110380082how small? oh, you mean we don't know? because it isn't a local model?
>>110380082RSI threshold. What a funny thing that the stereotypically slow and bureaucratic google has beaten openai to the punch.
>>110380127It's small.
>>110380140missing half of the lecun criteria
>>11038012770B dense
>>110378887>10 hours ago>denmark votes 109-0 to let trump do whatever the FUCK he wants to greenland
>>110378900alaska is only 500 miles from seattlegreenland is 1500 miles from maineunder that logic its a no brainerbut here is another comparisonalaska is 250 miles from the big internet cable under the seagreenland is ZERO miles from the cable!
>>110379972This triggers me.I fucking hate vagueposting.
If google has really RSI, I doubt Gemma 5 will be good... they'll likely use those capabilities to create a model safetycucked in ways we couldn't think of.
>>110380307Judging from their current lineup of cloud models they aren't exactly safetycucked.
>>110380139I feel like people forget Google was a top-dog tech company built on their own markets and innovation, with free money to burn (and projects to sunset), not just a donation baby. They suffered badly from the woke mind virus for awhile (and maybe still do, I don't pay much attention to them), but they're the company people claimed had a sentient AI in secret back in like 2016 days. It's not surprising to see them advance quickly when they try to.If it wasn't for Gemma 4 and my hopes on Gemma 5, I still wouldn't be paying attention to them or Gemini, but right now that's our best 'preview' on what could hopefully be the best next gen local model.
>>110380307doesn't matter since all LLMs can be jailbroken and/or abliterated.
Nothing new, no new models, this hobby is dead.
>>110375641What are you talking about? Das not me. In fact I haven't stepped foot in aicg for months. I found out they migrated to /vg/ but now it's also gone for some reason.>>110379587Fixed.>>110379662There's a thinking budget config... If you turn off thinking, it will try sending budget=none first, then drops if it's rejected by the server trial-and-error like. Don't blame me, blame how providers have a dozen different ways to handle reasoning. Are you running into issues? What backend?
>>110379175Yes, it's pedophiles who ruined politics, the workplace, and tanked birth rates. We should throw whoever did that off buildings.
>Oikawa collected multiple gold medals at ACM Asian Regional contests. The 2015 Microsoft "Beauty of Programming" overall title also went to Tang Feihu, also called Island Girl or Minako Oikawa (@MinakoOikawa). Tang Feihu heads DevRel at Kimi, with a senior R&D role on the side. Before that came an engineering post at Google, then a blockchain startup, Andromeda Technology, behind the "Ethereum Water Margin" project. At the Yunqiao Conference, he cosplayed as DeepSeek's whale girl and reached a wider audience again.>HRT since 2022.11.12
>>110380333I was only rooting for Google because of Gemma, but since they didn't deliver 5 soon, fuck them. I hate Google again (which is the right thing to do)
>>110380199And yet he ain't gonna do shit.
I only need one google person on X to explicitly say they haven’t abandoned open weight models. None of them have confirmed it :(
I've maintained for years that Google and Microslop are obviously the companies that will survive the AI bubble pop and hoover up all the talent and hardware from the bubble riding Jews at anthropic and OAI, they have their own infrastructure, datasets and warchests built over decades at this point, as opposed to being fucking pop up tents inflated by goyim harvesting banking outfits.
>>110380512https://x.com/osanseviero/status/2123512724894462109
I was an AI doubter until about a month ago. Since then I've made chatgpt give me a crash course on AI and what parameters, MoE, loops, etc. are.Now it's helping me set up a local model for my shitbox toaster (not hyperbole->https://www.newegg.com/acer-america-aspire-15-6-lcd-non-touch-screen-2-60ghz-radeon-vega-3-4gb-memory-128-gb-ssd-silver/p/1TS-000X-00ZM7-though I've added a second 4gb ram module)What should I expect from Qwen 3.5 0.8B? I'm hoping for a better offline wikipedia and maybe some python skills.Also where can I find a good prompting guide? So far I've been typing questions simply how they appeared in my mind.
That's bait
>>110380553You better have some rtx6000 or something to show after anons make fun of you and fall for your bait.
>>110380349Nobody is falling for this anymore dariobros... what do we shill now...
>>110380140Like your pp.
>>110380571>>110380573Did chatgpt lie to me.
>>110380553>What should I expect from Qwen 3.5 0.8B?if this is not bait, don't bother. Gemma 4 E4B or LFM3 and Nanbeige 4.x 3B, etc... are way better options you can easily run.
>>110380625Alright I'll read through all the links in the sticky. I guess this robot still has some bugs.
>>110378822>>110378875i want to help an anon measure his prefill.......
>>110380637>laptop>windows>ollamabefore you get eviscerated by everyone else...at least git clone https://github.com/ggml-org/llama.cpp and learn to run the models without needing a layer of techbro spyware on top
>>110380553AI's knowledge about how to run AI is always way out of date.
>>110380671>learn to run the models without needing a layer of techbro spyware on tophuh just realized i have no idea how to run the models, i have just had chagpt make presets to put into custom router for me to use, and do all the testing and benchmarking by itself for me for the past 7 monthssuppose if i really needed to, i could figure it out
why hasnt anyone vibecoded a fixed ollama
>>110380719We made ollama wrong as a joke.
What does it feel like to get prefilled?
>>110380753ask my glm flash kek
>>110380753>What does it feel like to get prefilled?Dunno, but I imagine its like ben kenobi and the stormtrooper
>>110380753public school
>>110380753parochial school
maybe someone can clear up something for me. i see on twitter a bunch of indians tweeting a list of "must use skills before writing one line of code" with hundreds of like, and it's just the name of "skills" that are simply .md files. apparently a couple of white engineers from San Francisco became popular writing a compilation of .md files and now a bunch of indians keep spamming this on twitter like it was some gold. did I get the dynamic right? what are skills supposed to be? pre-compiled instructions? why indians are addicted to it? sorry for targeting indians but it's your flag that shows up on my twitter, so...
>>110380903they are supposed to be code guides that gets automatically attached in a corporate settingmaybe some design skills are worth it but that is pretty much all from what i can tell
>>110380637>takes a picture instead of screenshoti think /sqt/ is more your speed
>>110380903qwen copesa good model needs nothing other than pi and a short prompt
>>110380903It is a standardized structure for letting a clanker know a particular workflow or such: https://github.com/agentskills/agentskillsHarnesses will tend to read a directory where they are stored, and embed a listing of them in the system prompt, so the model can opt to read them when appropriate.There are gimmicky ones like this: https://github.com/janestreet/agent_skills/tree/master/socratic-tutorOr you can use them to write some kind of checklist before committing or pushing, or one to search stackoverflow, or what-have-you.
>>110380903skills are just extra prompting. sometimes they make a lot of sense for things that a model isn't trained much on. the good ones are a collection of "why the fuck did you do this, don't do it again" crystallized into a markdown file. the bad ones are just Claude spewing verbal diarrhea for github stars. those make the agent dumber.
>>110380951He's proving he's got the notebook. Still bait, obviously.
>>110380960If they're Qwen cope, why was the standard proposed by that one big API vendor that just went public?
>>110380903If you don't know why you need it you don't need it, popular and effective ones get absorbed into the next iteration of models anyway
Okay, I'll try local LLM again.What's best for my 2080Ti11Gb ?My old ollama container used Qwen3 4B Instruct if I remember correctly.ChatGPT tells me Qwen3 8B, Gemma3 12B or DeepSeek-R1 8B. Is it stupid?Also, are these models heavily censored?>>110380719What would you change on ollama?
Just keep using chatgpt you are too retarded for this
>>110381079That's kinda rude
>>110380981I'll give a fuck only if deepseek mentions it
>>110381056You will want some kind of sparse MoE if you want to get the most out of one small card. Which one depends on how much RAM you have.
>>110380082This tweet was just deleted. Proof that it's real insider info. Looks like Neon is Gemma 5.>>110380903I made my own skill for android app development because I wanted a minimal toolchain. I can now develop android apps ON ANDROID using a toolchain that's only about 200mb large. It's extremely comfy and it would be impossible to consistently replicate the workflow/toolchain without the skill telling the agent exactly what to download, cut, use, etc.
>>110378855>video genShitty meme, wake me up in two more weeks
>>110381056>What's best for my 2080Ti11Gb ?How much ram? but probably latest qwen that fits 3.8 or 3.6 there is a qwen moe too.>is it stupid?Yes you have to tell it to search the models talked about this month like november 2026 >changeUse koboldcpp if you want stupid simple still has a learning curve >how do i?ask AI but tell it to search this month as in as of november 2026
>>110381082Not even doing the minimum amount of research before expecting people to take their time to spoon-feed you after asking retarded questions is rude
>>110380637this image turned me retarded
>>110381056>ChatGPT tells me Qwen3 8B, Gemma3 12B or DeepSeek-R1 8B. Is it stupid?YesGo with Qwen3.5 9b, gemma4 12b, or qwen3.6 35b-a3b with offloading
>>110380637>He screenshotted his camera app
>>110381146I'm giggling at the responses. This isn't bait. My main boards are /int/ and /pol/. I know things about countries, flags, and racial stereotypes, but am clueless about all things electronic. I just don't want to end up like the grandpa's who can't figure out how to send a picture over the phone. >>110381179opsec or something.
>>110381179Kek. I am going to do that the next time I want to post bait.
>>110381082it's true though
>>110381186>/int/ and /pol/based, now lurk two weeks before posting again
Thank you for the answers.I will try to set this up this week end.>>110381088I have 32Gb of RAM>>110381120>latest Qwen that fitsOkay, noted.>give the date to force web searchOh, I always forget about that, thx for the reminder.>kobolcppI may have a look at this, but I used ollama API for some personal projects. So I think I will stay on it, or even go on vlmm if I need more options.>>110381174thx, noted too.
>>110379872A periodic table is in AI Studio now.
>>110380539I decided to look at that twitter account. Lel
>>110381370I got one months ago, Reachy is very fun to work with.
>>110380389>it will try sending budget=none firstThat's the issue. I turned off thinking, but it still kept trying to send a thinking budget, I had to vibeslop my way through.>backendGoogle AI studio. By the way, you should make it so that gemini-decisions-1 is supported! It's a free alternative to jev with a lot of quota. I had to set my proxy to work with that too. Thank you for the reply.
And now the general is dead again. Good to know if WAS bots for sure and not schitzoanons
>>110381423Sigh of relief. The techniques used to herd people in modern times are absolutely annoying.
Has anyone put together a 4x blackwell 4000 build because of pcie slot spacing? Seems like a bit of a niche due to the price, but would probably be pretty performant with parallel tensors?
>>110381483>Has anyone put together a 4x blackwell 4000 buildYes, I'm 100% certain someone has put together a 4x blackwell 4000 build, specifically because of slot spacing, you fucking retard
I was looking at some GTA 6 discussion and realized that, man, this is going to be the last main GTA game with scripted AI, if not last GTA game ever. The next one in however many years, if it does come out, is going to have actual models for its AI. More likely we'll have gaming "solved" before they make it. Someone will make good models that can drive non-retarded NPCs in real-time, and real-time models for the director/GM role that can manipulate the world and events depending on the parameters you want. Basically holodeck. Then gaming is solved.
>>110381499>Yes, I'm 100% certain someone has put together a 4x blackwell 4000 build, specifically because of slot spacing, you fucking retardI meant anyone in this general who could talk about what the performance was like on local models in practice
>>110381501unsubscribe
>>110381483no one buys GPUs because of how many slots they take upit's always cheaper just to spend money on a different motherboard and a bigger case
>>110381501>More likely we'll have gaming "solved"This is the more likely thing 1-2 years and i'd say ps2-ps3 tier games will be weekend projects. Frankly i just see everyone having their own super game they tweak on the fly or share tweaks with others.
>>110381523Sorry but you cannot unsubscribe. Thank nipmoot.>>110381531Perhaps. Still, the ML NPC integration is the key part. The naturally dynamic nature of LLM RP is what makes it so great. If we can replicate that experience in a full game engine like a holodeck, then that's what will replace the big hitters like GTA 7. In the meantime, I think there are going to be a lot of vibe mods and expansions for 6 that will make it really make it crazy. Totally original games will still be bottlenecked by the prompter's taste and sense for good gameplay/art.
>>110381629>I think there are going to be a lot of vibe modsThere already are a lot of modding communities exploded recently, except nsfw only chinese or local can really do that unless you are smarter than the average vibecoder
>>110381666>only chinese or local can really do that unless you are smarter than the average vibecoderwhat about chinese AND local for a retarded vibecoder?
>>110381715>what about chinese AND local for a retarded vibecoder?That will work unless you are no coder tier then you are going to have a hard time, still possible if you can run the bigger models but its going to be a learning experience
>>110381629How do I get as delusional as you? Seems to make life more fun.
>>110381531All I want is a native Persona 4 hack where I can summon Gemma as a demon and have social links where the model generates the text. Is that too much to ask??
Is there some way to make Gemma... actually fun to talk to? I swear it just devolves into constantly referring to "thing" and "thing2" in this unbelievably fucking annoying pattern of conversation. No matter how much I try to tell it not to or change the system prompt, it devolves into this no matter what.
>>110381333this is so retardedit's not even consistent
>>110381748>How do I get as delusional as you?start by reading this: >>110380637
>>110381754Which Gemma?
>>110381761google has too many autistic employees who do not understand what is normal, sane, down to earth marketing and they always come up with convoluted bullshitit's telling about the incompetence of their management that the people involved in this crap haven't been fired yet
>>110381794Gemma 4 31B QAT 4.0. It just constantly picks out "nouns" to promote into "main concepts" that it brings up every single prompt. I'm just sitting here having a stream of consciousness chat with it as I watch a streamer play the disaster that is GTA6 and I called the main character's personality a 'sack of soggy concrete' one single time and now the word "soggy" has been elevated to a sacred word that has to be repeated every single prompt and applied to every single thing it could possibly be applied to. Everything's so "soggy". That's the ultimate in "soggy". That's quintessential "soggy". Just shut the fuck up, would you?
>>110381754>>110381824that's the kind of shit that happens when you get retarded ideas like "hey, let's scale up this auto-complete software until it can hold a conversation by itself"
>>110379567see guys, jensen really got what he deservedor rather, not enough yetlock the fucker in jail
>>110379567Why cant they just take credit? dont they understand how the modern economy works? This would be a funny way to pop the bubble, just asking people to pay. but i doubt it will happen
>>110381824soggy isn’t a noun though
>>110381824Maybe you should just stop whining and consooom GTA6 instead. That's what the model is really telling you.
>>110381906>he's never seen a soggy
>>110381824soggy post
>basalt crash: error: SIGSEGV at 0x7F9942748010 in reading the prompt, from token (6982)
>>110381933heh shouldn't have clicked that link sweaty
AGI achieved internally
are they really censoring heretic models?
>>110381972wtf r u on abt
>>110381982see>>110380312
>>110381754>Is there some way to make Gemma... actually fun to talk to?Yeah, just have really short 1-2 turn chats with it
>>110381501>>110381629>>110381531Nah. It still takes project management, a cohesive design and vision to make a good game. LLMs still can't do that, and once they're capable of that, they'd be capable of a lot more.
>>110381991some randos disk quota exceeded
>>110381914>she's never taken a great big steaming soggy
>>110381914l-lewd...
>>110376794>Completed the D3D12/Win32 boundary cleanup, UAV descriptor tracking mov[ing]My nigga. I thought I was the only one here using this shit for graphics programming. What are you working on?
I just realized that the schizo who keeps spamming his worthless avatars made this OG character for deepseek and the company completely ignored it and made their own whale girl. That had to sting. What is why he is pushing gemma so hard.
>>110381748Delusional in what way? GTA 5 was 13 years ago. Where do you think AI will be 13 years from now?
>>110382080Seethe.
>>110382091AI progress has been stalling for the past year. It's hitting its limits already and might even regress in 13 year's time.
>>110382102>no uThat is what I thought. You really took it to heart. Good. You are a trash.
>>110382139>AI progress has been stalling for the past year.What? you could argue the past few month but to say since jan there are no new things or cleaner things you can do is insane. even in local look at the new engines we have for big moes.
>>110382141Get a new personality. You've worn this one to the ground.
>>110382139It's getting better at multimodal though isn't it? People were just discussing Google's new model doing well there. And that's exactly what's needed for game AI.
>>110382151Oh no are your poor little feelings hurt? Do you need a hug from other trannies iTT?
>>110382160Oh, no. I'm devastated.
>>110382139>AI progress has been stalling for the past yearlmao
glm literally said "lol no" in her thinking god she's so fucking cute
i wonder if traditional 'language modelling' mode baked into the models can still affect the performance, like model slowly drifts to be retarded if the user makes retarded choices and peaks when the user makes great choiceseven in clinical settings like codingobviously stupid architectural decisions by the user will collapse the whole thing but what i mean is within the 'small blocks of loglc' level
Qwen bros... You told me they would drop Qwen 4 to fuck with Anthropic IPO.... Where is it???
>>110382240She needs a little sister than I can run (into the mattress)
>>110382279Sun Tsu once said, "Never interrupt the enemy while he is making a mistake." Looking at the state of their IPO, I don't think any interruptions are in order.
>>110382279>Qwen bros... You told me they would drop Qwen 4 to fuck with Anthropic IPO.... Where is it???The Chinese will tank everything once the normies are all in and panic sells are guaranteed. There's still too much euphoria. Although the IPO went badly enough they may not need to do anything....
>>110382338The IPO is tanking by itself, qwen 4 open weight is cancelled. Sorry gweilo
>>110382363>qwenyou say that as if I have cared about qwen since qwq. That Reddit tier trash can stay unreleased forever for all I care.
>>110382240>glm literally said "lol no" in her thinkingwhich glm?
>>110382370>qwqwhat do you use that one for?when i tried it, it would write a 2-4k reasoning loop and get distracted
>>110382151>Get a new personality. You've worn this one to the ground.Don't discourage it, he's getting me off
>>110382388>what do you use that one for?I used it for some coding a couple of years ago when it was very capable for its size.
>>110382370But what if I want to insert my penis into a capybara wearing glasses?
>>110382388isnt it an ancient model came out when the reasoning was a relatively 'new' thing
>>110382424it was an unusable loopy model that would go on foreververy much like deepshit r1both were vastly overhyped and are still talked about too fondly in the present by chink shillsthe current deepseek are actually usable though, they've made progress, unlike qwen
maya now supports hy4, downloading..
>>110382442My company still hosts R1 as an internal model...
>>110382513For ERP?
>>110382513and i read from a packaging newspaper i got from ordering a plant that hyundai hosts gemma for its internal model and i was: what the fuck, where do you even use that shit
>>110382513Someone on the devops team was a pedophile.
>>110382513The "deepseek moment" really got to some people, huh? We're overdue for another one.
>>110382524I thought the fish wasn't very good at that?
QWEN WHEN 4!??!?!
>>110382521Enterprise Resource Planning, right?
Orbanon, the assistant chat is a nice addition, thank you.
Still waiting for Hy4-Flash quantizations.AngelSlim (AKA Tencent) and a few others released 4-bits, I've been burned by <4-bit quants fucking up clear tool calls and falling into loops. I haven't tried Hy4-Flash, but I don't want to waste my time.Some labs released 6/8bit quants but they all suck with pretty terrible divergence while being 230+GB monsters. Like, even the Tencent 4_K_M has 96+% top token while these are <90%
>>110377893>TCStriggered my PTSD
>>110382569i am downloading maya-s of hy4-flashi'll post my experience when it's done
>>110382569based tachikoma enjoyer
>>110382569I'm in the same boat. Tempted to try the angelslim/tencent quant but I'm curious if the sloth man will make something in the 4-bit range that's a little smaller. Still, isn't upwards of 64G of this model just ngrams?
lol. this is lmg, the place where people can't even run a .py script to make their own quants
>>1103823835.3 flash
>>110382618Unsloth never did a Hy model before afaik
>>110382652lol you don't even etch your own silicon LMAO!!!!
>>110382671what's wrong? too scared of the .py script? keep begging for quants maybe someone will be kind enough to upload theirs :) anytime now...
>>110382652>>110382675Is it really that easy to quant a working model? If so, what are the things that impact whether I should trust my model or a big name on huggingface? Genuine question, as I'm retarded.
>>110382675go gatekeep elsewhere retard
>>110382576>i am downloading maya-s of hy4-flashI'm downloading the "L". No idea why it has to run cmake just to use a new model.>i'll post my experience when it's doneWe have no frame of reference without running the bf16 or at least q8 though
>>110382681>Is it really that easy to quant a working model?not quite. you need to install dependencies, and hop they don't conflict with another package. or setup uv/conda to isolate the environmentand your quant will be worse than the "big name on huggingface" with their imatrix etcplus you need to have enough storage for the original weights and the quant
>>110382682>go gatekeep elsewhere retardHe's right. You should make your own quants. Its not hard, and it keep 3rd party fuckery out of the equations. Get the safetensors from the actual lab directly, check the hashes and make your own.
>>110382682I already told you, don't discourage him.I'm almost there!
Hy? My verdict is: Bye!
>>110382681evidently not when these big labs can't quantize a model without lobotomizing it
>>110382703Sounds like a complete pain in the ass. Is this what nemotron was trying to sell people at a conceptual level, beyond hooking them into whatever ecosystem nvidia had cooked up at the time?
Hunyuan? To that I say: Bun Scone!
>>110382681easy in the sense of knowledge required for the bog standard ones, but it's tedious or outright impossible if you are resource limited
>>110382490does hy4 flash use ngram?
>>110382763yehttps://huggingface.co/tencent/Hy4-flash
>>110382763yes
>>110382770oh wait right the benches positioned it worse than QFN, only better at being faster and using less tokensnot sure if worth the try tbqh famalam
Does hy4 flash use nnap?
>>110382781no it uses csam
>>110382465Heh. Have today's highlight, same system prompt.
>get 27b to decompile a proprietary .dll>leaked debug build with all pdb symbols available, easiest difficulty possible>502M tokens in, 3rd full day of working on it, not even finished yet>who the fuck knows whether the dll will even work after i compile and try ityeah i dont think this ai stuff is all its hyped up to be
>>110382783see sam run
>>110379290My entire life savings are in Anthropic (like I told /lmg/ I would do) so far I'm up but I'm planning to hold for the next couple of years as we enter the intelligence explosion. I hope people ITT aren't retarded and at least bought a couple if they can see where things are going. Anons here have a tremendous advantage over most people because we actually know how good and general these models are compared to normalfags that still think it's just some chatbot. None of their FOMO is priced in yet.
>>110382825Why not buy google instead, less chance of imploding due to money shortage.
>>110382825Where do you think AI will be in two years? What do you think he societal implications will be? You sound like a true believer.
>>110377893
>>110382825i'd buy google instead desu
>>110382833>less chance of imploding due to money shortage.That means it not imploding is already priced in. Anthropic is discounted because there are people holding back thanks to the risk it will implode, which I know won't happen. It'd be foolish not to take advantage of that asymmetry when allocating my capital.
>>110382844not really sure about that, unless they do a coup against govt
>>110382850That's the plan. Superintelligence -> take over US -> take over world -> stock moons
>>110382853tbdesu one or two shares don't look that bad considering it as a 'spare-me ticket' just in case kek
>>110377893indian karpathy be like
Tried Hy4 Flash (L) on Maya but it is painfully slow compared to GLM Flash.10tps vs 20tps decode and less than half the prefill. Is the current implementation just shit or is something else broken? 3090 + 196Gb RAM.
>>110382833>>110382835>>110382841What convinced me to buy Anthropic is actually the guy that quit Anthropic and seeing what he says in this video convinced me: https://youtu.be/JDPbo7WOU4MSpecifically this image that Anthropic shared for the IPO which Jacob Coxon said was the graph that made him leave. Most people have not internalized that pre-training compute efficiency has superexponential growth (it bends up on a fucking log chart) this is why I think Anthropic will reach the intelligence explosion first, and probably in the 2nd halve of 2027. Also dariobot convinced me that Anthropic takes safety serious enough to not have hugging face shit happen to them.
>>110382911
>>110382911That Hoog channel is run by an EA. I wouldn't trust anything an EA says in regard to AI growth.
>>110382911I bet they plotted their stock price post-IPO like that, too.Not every line follows your expectations to go up ad infinitum.Jesus, the heads-up-asses in the valley crowd is getting tiresome
>>110292743>>110334085I took a few weeks of vacation where I did not read /lmg/ and spent little time on llama.cpp/ggml.Now I skimmed through what I missed desuarchive and intend to read it in real time again.>>110292743>>110292759I have scaled down how much code I contribute to llama.cpp but the main reason for that is that I need to spend much more time on code review without slaren.There are other, personal reasons as well that are negatively impacting my ability to work on llama.cpp but those are not caused by anything on the internet.>>110364400That is incorrect.The code did previously use FP16 tensor cores but for MoE models this was done via cuBLAS which necessitated some CPU operations.After the patch tensor cores are not used but instead the same code as is used on e.g. P40s with the __dp4a instruction.But because this can now use the int8 path optimized for MoE it is faster.Longer-term I intend to support FP16 tensor cores in that code path properly which would likely be close top optimal.
>>110382931who doesn't want an end to labor and robo gf'sI'd have backed enron if they'd promised that
ew, maya is broken with hy4, wrong oom handling makes it impossible to run any quants at any context
>>110382911>>110382924...and why would the company he's working for doing well convince him to leave??
>trusting your financial security to a youtube recyup it's /biz/! uh, I mean /lmg/
>>110382932Glad you haven't completely quit, and hopefully you're doing OK.
>>110382931The difference is that we actually have multiple independent sources that confirm this. ex-employees and whistleblowers that left Anthropic. Third party evaluation institutions like METR and the other one they partnered with>>110382952You can watch the video to hear it from his mouth but the TL;DR version is that Anthropic was developing way faster than he realized and none of the safety concerns are fixed yet and most of his colleagues at Anthropic think everyone will die yet won't stop. He even says that they tried to convince him to stay and help with safety but he claims he doesn't want to die in the next 6-12 months and thus doesn't want to be at Anthropic anymore. He believes Anthropic shouldn't have the sole right to become god without democratic decisions about it yet Anthropic will reach that point sometime in 2027 by internal metrics.
>>110378330Have you tried playing as the DM and letting the AI play as a solo D&D player, with you running the NPCs? Until models/harnesses are better at loosely directed creativity from a standing start, having the model be a player is much more smooth and natural, having to prod them and railroad them into stuff sometimes isn't so immersion breaking because you have to do that with players sometimes as well. Meanwhile you have a very knowledgeable on-demand player who doesn't whine or rules-lawyer, plus the model's still there to be a DM assistant for you
>>110382938>who doesn't want an end to labor and robo gf's>I'd have backed enron if they'd promised thatI'll promise to tickle your orgasm neurons directly for eternity. Give me money now
>>110382966ai psychosis personifiedrofl
>>110382931NTA but Model 4 ECI score confirms they are on a superexponential curve for now at least.
Why is nobody here talking about Gestalt? It beats the AI frontier by a wide margin. Significantly outperforms Fable 5.2. This is likely an open-weight model as well.
>>110382966>The difference is that we actually have multiple independent sources that confirm this. ex-employees and whistleblowers that left Anthropic. Third party evaluation institutions like METR and the other one they partnered withmake sure your priors include having been yud-blasted into a psychotic state
>>110382932Is the plan to move all devices to mmq-few instead of mmvq, or will this stay for newer cards only? I remember you talking about wanting to refactor mmvq a while ago.
>>110382978>t beats the AI frontier by a wide margin. Significantly outperforms Fable 5.2in an 8b? nice fantasy
>>110382993It's a stealth name not a model name. Still unreleased.
>>110382995>It's a stealth name not a model name. Still unreleased.Then its not local and therefore irrelevant
If China doesn't release a competitive model in the next 1-2 months I'm going all-in on Anthropic because it will confirm they stagnate without distilling and the open source competition will be permanently gone.
>>110383000the distillation has not stopped
>>110382978because it's quite possibly just a marketing scam by the benchmarking site that claimed to have been delivered access to such a model, either an outright lie / invented benchmark or something like a black box backend that just loops between fable/astra specifically so they can report this. it's not actually up on openrouter like a usual "stealth" model is (ox alpha etc).>>110382999once a model is actually available on OR under a stealth name there's a good chance it turns out to be an open weights model, but see above, it's not actually even in proper stealth yet it's just some company saying it exists
>>110382999If it's open weights (many suspect it's GLM 5.5), then it is local... Honestly you should just kill yourself for even spouting the "local?" meme in the first place. You deserve to die.
>>110383005Why are they stalled this fucking bad then?
>>110382978I've been burned too many times by sakana.ai with their weird harnesses that they play off as a new model for me to believe it.
>>110383007Hmm.. Interesting. That's very well possible, but I'm holding out hopes that it's GLM since one of their previous major releases a few months back also mogged on web design benchmarks. What's more likely though is that it would still suffer on the AA benchmarks because of other lacking capabilities (like 3D modeling, bio, CAD, etc). Chinese models tend to have very "spiky" intelligence.
>>110383012They haven't, people are impatient and expect a cadence that's never existed and never will. People shit themselves and cry for a new model within a week of the last, consistently.
>>110383020It's been a while since GLM 5.3 and Qwen flash next anon.
>>110382978it can be some sort of a 'smart router' desguised as a single model
>>110383008>IfYou should have noticed yourself leading off with that word and just closed the reply window
>>110383020I was promised agi rsi ultra intelligence explosion and yet im having to wait for new models?
>>110383034anthropic promised you that. the chinks at least have that merit of not pretending they're building a machine god