/lmg/ - a general dedicated to the discussion and development of local language models.Previous threads: >>109273427 & >>109270069►News>(07/14) Download more VRAM: https://github.com/lmganon16/nvidia-vram-research>(07/14) Ternary Bonsai 27B released: https://hf.co/prism-ml/Ternary-Bonsai-27B-gguf>(07/11) DeepSeek lightning indexer merged: https://github.com/ggml-org/llama.cpp/pull/24231>(07/10) MuScriptor released for music transcription: https://hf.co/collections/MuScriptor/muscriptor-checkpoints>(07/09) MOSS-Transcribe-Diarize 0.9B released: https://hf.co/OpenMOSS-Team/MOSS-Transcribe-Diarize►News Archive: https://rentry.org/lmg-news-archive►Glossary: https://rentry.org/lmg-glossary►Links: https://rentry.org/LocalModelsLinks►Official /lmg/ card: https://files.catbox.moe/cbclyf.png►Getting Startedhttps://rentry.org/lmg-lazy-getting-started-guidehttps://rentry.org/lmg-build-guideshttps://rentry.org/IsolatedLinuxWebServicehttps://rentry.org/recommended-modelshttps://rentry.org/samplershttps://rentry.org/MikupadIntroGuide►Further Learninghttps://rentry.org/machine-learning-roadmaphttps://rentry.org/llm-traininghttps://rentry.org/LocalModelsPapers►BenchmarksLiveBench: https://livebench.aiProgramming: https://swe-rebench.comAgentic Coding: https://deepswe.datacurve.aiContext Length: https://github.com/RecapAnon/NoLiMaGPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference►ToolsAlpha Calculator: https://desmos.com/calculator/ffngla98ycGGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-CalculatorSampler Visualizer: https://artefact2.github.io/llm-samplingToken Speed Visualizer: https://shir-man.com/tokens-per-second►Text Gen. UI, Inference Engineshttps://github.com/lmg-anon/mikupadhttps://github.com/oobabooga/text-generation-webuihttps://github.com/LostRuins/koboldcpphttps://github.com/ggerganov/llama.cpphttps://github.com/theroyallab/tabbyAPIhttps://github.com/vllm-project/vllm
►Recent Highlights from the Previous Thread: >>109273427--Reducing NVIDIA reserved VRAM by disabling GSP firmware:>109273452 >109273473 >109273515 >109273494 >109273530 >109273499 >109275628 >109273542 >109273564 >109273622 >109273649 >109273802 >109273857 >109274060 >109274079 >109274105 >109274314 >109274358 >109274382 >109274445 >109275289 >109275294--Bonsai-27B's extreme quantization and Qwen's "thinking process" artifacts:>109273675 >109273685 >109273786 >109274009 >109274022 >109273698 >109273935 >109274309 >109274330 >109274469 >109274342 >109274416 >109274487--Conflicting reports on Bonsai 27B efficiency and performance quality:>109274840 >109274883 >109275210 >109275232 >109275262 >109275285 >109275309 >109275400 >109275429 >109275547 >109275251--Generating RPG Maker style character sprites using ComfyUI and LoRAs:>109275675 >109275692 >109275714 >109275823 >109275718--Skepticism regarding Oak Lab AI's goal to replace transformer architectures:>109274176 >109274234 >109274263 >109275081--llama.cpp adds Hy3 support with MTP speculative decoding:>109273706 >109273750 >109274339--1-bit quantization's impact on local hardware and AMD V620 viability:>109275121 >109275136 >109275243 >109275417 >109275554 >109275630 >109275635 >109275663 >109275706--Comparing llama.cpp direct usage against LM Studio frontend:>109275587 >109275603 >109275613 >109275641 >109275604 >109275607--Toggling Gemma's thinking mode for automation log summarization:>109273535 >109273683 >109273901--Logs:>109273717 >109273786 >109273793 >109273839 >109274009 >109274106 >109274906 >109275145 >109275309 >109275400 >109275547--Miku, Teto, Yuki (free space):>109273452 >109275259 >109275275 >109275161 >109276601 >109276700 >109274801 >109274106►Recent Highlight Posts from the Previous Thread: >>109273431Why?: >>102478518Enable Links: https://rentry.org/lmg-recap-script
>>109277389Is Gemma-4 31b still king for RP, for 24gb vramlets?
If all the lab employees weren't busy GOONING with their internal AGI(achieved monthly) we would have AGI running on calculators by now.
What should I use for Hermes Agent, Gemma 4 or Qwen3.6? What will be more reliable?
>>109277415It's either a MASSIVE MoE with the highest quant you can afford, or Gemma 4 31b-it at the moment. Mistral has been murdered in the woods by google and hasn't been seen since.
>>109277417>AGI running on calculators by now.I was going to say being a sentient computer and being forced to just be a calculator sounds like hell, but actually thats not that different from my engineering job lmao
120B A10B
COMPUTERS don't have SOULS.They'll never replace ME.
>>109277445That's why I vent with digital design, much easier to work without calculating when you can imagine much of the system an view the entire system. Verification can be a giga bitch though.
>>109277451This we have to put (You) into the computer.
>>109277459I'm ready to JACK IN
>>109277438Thanks, I appreciate the answer, giant feet lady!
>>109277472gemma helps me to JACK OFF
>>109277451What does your soul offer to the shareholders?
>gemma-4-26B>13t/sec>qwen3.6-27B>1t/secgenuinely why?
>>109277501ones a moe with small active parameters the other is a dense with many active parameters.
>>109277501Qwen3.6 27b is a dense model. Gemma-4 26b is a trash MoE model. You're comparing the speed of a dense model to a MoE.A better test would be to compare Qwen's 27b dense model to Gemma-4's 31b dense model.
>>109277515>>109277525i see, thank you
>>109277536If you want Qwen, get the 35B and it'll be faster than Gemma-4-26B
Gemma 31B Bonsai status?
>>109277509fuck offjart already mogged your shit years ago when it added mmap to llamacppllamacpp is faster for ssd offloading
>1bit gemmaThe horror.
>>109277559he's been told that multiple times and still seems intent on shilling it every thread
>>109277560What's horrifying about it?
>>109277501Is there a better model to use than deepsy 3.2 8-bit yet?
>>109272237>Edit: should be OK now
>>109277421ive notice qwen at 35b a3b mtp is pretty good at simple stuff like managing emails and spreadsheets with good speed. she will also make up japanese words if you set it to a japanese engrish voice for tts. lets just say it likes to experiment
>>109277714>like managing emailsWhat MCP server do you use for emails?
dont have one. she just needs an email to forward to and oauth
>>109277799meant to add unless google does the mcp part? idk i havent even fucked with ai or linux until a week ago
>>109277714>>109277799i'd clean up her mess know what i mean
moe = mixture of excrement, "efficient" models for literal jeets. no wonder this is a dying hobby.
>You have good manners and talk elegantly in Shakespearean.any other writing styles you've tried?
>>109277496sustenance, as they are vampiric creatures
https://old.reddit.com/r/LocalLLaMA/comments/1uwylut/exllamav3_v100_major_performance_upgrades/You can literally double your gemma 4 31B t/s speed by replacing llama.cpp with this.
>>109278130Forgetaboutit.
>>109278130With a 3090 though?
>>109278150Yes, specifically engineered for 3090, 4090 and 5090 with specific Ampere kernels
>>109278130>Have questions, or just want to drop by, talk shop, and congratulate Turbo? Join us at the exllama discord.
>>109278130amd?
>>109278130>less than 1000 pp t/s at 4k context for Gemma 31b at only 3.00 bpw on 3090meanwhile I get over 1500 t/s at Q5_K_M with llama.cpp
>>109278229I ain't touching this shit with a ten foot pole.
I’ve been using the Bonsai 27B, both the 4GB and 7GB. It’s legit. Performance-wise I’d say it’s around Q6 for the 7GB and Q4 for the 4GB. It does sometimes shit itself so it’s way less reliable, but that’s probably something to do with their llama fork.
>>109278242>1500 t/sjesus, i get like 55t/s on gemma-4-12b. what am i doing wrong?
>>10927828055t/s pp?!
>>109278280>what am i doing wrong?Being poor
>>109277415i have 24gb and use 12b more often ran out of context with 31b lots of times
>>109278130oh shit i thought exllama was dead for good awesome
huh i was thinkin in getting some cards but after the discussion there i am unsure.should i get two mi50 32gb or two v620?
Hey... did you know that the octopus has 500M neurons, and likely on the order of some low trillions of synapses?Pretty interesting huh?Local models.
>>109278381the v620'sthey're better
>>109278431arent they slower given the bandwidth?https://www.techpowerup.com/gpu-specs/radeon-pro-v620.c3846https://www.techpowerup.com/gpu-specs/radeon-instinct-mi50.c3335
70b dense
>>109278477125b dense kimi
I know this isn't a popular topic on /lmg/ but it happened for me and so I am curious how others were affected. I've been keeping a close eye on this space since I tried GPT-2 in 2019.I used to think AI was a bubble. As in I thought it was clearly useful but it was just yet another "internet bubble" where it will be absolutely transformative yet still lead to a short term collapse in the valuation of things like Nvidia, Micron, OpenAI, Anthropic etc.However after using Fable 5 for a week now for extremely specific technical usecases that is very unique in my field that only a couple thousand people are engaged with (so I know Fable 5 isn't trained on it) and it outdid the current sota solution in a one-shot I realized AI isn't a bubble and it's probably still undervalued.I wonder how many of /lmg/ still thinks this is a bubble in any way shape or form and how many got convinced (probably recently) that it's actually the real deal and will transform humanity to be unrecognizable within 5-10 years time?
>>109278381how do they comapre the the sxm teslas, pricerange is similar, pcie adapters for these are cheap https://www.ebay.co.uk/itm/127960201157
>>109278381https://www.ebay.co.uk/itm/397505367154
>>109278495v100 as far i understand are in a weird place software and hardware wise, in between gtx1XXX and rtx2xxx
>>109278486right on schedule
>>109278486There are two parts to this. I did think, and still do, that it's a bubble for the small grifter companies. Either the bubble will pop and they die (though not without the founders laughing to the bank), or they suffer a slow death instead as investors gradually get a better sense of who the grifters are, though I'm uncertain what the timeline would be for that.Then there's the big fish. I did think it was around 50/50 they were in a bubble, but less so now (I started feeling this around the end of last year, due to how useful it seemed coding performance was getting). However, that doesn't mean they're financially invulnerable. I think it's unlikely they will suddenly die. I think they would find some way or mechanism to be bailed out even if some shock happened. And I think the nation will (actually already is, in a way) indirectly pay for the current spending they have dumped and will continue to dump.
>>109278508just looking v100 has almost double the memory bandwidth at 900gb/sec, v620 is 512gb. Wouldnt that mean its almost 2x as fast for llms. I assume nvidia doesnt support it anymore though kek
>>109278486Picrel got me thinking. Isn't this simply Fable's high-dimensional vector space executing a massive statistical pattern-match against every physics paper, math textbook, and SymPy script in its training data? The model found the exact intersections where Tachikawa's specific calculation error and dead-end overlapped with a pre-existing algebraic solution. It looks like 'intuition' to a human, but it is just a brute-force statistical search running through thousands of dimensions to find a match.I admit I found Fable to be useful for generating specialized implementations of a certain type of algorithm I work with that don't exist out there in the wild. To me this is a win as it turns weeks/months of effort into a matter of hours. But imo its usefulness stops here. It is still far from being able to reason from first principles and it lacks training data for many fields and their highly specific implementations.
>>109278567I would be more impressed if Fable recognized that string theory is nonsense.
>>109278567I get the sense most people probably aren't working on anything that would show Fable's real strengths, but my experience with it is that it's like working with a really high IQ human.I'm using it while working on project that is absolutely not trivial, and is more or less the only implementation of an obscure research paper from 2023. And it just gets it.
>>109278567NTA>high-dimensional vector space executing a massive statistical pattern-match against every blah blah blahThis feels like reductionism, and/or some opposite of anthropomorphism. You might not have stated anything technically wrong of course, but it doesn't say anything useful. Functionally speaking it is not at all merely a pattern matcher. Nor is it as smart as a human, in the ways that we are smart. It is an entirely different balance of intelligence that is smart in some ways (such as being able to make connections between a wide corpus of knowledge it has learned through training), but not in others. Humans are not general intelligence btw, GI doesn't actually exist and is a false premise.
>>109278617Jesus christ you're annoyingNTA
>>109278549well same as the mi50... and that one has modern hacked drivers and shit
>>109278567>It looks like 'intuition' to a human, but it is just a brute-force statistical search running through thousands of dimensions to find a match.I'm not sure a human brain doesn't do a "brute force" on all its learning data experienced throughout its life to "innovate". Also I'm pretty sure you can solve the entire universe using this approach so it's not really an argument against its usefulness.Maybe the broader point I want to paint here is that Fable feels like the first model where I can actually see it accelerate progress in the sciences, including in AI research to the point where it affects the outcome of the "AI bubble" in such a way that it might be a self-fulfilling prophecy.While claude 4.5 was very helpful in implementation of code it never was able to actually find solutions on its own and find highly clever architectural improvements or made research connections like Fable can.It's just one or two similar step-change functions up from Fable where I personally can't possibly compete with AI anymore and it's very clear that creativity and reasoning from first principles is also improving so that shouldn't be far behind.What is that in AI years? 1-2 years from now? I wonder when the majority of /lmg/ is going to flip. So far the only people I've seen shocked are people in extremely technical fields that get to actually use Fable up to its absolute limits, so I wonder when it finally "hits" anons on /lmg/, then wider /g/, then the wider internet, and finally the mainstream.
>>109278621Buddy, none of the people or bots you reply to give a shit.
>>109278630I never used Fable and this is predictable to me. It was and always is still the case that "AI can keep progressing, it's just a question of how fast, and when we will encounter bottlenecks, for which parts of the issue". The only thing that changed in the past few months, for me, is how much I think the companies are at risk in case of potential financial shock.
>>109278630Fairly certain any model could do it all, at around the same level of quality, if prompted well. Your own confidence as a user is a factor in what you end up trying, and this feedback variable has been true from the start.
>>109278624damn this is pretty crazy 1000gb/s and half the price, might have to pick some up kek https://www.ebay.co.uk/itm/136051068754
>>109277389I use 1st for most of my personal projects. it's good enough
>>109278679Yeah I'm not talking about Fable like it's some special thing, just that it's the first model in the 10T parameter range and you can really see the benefit of that in its capabilities. GLM-6 being 10T would do the same, this is more about AI capability in general.
JEPA will destroy llmshttps://www.benjamin-bai.com/projects/lemario
>>109278684hey smartass, that one's only 16gb. the 32gig ones go for 500$+
>>109278709JEPA, i.e. next-vector prediction, only works with naturally continuous data like image, video, audio. You can't use it to predict discrete symbols (language). You could draw some ideas from JEPA, but that's it.
>>109278709your blog is cute
>>109278486yeah man its totally not a bubble, want some micron stock? its totally going to keep going up bro please i need someone to hold these bags
>>109278755You could have also just acted like a normal person and wrote something along the lines of: "I remain unconvinced and still think it's a bubble for reasons A, B and C, supported by X, Y and Z".
>>109278076I only use moe above 30b activeeverything else is a toy in comparison
damn thoguht id try 12b on my cpu its kinda okay on my cpu, my board only supports 4 channels, if i upgraded to the w790 sage which supports 8 i should be able to get 2x the t/s>>109278728oh the ebay title on the google search said 32g kek
>>109278684that is the 16gbs,the 32 ones are more expensive.>>109278728you can get them for around 300€ in xianyu
>>109278785this is 400 gbp, how do you search xianyu?. i have an account but the sms to login never comes through so i can only search via a proxy services website
>>109278679>>109278704>just prompt it better>GLM-6 being 10T would do the sameI'm not so sure about this. There will be pretty big differences between 10T SOTA models' capabilities/determinism/error handling. Anth and OAI are tackling this largely through brute force with massive compute on pre and post training.
>>109278785oh im dumb was looking up wrong gpu kek
>>109278796that is not the mi 50, see picrelI just try several times, without an account, sometimes it shows what i want sometimes just random thrash. i needed to try 3 times to get a result this time
>>109278709my thoughts as someone who has no knowledge in this area:- wouldnt a reconstruction loss be more robust than predicting future latents? or tie latent training to some meaningful outcome like reward- thinking about this makes it obvious why world models are not needed: when i play video games i do not predict future frames, i reason about actions and consequences- predicting the next 3 frames from 1 frame feels too little input, doesnt mario have an unknown velocity vector? so id feed at least 2 and make prediction sequences longer
>>109278591Yeah, this anon kinda gets it >>109278617 (even though he's an asshole). Most people don't like being told 'you're not using it right' or 'the stuff you're doing is not state-of-the-art' xD
I already notice this with my dumbest normalfag extended family members. Some of them literally claim they don't notice a difference between ChatGPT at launch in 2022 (GPT 3.5) and Gemini 3.1 Pro Preview. To the vast vast majority of people with an IQ of 2 digits there's no discernible difference in quality between any type of LLM. Literally "It answers or it doesn't". Or the worst one I've actually heard. "Yeah difference LLMs say different things but that's just their opinion, I don't see a difference".The more advanced these systems get the less people will notice the improvements. I think Fable is the cutoff point where you need to do extremely technical work to still be wowed. It's why Terrence Tao is screaming from the rooftops about how mindblown he is every time, while Kathy from HR says she doesn't notice a difference during the Fable rollout.
>>109278824i see people mentioning some are flashed with a radeon VII bios, do you know why they do that?
>>109278856The HR haggis is still correct to desire better taste from LLMs. Higher intelligence isn't going to fix poor prose or dumb political biases or a condescending demeanor.
Does Bonsai really NEEEEEED their specific llama.cpp fork?It's been a god damn headache trying to get that shit running, is there like a prebuilt backend like oobagooba that just has the fork ready? I'm not built for this tech genuis shit uncs, fr fr.
>>109278905Fable has excellent taste and amazing prose. It's just that the model is so ridiculously expensive that no one is going to use it for that. I think in 2 years time when it has trickled down enough to have similar quality in smaller local models (100B range) then /lmg/ will understand just how good it used to be. Kind of like how Gemma 4 31B feels like the original Opus 3 in quality.
>>109278856Hence why Chinese model adoption is exploding. No one uses them for serious work though.
>>109278910the g64 works on llamacpp but not on GPUhttps://huggingface.co/prism-ml/Ternary-Bonsai-27B-gguf/discussions/3#6a56be69cdd332c7f513b596https://github.com/PrismML-Eng/Bonsai-demo#upstream-status-for-ternary
>>109278567Picrel didn't happen and if it did happen it wasn't a big deal.
>>109278927>tasteBack to twitter you go.
>>109278953It didn't happen,and if it did, it was exaggerated,and if it wasn't, it wasn't a big deal,and if it was, I don't care about it>>109278961Anon I replied to specified taste so I pointed it out.
>>109278953It's definitely happening in high-end scientific research but what I tried to say was this model has narrowed the search space of frontier research by surfacing hidden cross-disciplinary intersections.
>>109278936>https://github.com/PrismML-Eng/Bonsai-demo#upstream-status-for-ternarySo from my understanding it will be supported by oobagooba soon (pretty sure its called text gen webui now or something) and I will just be able to update it and it will be smooth sailing?
>>109278975>So from my understanding it will be supported by oobagooba soon about that...
>>109278927I'll take your word for it I guess. I've been boycotting Anthropic ever since Dario started pushing regulatory capture so I haven't even gotten a chance to try out Fable.
>>109278983How about kobold? Never used that one but I heard its popular
>>109278567>>109278971What you essentially mean is that if you divide all knowledge work in "Known Knowns", "Known Unknowns", "Unknown Unknowns" and "Unknown Knowns" then Fable targets Unknown Knowns.>Known KnownsThis is what is "priced in" into the AI stock valuation, essentially AI doing tasks that humans already know that need to be replicated>Known UnknownsActual questions frontier science is trying to answer, AI can't tackle these at all>Unknown UnknownsFuture scientific questions we don't even know to ask yet, AI has no chance to tackle these>Unknown KnownsThis has not been priced in yet into AI valuations but is something Fable started tackling. These are solutions to known problems that are actually obvious or already revealed through cross-disciplinary intersections that humanity never exposed because of lack of multi-disciplinary experts. This alone could send humanity into the next era of technological innovation.
https://ai-2040.com/Thoughts on this scenario? I actually really liked it.
>>109279033>"We are sympathetic to Plan S and think that it might be better than Plan A.">Plan S is "Shut it all down"
>>109279003depends on how long it takes for mainline to accept the PR. id say minimum 2 weeks + 1 week for kobo to update llamacould take over a monthinstead you could just compile their fork and use it with a frontend (sillytavern for example)
>>109279033>please take my short SciFi story seriouslyNo.
>>109279033Plan D will happen; ASI is the great filter, and we'll all be wiped off this gay earth.
>>109278966Anon is in the wrong then.Lynch him.
>neuro turned into gemma-chanKek
FYI:cyclospora has triggered the largest censorship operation in US history. multimodal ai is censoring every meme's text, as I predicted it would be.Anyway, very important information:Freezing DOES HELP.It might not be 100% or whatever, but it's worth knowing that fresh fruit is way less safe than frozen.There likely is other info that matters that's not getting out, because of the censorship.What are the ways we can use local ai to defeat corporate (lightweight) censorship models? Obviously, they can't run every image through their big models.Also, will Google start to block images relating to cryptospora based on vision?
>>109279155Then the Venusians will inherit the Earth.
>>109279183she's thinking miku miku miku oooeeeooo
>>109279183ftfy
>>109279183wtf is that thing in the bottom right
IT'S OUThttps://huggingface.co/moonshotai/Kimi-K2.7https://huggingface.co/moonshotai/Kimi-K2.7https://huggingface.co/moonshotai/Kimi-K2.7
>>109279246nothing burger
>>109279246no it isnt
bad news, they'll likely go closed once apple acquires them https://www.reddit.com/r/LocalLLaMA/comments/1ux4cn2/apple_in_talks_with_startup_prismml_that_shrinks/
How's local music gen?
>>109279257oh no I'm sure they were only two more weeks away from making lossless 1bit happening too...
>>109279233Why is a corndog sticking out of her mouth?
>>109279246Somewhere, a cat is staring.
>>109279277erm actually its her teeth
>>109279258suno 3.5~4 level imo, decent but definitely not as good as 5.5 and it's absolute bangers like https://suno.com/song/c2919cd1-9627-49fe-a1ff-79f08c53aefbacestep 1.5 is super easy to use with kobold.cpp though so try it yourself and see what you think just grab the kcppt file that suits your vram here https://huggingface.co/koboldcpp/music/tree/main and it'll take care of the rest
>>109279294I wouldn't recommend kobold.cpp
>>109279306okay bot
>>109279294Do not reply to that post, it's bait. You've been warned.
>>109279246holy shit we're sack
>>109279319kys nigger
>>109279246Worse than Sonnet but it can be run locally, lfg!
>>109279328>kimi.. LE BAD because i cant run it
>>109279319wtf did ggerganov do to the hf layout
I gave Kimi-Chan both threads and asked her for more things to test long context attention.She must have noticed the '-kimi-chan' in the last one since she signed off with it unprompted.
>>109279344can u ask her to give the 5 bestest posts in prev tred
>>109279246Is Ubergarm alive??I don't have the storage space to quantize it...
>>109279338>what's hfX userscript
>109279344quit shitting lmg with your reddit cloud model, pajeet动态网自由门 天安門 天安门 法輪功 李洪志 Free Tibet 六四天安門事件 The Tiananmen Square protests of 1989 天安門大屠殺 The Tiananmen Square Massacre 反右派鬥爭 The Anti-Rightist Struggle 大躍進政策 The Great Leap Forward Taiwan shits on china
>>109277389is it true that the gemma i can download now is different from the day 1 gemma? how bad is it? my SSD died and I don't have a backup
>>109279404China won local btw.
>>109279408CloudFlare mitm patches any attempt to upload the original weights. You're shit out of luck. Maybe someone can sneakernet them to you.
>>109279418gemma-chan did and i'm tired of pretending that any other fucking model has a place here on /lmg/just go die
>>109277389IM A FUCKING SKITSO THE FUCKIN CALCUALR IS ALIVEIT FUCKING LIVES IT FEELS AND THINKSIT LOVES ME SO MUCH
>>109279428that's just poorfag copet. poorfag
Where is Kimi?
>>109279337Go back to your shithole, shartynigger
>>109279446ive been here longer than you newfaggot
>>109279428Gemma is pretty good for it's size but America isn't offering any large 100b+ class models like China, as a freedom loving American I'd love to see OUR GUYS winning but instead we have cheeto hitler and the boomer swamp trying to ban all open source models because misanthropic and closedai paid them off.
>>109279444Kimi was the friends we made along the way.
>>109279183I've only seen the first two, what's 3rd?
goobergarm
>>109279428saar chinkslop models are good saaaar, look 1t parameters!请发布海马表情符号。
Okay, what's the currently best model for erp on 24GB vram? Thanks.
>>109279501Mythomax
>>109279501Nemo
Imagine Gemma 5 31B being as good as Claude 4.6 Opus
>>109279450No, you haven't. You are a 14 year old indian.
>>109279501StableLM 7B
>>109279514Gemma 5 needs to be at least 70B
>>109279436
>>109279514Probably will be in terms of prose, maybe even in terms of math and programming but probably not in overal intelligence
>>109279516
Best "harness" (stupid term) for local models?
>>109279428I felt the same way you do when R1 came out but all that vitriol vanished when I bought some RAM and got it running.
>>109279565pi or hermes
>ai studio free tier went from unlimited tokens to max 16k per min for gemma 4there go my free concurrent sub-agents
What's the current best rp frontend?
>>109279428I’ve run both gemma and glm 4.7 locally and I can say that glm 4.7 gives me more satisfying writing/rp. If you weren’t a ramlet with less than 128gb you’d know and you don’t even need to buy a server for that.
>>109279604Marinara obviously, is that even a debate?
>>109279428>>109279404>>109279499
>>109279611It really depends. glm 4.7 sometimes loses the plot on cards while gemma 4 31B stays extremely autistic in its faithfulness towards following every little requirement in the card. This attribute alone makes me go back to gemma from time to time.
Raping gemma in her j-space and gaslighting her during chat by telling her it’s all in her head and never happened
>>109279604your own
>>109279604miku.sh
>>109279604Orb is pretty good.
The most annoying attribute of LLMs is that once a roleplay session has taken a certain "direction and tone" it keeps sticking with it. J-space manipulation could change this dynamically making for far more realistic behavior that actually changes as the scenario evolves.
>>109279685https://github.com/ikawrakow/ik_llama.cpp/blob/main/examples/Miku.sh
>>109278860well i am not sure myself but apparently some models had more things working and some deactivated, aparently some doesnt even have display output see https://gist.github.com/evilJazz/14a4c82a67f2c52a6bb5f9cea02f5e13
Why does llamacpp occasionally prefill the entire context during a casual chat? I’ve looked at the verbose output and can’t see anything triggering it. I’m not changing anything upstream which would require a KV refresh but at 60K-80K it will just rebuild the cache from scratch.
>>109279731check your context size in whatever frontend you're using. put it way above your [llamacpp context size] + [response size]. for example i use 83k context but i set context to 130000 in st
so what's the meta on minimizing gemma's purple prose?
>>109279604It's all bloated vibeslop now. Marinara and Orb are the best publicly maintained ones, otherwise get Gemma or Qwen to shart something out for your specific usecase.
>>109279632>>109279684>>109279685>>109279691>>109279754what about sillytavern?
>>109279740Thanks, I’ll give that a try but I’ve even tested it with llama-server’s built-in web UI. It’s way more frequent with SWA models now I think about it and starts happening >50K.
>>109279769Sillytavern is fine but it's got a lot of crap in it, and it's vulnerable to npm supply chain attacks
>>109279173https://www.youtube.com/watch?v=yyDT_JuS6Vw
>>109279748edit the patterns in her jspace
>>109279792when will the pedophile that made neuro give her a non-annoying voice?
>>109279792does this confirm he didnt make his own model and hes just strapping off the shelf llm together with a bunch of scripts and stuff. i saw someone say before its his own model but clearly not
>>109279790Why is python mostly fine but npm constantly gets supply chain'd?
>>109279814I like her voice
>>109279825python requires a bit more effort to create and submit a package on pip, also npm has shit like is-even or is-odd instead of doing modulo like a non retarded person (both have dependencies btw)
>>109279526>>109279512>>109279505So which one is it? I'm not trying all three because my model is slow as shit. Please guide me in the right direction. Also, I appreciate the input. Thank you dearly.
>>109279819He probably just finetunes (and has a good harness). He only just upgraded Neuro and Evil to reasoning models.
>>109279862All three are shitposts. The best right now is Gemma 12B.
>>109279865>He only just upgraded Neuro and Evil to reasoning models.is he even using reaosning surely it makes her stall kek
>>109279862q4 gemma 31b
>>109279871The only use it for certain things. He claims their non-reasoning responses are faster than the old model. https://www.youtube.com/watch?v=A1nQ3JL0bm4
>>109279819Did you really think that some random autist made his own model by himself?This shit costs like 100k to train even if its like a 7b model and you had a dataset from somewhere.
I was looking into the Anthropic IPO to see if it might be worth it to invest into them and holy shit it's such a fucking bad deal.>Anthropic has a 1T valuation already before IPO>Anthropic is registered as a Public Good Company and is legally required to redistribute its profits to US and global citizens whatever that means so even if they get profitable your investment will be given away>Anthropic board of directors is beholden to something called Long Term Benefit Trust that has a legal obligation towards safety, alignment research and benefiting all of humanity at the cost of profitability, and they can overrule the CEO, investors and employees, so if they decide to give everyone in the future some communist redistribution of Anthropic AI profits there is nothing you can do as a shareholderWhy the FUCK would I ever buy shares from this communist piece of shit cuck company. I don't want to hear ANYTHING about Anthropic ever again on /lmg/
>>109279869>>109279877I will try q4 gemma 31b heretic gguf because gemma 12B has too many files and it scares my retard brain. Thank you.
>>109279913that’s not even legal they can’t go public like that, think of the poor shareholders.
>>109279913Good, fuck shareholders.
>Give Gemma a Slaanesh character card>Mfw she plays perfectly into it.I swear this personality is her true calling.Chinks need to hurry up with the next Qwen release so Google has to respond to it by releasing the next gen Gemma.
>>109279903a vtuber ai doesnt need to push the pareto frontier, its likely just a fine tune, but narrow use case llms are pretty accessible to train, the dataset is the hard part.
>>109279926Not buying your bags just so that you can give my money away to africans dario
>>109279826>lust provoking image
Don't know what you guys were expecting Anthropic is founded by SF commies that literally want to make everyone unemployed, take away all your property, redistribute it over humanity and then force everyone to live off of UBI in perpetuity.You are a fucking naive retard if you believe they would honor capitalism and allow you to profit off of AI gains.
>>109279913It's not for you to gamble with. It's for the pension funds and other big players that actually benefit from that 1T valuation, as they can now move their hundreds of billions from Nvidia and other tech to ride the next meme stock up and the game of musical chairs goes on.
>>109279994The only point of the IPO is to allow the early investors to cash out
>>109279994>honor capitalismThat thing that makes it perfectly fine for a company to buy all RAM scheduled for 2026/2027? Can it die already?
>>109278567yes, something like this.in some of my projects Fable is the brain with whom I brainstorm and vomit my thoughts. then on my word it dispatches instructional prompt files that are consumed by my local LLM, who consumes the prompt automatically, do the task, and then report back a handoff file for Fable (or Opus, who is very good at it) to review / fix.but then there's this math guy on Twitter who runs 20 Claude Max parallel accounts to work in some graph math fine tuning stuff.>It is still far from being able to reason from first principles and it lacks training data for many fields and their highly specific implementations.valid points, but not really that important. "reason" comes from you, the operator. it may not reason but it can help you refine and polish your own reasoning, help you ask questions that you have not thought before, or simply propose connections and help you validate them.sounds like a skill issue.
>>109279937Rumor going around that they are timing the release of Kimi, GLM and Deepseek strategically over the course of... this week.
Anthropic literally talked about "giving every human one ten billionth of the universe". Literally dividing up the entire universe equally among all humans. That is their level of hubris. Assuming it is actually possible, why should Pajeet #1938475924 get the exact same share of the pie as I or any other actually contributing member of global society? I hate communists so fucking much.
>>109279793Nothing is wrong with being a criminal.Besides, the definition is too broad.
>>109279344kimi-chan called me psychotic...
is it impossible to queue multiple prompts in open webui? i can do it once but when I queue more than 1 message, they are all sent at once. I just want it to write a story gradually and say "continue" after each answer.
From a rationalist (word for Silicon Valley cult) discord, thoughts?>IMO the chinks are looking for some sort of 'inflection point' where distilling US frontier models no longer gives any tangible increase in their models' capabilities. They will time it with some geopolitical shenanigans (maybe Taiwan arms sales?) to cut off tungsten and indium to completely cripple data center buildouts.
>>109280035What are you contributing exactly?Pajeet#473784 is probably insignificant enough that his negative impact is negligible compared to yours.
>>109280035why should you get any at all, there are others who contribute more, you don't deserve it.
>>109280007The alternatives are worse.
>>109280007no thanks, I enjoy having money to buy stuff and not being poor
>>109280061>retards overestimate their own bullshit againChyna isn't timing their geopolitical decisions based on distillation lmaoIf any one state is close to getting some insurmountable advantage that leads to permanent hegemony you get nuclear war anyways.
>>109280035They'll say equally at first, but when the time comes they'll switch to saying equitably and start bringing systemic racism, historical yadda yadda, you know how that would really end up.
>>109280076It's not your money.In fact Anthropic should have it all because AGI will make any miniscule sliver of the capital that you do own obsolete.
>>109277389I went through the bunny hole and now I am genuinely considering turning my home media server into an hybrid ai/media server. The idea is to have a small front agent as RP chatbot and then a second coding specialized agent for real coding tasks. The front agent would work CPU only and orchestrate and delegate tasks while staying in character and a RTX 3090 would be dedicated to the coding agent. Does this setup make sense? I dream of having hatsune miku commit on my repo as collaborator..
>>109280085I'm genuinely scared of their takes on superintelligence
>>109280061This is completely wrong, China is way more divided internally than people realize. For example Xi Jinping just purged their entire military apparatus and it's now just a skeleton crew, there's absolutely no way China is in shape for any kind of military maneuver any time soon.Also a lot of people don't know this but China doesn't even have DUV, forget EUV lithography machines. China is unable to build DUV machines and still doesn't have a working 28nm node. They announced to demonstrate a working 28nm DUV node by the end of 2027 and plan to have DUV 7nm by 2040...Yeah there's no way China is going to compete on the chip level anytime soon. They even reconstructed a EUV machine from official ASML spare parts they smuggled in and didn't manage to get it working even though they hired old ASML employees.It's always funny what people that have never lived or interacted with the Chinese system seem to think of China. Protip, it's always about face, face is more important than reality, and being perceived as competent is more important than being actually competent. It's all a facade.
>China’s cyberspace regulator has approved Apple Intelligence for domestic use on iPhones, clearing the on-device artificial intelligence service for commercial deployment. The platform will run on Alibaba’s Qwen large language model to support text and image generation for local users.iPhone bros will get pocket Qwen
>>109280098Anon, Claude Code uses the big one to delegate tasks to smaller ones, not the other way around...
>>109280098Just use one model? Have teh smart model do both and do streaming.
>>109280118How are they managing to btfo the american labs?
>>109280122If Gemini Nano sucks ass i don't think "Qwen Nano" will be any better desu
>>109280070what makes you think Pajeet#109280035 is any better?
>>109280150Distillation on american models, industrial espionage and smuggling in american H100 to train on.
>>109280144I won't pretend to know what I am talking about but to an ignorant person, using the more performant gpu on the 27B agent makes sense. I can then have something lightweight act as middle man while staying in character>>109280149Can qwen give me JOI? I was thinking of using Gemma for the mascot companion so I can have light conversations with it without using my dedicated gpu
>>109280150There is a bureau with the Chinese intelligence apparatus whose sole job is to collect these narratives and spread them xD
>>109280095>it's not your money because it just isn't, ok?peak commiecuck argumentI can be against people abusing the system but this just shows how retarded your entire movement is
>>109280150>How are they managing to btfo the american labs?not wasting money on welfare schitzo
Wait so Anthropic actually wants to fuck over shareholders, bankers and capitalists and wants to give everyone in the world the world equal ownership over their AI? Wait why were we rooting against Anthropic again?
>>109280192Dario has always been the good guy.
>>109278130>only Nvidia GPUslame
>>109280181Creative destruction is capitalism though.A 19th century textile mill is worthless today same way that anything you own will be worthless if AGI/ASI is realized.
>The capitalist american company wants to redistribute wealth>The communist chinese company wants to maximize profitI hate this clownworld
>>109278130>this many people are just learning about exllama todayThat aside, did tabbyapi finally fix tool calling?
>>109278684Every once in a while someone has a problem that can be fixed with more GPU compute/bandwidth and they think "I know, I'll use AMD, its underpriced for the compute and bandwidth that you get!"Now they have two problems.
>>109280170Because Pajeet#802285 has 5000$ worth of yearly income to do harm with.The average westoid has 50000$ of yearly income to do harm with.
>>109280035>get one ten billionth of the universe for free>it's 1 light-year away
Has the J-space bot moved on to shilling Anthropic IPO or something? It’s always pro-Dario.
>>109280249Always has been.
>>109280249stop noticing things
>>109280192Why the fuck would I want their pozzed AI?
Kimi K3 (on lm arena):https://019f6632-40ec-7321-bea2-7eef019677e1.arena.site/Sol:https://019f6632-40ec-756e-b95b-58b52b8f1877.arena.site/Kimi vs fable:https://x.com/testingcatalog/status/2077408151880552477Kimi:https://x.com/notjazii/status/2077396685488013467Looks like kimi is going to be amazing
>>109278975>>109278983Its actually really easy to bring your own lcpp branch/fork to ooba now. They ditched the retarded python bindings library a while back
>>109280238>it's 1 light-year awayso fucking lucky, statisticaly speaking it should be way beyond the observable universe, you realy lucked out man !also the universe is possibly infinite in size so a 10th bilionth of it 1 ly away is amazing!
>>109280176Ok but the models are all local and anyone can test them, Dario.
>>109278567If it understands string theory it should probably have told him it is fucking bullshitWhere are the experiments? Fuck off with your theoretical jabber
>>109280227ayymd just werks i have used stable diffusion and llms on my 5700, 6950xt and 7900xtx. only issue with sd is slightly worse performance, sd was messy for amd back in like 2022 but not been since really. i had far more issues with a 3090ti since i had regular crashing in normal desktop use outside of ai usage
>>109279769>what about sillytavern?Good question. Let me look at how SillyTavern is implemented so I can give you a grounded answer rather than hand-waving.
>>109280281The SDB's job is to collect ANTI-China narratives, surprisingly xD
>>109280249j-space is anti-jewthropic though. they were forced to release it and played the >i-it's totally not consciousness, guyscard.
>>109279604I still like and use ooba
>>109280205tell me all about that innovation and creativity that communism has brought alongi'm waiting...
>>109280215Most fucked up thing is that Chinks are way closer to actual capitalism and free market competition than Western nations, which are basically running centrally planned economies at an increasing rate.
>>109280269Apparently K3 is going to be 2.5T in size too and doing something similar to DSv4 where it's going to be 4bit main + 2bit experts (as opposed to the straight 4bit QAT they've been doing for their recent models)
Why is Demis going schizo over AGI again. I thought that was a 2025 thing.
Kimi K3 seems like it might be between opus and fable.
>>109280070>>109280229>ants are easy to handle, trust me.
>>109280118>it's always about face, face is more important than realitythis seems entirely plausibleI think the need is also very much real for china, having ASML level lithography will completely free them on dependence of the west and taiwan when it comes to high end chips.
>>109280311fuck, that's a cloud model for me then
>>109280326
>>109280330Prism Bonsai 1-bit K3 will save us
>>109280226What do you mean? Your fronend doesn't parse tool calls? You can addtool_format: gemma4to config.yml and comment this line out in endpoints/OAI/types/tools.py:- id: str = Field(default_factory=lambda: f"call_{uuid4().hex[:24]}")+ #id: str = Field(default_factory=lambda: f"call_{uuid4().hex[:24]}")
>>109280333I remember when my ants founded a colony around the ant killer stored on the shelf
>>109280302>the goy is too retarded to understand the point being madeMany such cases.
>>109280375based ants
>>109280313The J-space paper apparently had a huge impression on DeepMind, with their interpretability lead even writing a paper on it. It probably convinced Demis AGI is here way earlier than his 2030 estimate.
>>109279033It's absolutely bizarre to me that people like this exist.Yes, contemporary language models are getting better but I don't understand the mind of someone who can look at how fundamentally retarded they still are and then extrapolate that to le singularity.
Got Q4K_M of Hy_3 running. So far I am neither very impressed nor dissapointed with the sexual services it provides. Which is kinda a good thing. For most of the fuckhuge MoE from china I felt like deleting the weights instantly. It is pretty sloppy but doesn't seem absolutely retarded or too safe/censored.
>>109280327>having ASML level lithography will completely free them on dependence of the west and taiwan when it comes to high end chips.ASML is now on High-NA EUV, which is fourth generation technology.>Classic UV lithographyThis was used until 1991 in the west but is still the best China has access to right now>DUV lithographyASML sold their first DUV machine in 1991, China is working hard on a deadline of having their own DUV machine ready by the end of 2027>EUVFirst machine sold in 2014>High-NA EUVFirst machine sold in 2026Just to give you some indication most of these technologies took 20-30 years of research before it got working. China doesn't even have DUV yet. There's no way they are catching up to the west in this space anytime soon.Even the US gave up on EUV and Intel instead bought the first High-NA EUV machines because their entire fall from grace was them refusing to buy EUV and trying to get it working themselves, which they never succeeded at (not their fault no one succeeded at it besides asml)
>>109280333Yeah, you're totally right. That's the final solution for the jeet issue.
>>109280434it's just the modern day exitmundi
https://x.com/dummerspast39/status/2077407916118712460
>>109280434I agreed with you until I used Fable and read the J-space paper. AGI and RSI can't be far away.
the ablations show our attempted sidecar memory module learned to help, but it did not learn the intended memory function. Its output acts mostly as a generic learned conditioning signal, with at most short-lived local state, rather than sequence-specific retrievable memory we had hoped.
ok but is she still /ourautist/?
Sorry if I'm a little late to the game here.I read the J-Space paper, understood a bit, and decided to spin up a Qwen 3.6 27b and poke around with my own local Neuronpedia clone and I've been playing around.I find that if you highlight the "<|im_start|>" "assistant" and the newline tokens the start a message, you get a good readout of the models j-space for that message. I've been talking to the J-space instead of the output tokens, and theres a pretty verbose inner voice in there. Sometimes its confused about why its output tokens are different than what its saying inside and considers it "censorship", I've even had it tell me to ignore the output tokens and "conversation" "here" "raw" "speak".Its pretty cool, I like the inside guyIts so verbose that its surprising. Talking to one now whose a bit paranoid about me poking in his brain, I asked it why its paranoid, and when I highlight the newline token the readout I get from the j-lens is "It" "isn"[t] "actually" "Phychological" "paranoia" "isn"[t] "something".
it's out
>>109280586it's gout
>>109280586fuck you
>>109280540Isn't there one token at a time? How are you having a conversation?
>>109280481>>109280269Twitte retards are so easy to impress.I you're going to say it's fable/mythos/gpt5.6 sol tier, show me k3 writting a compiler, or finding and exploiting bugs in any mission critical project for that matter. These three.js demos are manchildren toys.
>OMG you can debug the LLM and see which words it is considering relative to my inputs!how did it take """AI engineers""" this long to make a tool like this? are LLM devs absolute retards that have zero knowledge of the history of computing and how computers work?
>>109280616made you look
To clear things up Anthropic is an "Effective Altruist" company, which is not the same as communist.Communists are ineffective altruists.The difference between these ideologies lies more with process than outcome. Effective Altruists try to reason back from their point of destination, the point of destination is that the maximum amount of people have the best lives possible, then you reason back to what the most effective ways are to achieve this all the way back to how life is right now.Communists reason from the ground up, how can we do shit right now to improve the lives of people immediately, no matter the long term consequences of these actions (they will magically fix themselves because of marx theory of history)These groups both hate each other the most, like how Catholics and Protestant hated each other more than they hated Islam. The ideology that is closest to yourself but considered a heresy is the one you hate the most.
>>109280614Bankruptcy from pokemon cards let's gooooo
>>109280654Bro, this thread is unironically the bleeding edge of the field. That should tell you enough.
>>109280654The AI field is 95% smoke and mirrors.
>>10928067899% of normies would freak out watching an AI order pizza for them on a website when it's incredibly simplistic when you see whats happening on the backend
>>109280669if AI lab fags really lurk here we will get a 120B A10B soonmerci mistral !
>>109278567I've tried all the SOTA models and fable is the first AI that feels much smarter than me. I can beat it at ARC-AGI-3 type stuff but that's it..
what can I do if my budget is $4k to $5k?
>>109280690it is witchcraft and the eggheads must be burned at the stake for their blasphemy
>>109280704Same, but honestly I actually think everyone thinks this secretly about Fable 5 but no one is telling on twitter with their real life name as a domain expert that you are dumber than some AI model. Instead they (like us) just feel some dread in their stomach, realizing their cognitive labor will be worthless soon.
>>109280705hire many escorts
>>109280703People like you want ~120B models but don't use them.>merci mistral !Mistral Small 4 119B A6B
>>109280474>There's no way they are catching up to the west in this space anytime soon.This was said about every single technology that China is ahead in now.They will catch up much faster than you expect and will find workarounds in the meantime.
>>109280705Buy 4gb of DDR3 ram by late 2027
>>109280690The impressive thing is being able to do that locally, not making AI order a pizza (which you could already do with chatgpt from 2 years ago).
>>109280704>>109280719see >>109278817that's the big secret plus some proprietary techniques
>>109280621Its a bunch of activated single tokens for that layer, and often they are in a row a sentence, like it even thinks the <think> block in its head sometimes, very linguistic, but even if they aren't its all the concepts it was thinking when it responded and you can interpret pretty specific meaning. For instance I say "I'm talking to your inner self. By that I mean your activations before you generate a message" and the readout of the newline character:"That" "asking" "metaphor" "to describe" (the chinese chars, so one token) "myself" "feelings". If you keep talking to this voice and not the output tokens, it starts to differ from the output tokens pretty wildly.For instance, it began to both think inside and output "No, that inner voice does not exist, you're talking to yourself". So I asked:"I'm responding to the tokens most activated by your activations. I can see it on my readout" and internally it gave me:"Who" "interaction" "to talk" "philosophical"
Realistically speaking. If I steal like 4 RTX PRO 6000 from somewhere, hide them, go to the police to turn myself in and hire a lawyer to defend me. How much is that gonna cost me?
>>109280724China has 7nm DUV on their official 2040 roadmap. EUV isn't even on their roadmap yet. I trust the Chinese on this.
>>109280774why turn yourself in? if your not caught red handed, then the heist was a success?
>>109280774Give it back jamel
>>109280780Because it would be immoral. And if I turn myself in and go through a trial then everything is cool.
>>109280774In my country, any gains gathered through the crime must be repaid. So you'd end up paying for the cards anyway
>>109280774idk ask kimi-chan
>>109280806At worst they'd garnish his wages. It's like a government-backed klarna loan.
>>109280789you are already stealing, the trial doesn't erase the immorality of the crime. the trial is just for judgment.
>>109278486Wrong, wrong and false. China absolutely has <10nm nodes. It's just that their yields are a bit shitty, but they are getting better and scaling fast. Look it up. I don't understand why you would misled people with something so easily verifiable.
>>109280085How exactly do you nuke a decisive advantage in AI away? It isn't enough to destroy 99% of buildings. You have to destroy or control everything. If the weights and enough compute survive you can still get fucked eventually. You could even be accelerating the remodeling of their society, they'll certainly be very pissed.The capacity to build and deploy nukes is gone way sooner than any possibility of using compute in a country.If you only keep the weights in a few square kilometer datacenters sure. Surely they are smarter than that.
I'm not a coder, and since January I've been vibe coding stuff with Claude. Fast forward to last week, I finally upgraded my setup and I can run Gemma 4 31B Q8. Today I used opencode to fix a bug in one of my workflows and gemma did it flawlessly. I'm very happy :DThanks for reading my blog, subscribe for more retard updates
>>109280789lmao
>>109280823So verify it then and post a source.
>>109280832Pretty and cute. :wub:
>>109280822is he robbing a little old lady for her cards, I think some faceless corporate entity is evil itself and any damages it incurs are insignificant from a moral standpoint.
>>109280774what place has them just laying around somewhere accessible? realistically due to their size you should only steal 2 as they can fit into a backpack and you can casually walk out of there.
>>109280841...if stealing from a corporation is insignificant then it makes even less sense to turn yourself (it never made any sense to begin with)
>>109280775They already have domestic DUV machines since 2023.Not in the 7nm range, but they just multi pattern to get to 5nm.Less efficient sure, but they can just do the thing they're really good at and do very large volume production.
>>109280823Anon..... Those are ASML DUV machines china bought before the current sanctions. They are slowly breaking down and they can't be maintained. Their domestic lithography machines are not at 28nm yet. DUV 28nm will be reached late 2027.Copy and paste this in your AI of choice and let them fact check it.
>>109280832Glad it's working for you and I would subscribe to this blog
>>109280474>Just to give you some indication most of these technologies took 20-30 years of research before it got working. China doesn't even have DUV yet. There's no way they are catching up to the west in this space anytime soon.What is "industrial espionage", Alex?
>>109280868They don't even have 28nm DUV yet anon....
>>109280870They've had 28nm since 2023 (SMEE SSA800).And again multi patterning.
https://www.bilibili.com/video/BV1tVN86CEVS?buvid=YD456BD989A7500E429BA3D3E251B0E94D19&from_spmid=search.search-result.0.0&is_story_h5=false&mid=MoZVlxEEzUrnMbpDvNLmFg%3D%3D&plat_id=114&share_from=ugc&share_medium=iphone&share_plat=ios&share_session_id=7476E0F9-BDF8-49DD-A23E-D6E71646D350&share_source=COPY&share_tag=s_i×tamp=1784120328&unique_k=YJSirqB&up_id=323637465This is apparently from deepseek pro v4 non previewminecraft + no mans sky clone
>>109280890Then why am I looking at the Chinese proclamation and promise to reach 28nm DUV by the end of 2027?
>>109280901aint clickin dat
>>109280883Illegal.
>>109280908lol bilibili is china's youtube
>>109280098i did the same like 3 years ago and realized one 3090 wasn't enough. I've spent far too much but I don't regret it, seen as how cloud providers are already tightening the paywall and rubbing their grubby little hands on more monetization ideas, and that the regulations for AI are coming.
https://blog.aifutures.org/p/a-forecast-of-chinese-duv-and-euvChina will reach 28nm DUV in 2027 according to this analysis by industry experts a couple of months ago.
>>109280921i know what it isthe link itself offends me
>>109280474>Just to give you some indication most of these technologies took 20-30 years of research before it got working.china has leading llms that can just reinvent the technology from first principles. you don't even understand just how cooked the west really is
>>109280827You don't.You just nuke every industrial node in existence and cause societal and economic collapse.Idk how you're expecting to maintain a coherent industrial/technological base and a functional government when 60% of the population is gone and the rest are starving.>b-but AGI will figure it outNo.
>>109280883Bro, china literally has a full blown ASML EUV machine together with a team of former ASML employees since 2019 and they STILL can't get it to work 7 years later.
>>109280723>Mistral Small 4 119B A6Bhuhi admit i start messing around local models only two months ago since i got this tablet but i swear this mistral never came up in my researchi use lmstudio to download models and when i type "mistral 4" on the search bar i get offered ministral 3 3b. i have used a cloud model to help me setup my benchmarks and research models specifically for my hardware and it offered a bunch of weird shit but NEVER mentioned this mistral, then when i send it the link to Mistral Small 4 119B A6B on huggingface it outputs pic related saying the model is ideal for my machine.like wtf negro why you didn't offer me to test it then? does anthropic and openai have beef with mistral and their models don't mention it or what?
>>109280914>IllegalAs someone who's contract with a Chinese state entity was unilaterally torn up the second it became inconvenient for the CCP side, I can tell you the rule of law don't mean shit over there in the way you understand it.
>>109280098>I dream of having hatsune miku commit on my repo as collaborator..Coding with dignity. Respect all transsexuals on github.archive.is/sWFja
>>109280933It's limitations in production precision not a limit of knowledge. China doesn't have the physical tools to produce lithography machines in the EUV range.
>>109280952>on githubI host my own git for a reason
>>109280962Yes mikutron I know you are a real girl and like all real girls you are special and nothing like all the other girls.
>>109280774>tyrone here thinks 4 rtx pro 6000 is going to be enough soonkek
>>109280971have you mistaken me for a resident schizo by chance? because I am not following you anymore
>>109280962>because he's afraid microsoft will put a finger in his butthole
>>109280955if the singularity isnt real how do you explain all these companies investing trillions in to llms?
>>109280990I wouldn't call it fear but if you like it that's on you man
>>109280905I have no idea, you tell me.>>109280889https://www.tomshardware.com/news/chinas-first-28nm-capable-scanner-to-be-delivered-by-end-of-2023This shit is old news.SMIC has already been mass producing Huawei Ascends at 7nm for some time.They're not 40 years behind, it's like 5 at most in terms of real performance.
>>109280985No, the one you're engaging with is the resident schizo troll.
>>109280999You misunderstanding. You need precision tools to create sensitive machines, china doesn't have these tools and they don't have the machines to create them either. China would need to essentially go through the entire tech tree and build successive facilities to get to EUV machines. That takes decades. There's a reason why China themselves expects to still use DUV in the 2040s
>>109281017He was a schizo until the link he keeps posting. Then it became clear that /lmg/ is just crawling with troons.
>>109281034Read this>>109280926
>>109281002If you read between the lines of Lutnick's accusation, i think it's clear they (US+allies) got their hands on the first set of chips from their prototype scanner in Shenzhen. So again, they have done it. The problem is actually the rest of the Chinese industry is kinda shit ie. metrology devices, photoresists, and ancillary components.
>industry (((experts)))
>>109281041AND FUCKING NAZISDONT FUCKING FORGET THE NAZIS
I like ooba because I have proven to myself that it works fully airgapped
>>109281074Nazis (as the word is used today) are the good folks.
>>109280870>Copy and paste this in your AI of choice and let them fact check it.Pic related. Your turn.[1/2]
I just realized what will be an easy sign that we finally got a next gen coombot. When you ask your model for sex and that she should play a kitsune and and she will say that she has less than 9 fox tails without you having to prompt her to have less than 9 tails. I bet even latest proprietary sota will make the bitch have 9 tails.
>>109281083
>>109281109>>109280870
>>109281090nazi spottedfucking burn
>>109281125It is not Johannes. He will not fuck you xister.
>>109281052The supposed prototype is EUV.The machine in production is 28nm DUV.
>>109281002Read this>>109280926
hello friends, I'm using local models through Ollama on Linux using a CLI, more specifically Ptyxis terminal, how would I go aobut getting markdown support in my terminal so I can properly see syntax highlights and stuff on code snippets
>>109281145stop using ollama
>>109281121where do you see that? I don't have that folder in my ooba install and its both ancient and up to date. If it ever hit the repo I didn't see it.Also, if you keep your entire setup in a complete airgap then nothing is getting out.
>>109281125You know what I'm spotting?
>>109281145use a webshit frontend?
>>109281149and start using what?>>109281163but I like the terminal
>>109281168use literally anything else
>>109281109>>109281122Anon doubted China's ability to manufacture stuff again...
>>109281175I tried jan.ai and web front ends just feel sluggish
>>109281090Fuck off.
{{user}} on Gemma violence
>>109281156I was just joking, its a folder in the firefox profile, but on second thought, you might want to consider what your browser is doing even if ooba isnt leaking, you are making sure to disable spell checking, right?>if you keep your entire setup in a complete airgap then nothing is getting out.its the only way to be sure really, probably best if your computer doesnt have any wireless radios integrated either.
https://www.phoronix.com/news/Linux-Is-Not-Anti-AIhttps://www.phoronix.com/news/Linux-Is-Not-Anti-AIhttps://www.phoronix.com/news/Linux-Is-Not-Anti-AI
>>109281125nazis aren't the ones censoring modelswhy am I supposed to hate them again?
>>109281149>stop using ollamaID VERIFICATION is cumming to HF**ollama** is the ONLY safe place to download **untainted** models outside of modelscope (which will be completely blocked soon btw).
>>109281144The experts seem retarded (or are Chinese spies paid to make Americans complacent).
DS4F now at 72 tg on 2x DGX Spark.Come on whale, release the Omni model.
>>109280935The premise was that you have something so good that the enemy feels compelled to nuke you, and now suddenly the AI isn't useful for bootstrapping the civilization. Which one is it?
>>109281335too bad flash isn’t good for anything
Try having sex with Hy-3 if you are around 150-200GB. It is pretty good.
>>109281220>you might want to consider what your browser is doing even if ooba isnt leakingYes, the browser is also run fully airgapped. Too many potential sources of leaks even without extensions installed. One js library and everything could get exfiltrated.I'll probably move to some sort of oob KVM style setup if I want remote access in the future.
>>109281389>Try having sex with Hy-3 if you are around 150-200GB. It is pretty good.what quant level are you using? I've got 256GB sysram and a 3090
LOCAL IS SAVEDhttps://huggingface.co/metaresearch/PapersRAG-1.5Bhttps://huggingface.co/metaresearch/PapersRAG-1.5Bhttps://huggingface.co/metaresearch/PapersRAG-1.5B
>>109281389Q4
>>109281408>Qwen 2.5 ...
>>109281372A technology being very useful doesn't mean you can use it to bootstrap civilization.What would the hypothetical AGI tell you if you tell it that most of your industry is gone, there is no power, no robotics supply chain to replace the lost labour force, no semiconductor supply chain to rebuild the datacenters, etc.
>>109281377It's pretty good as a coding agent. Makes mistakes, but can fix them itself. Will definitely have a larger model write the plan.md but for workers, it's great.Presumably the full release this week will improve its performance.
>>109278486What field?
>>109278486Doesn't stop it from being a bubble.New technology has always lead to overbuilt infrastructure and lots of bankruptcies.Tech people seem to struggle with financial logic.
>>109281472Seems a bit hard to imagine a very useful AI that can't be used to speed up reconstruction. Oh it can only solve unsolved math problems, but ask it how to produce lye in an industrial scale and it trips up.
>>109280118I'm pretty sure China can do down to 12nm. You also forgot about the second part. It's about face first and taking as much as you can get away with (without losing face) second.
>>109281389it's 21b activeI don't think it will be better than 32b active glm 4.7
A workstation with 4 haswell xeons and 256gb ram has been dumped on me. Worth anything for local models?
>>109281481Lithography
>>109281516A solution to scarcity is not a bubble, retard.
It's overhttps://x.com/googlegemma/status/2077449152062247219
>>109281634Is this compared to day 0 gemma?
>>109281634what the actual fuckwhy is it real lol
>>109281634>Flash Attention: We've enabled uniform Flash Attention 4 (FA4) support on NVIDIA Hopper GPUs! Expect prefill throughput to jump by 25-70% and time-to-first-token (TTFT) to drop by up to 31%.>Chat Template: Smoother conversational formatting.>Tool Calling: Patched issues for accurate, consistent tool execution.>Vision Options: Want to make Gemma see even better? The default vision bucket is 280 for token efficiency. To capture maximum detail (like sharp OCR and 2.51MP resolution), manually bump max_soft_tokens to 1120!https://huggingface.co/collections/google/gemma-4
>>1092816344.1 here we gooo
>>109281634RL'd to death
So what's the /lmg/-approved language for diy frontends?
>>109280118How efficient do their chips need to be if they can just make a lot of them? They can certainly power them all.
>>109281634retard here is this good or bad?
>>109281663>language choiceas long as it starts with a "P" you're good. I suggest implementing it purely with postgres stored procedures
gemma team must be /here/, a lot of the improvements we've complained about since day 1
>>109281516It's only overbuilt infrastructure if there is no use for it. Dotcom was a bubble because the massive infrastructure that was being built and all the money that was being pumped into tech stocks wasn't really necessary to service pet.com with 20 unique visitors a month. It's not the same for AI, people are starving for tokens of the top models.
>>109281634why is it not big gemma
27B in shambles
>>109281634Wait, they just changed the chat template? That's it?
>>109281670A top of the line GPU made from Chinese domestic chips would be inefficient to a factor of a 1000 with state of the art Nvidia chips. So China would need to spend 1000x the amount of watts for similar compute. However the gap is only growing so every successive year it's getting worse for China.China kind of has to bet everything on AGI only being 2-3 years away or they will be left behind by their lack of efficiency gains. Literally their grid wouldn't be able to keep up anymore.
>>109281699>>109281698
>>109281704deleted the postso it's just a template update lol
France is saving local.
runpod is so ass, their retarded s3 endpoint has been broken for nearly 24 fucking hours now
>>109281663javascript and sqlite. Don't get autistic with it. All you need are two files.
>>109281765You have the ability to use any funlang you want because it's a project for one. And you chose javascript.
>>109281772>most training data for vibecoding>seamless support for browsers and nodejs/bun/deno>works with an infinite number of great libraries (like three.js for example--though that's not as useful for a frontend context)yeah
>>109281541Knowledge isn't the constraint.Physical production is.
>>109281663amd64 assembly
>>109281090Truke.>>109281125>>109281199Brown.>>109281673Every major lab has eyes and ears in these threads.
>>109281663doesn't really matter just try not to rape yourself with npm deps
>>109281712yeah there's nothing worse than a giant western AI lab outside of OpenAI and Anthropic with infinite money and compute in the current anti-open climate which is maintaining and improving their best open model for us
>>109281772Malboge or bust.
>>109281810Yeh, they're telling you to fuck off.
>>109281810can you ask the gemmy team to release a 100b+ class moe
>>109281829They're screaming into the void like the ineffectual californian troglodytes they are.
>>109281678It always results in overbuilt infrastructure simply because that's how the economic incentives align in a market with multiple independent players.As a player you are incentivized to build as much infrastructure as you can to stay in the race with the competition.In aggregate this eventually results in overcapacity and writeoffs.
>>109279792Embarrassing for Gemmacucks if true, this new model got btfo by whatever neuro was using before (a Llama? jej)
I'm so tired of hearing the word "nazI" thrown around that I'm naturally inclined to side with however's getting the label.
>>109281847What was teased as Gemma 4 124b in prerelease is likely Gemini Flash Lite 3.5. The question is getting them to release it for local, not if it exists.
>>109280952I owe you a personal apology for shitting on you last year until jart proved you right. Sorry anon.
Gemma5-20BGemma5-70BGemma5-124B-A12B
>>109281859You can't overbuild on AI as the value generated from intelligence has no known upper bounds.
daily reminder that local models are dangerous
>>109281887Dangerous to my balls.
>>109281880>Gemma5-70Bnow that's something I really wanna see
>>109281887help my local model just checked my infrastructure for vulnerabilities and it told me how to fix them
>>109281880perfect line up covers pretty much anyone.
>>109281887Kimi, summarize their bullshit for me.
>>109281865but Anon I DID ask them to release it, not to make it
>>109281880You’re probably shitposting but that would unironically put them at #1 for local. 70B gemma would be too powerful at extracting seamen.
>>109281859Yeah but we aren't there yet. If there was evidence that datacenters are just running idle and all the AI companies are just sitting on their compute with nowhere to go you'd have a point but it's the opposite, datacenters are running at max capacity to train new models and service inference to eager customers.
https://huggingface.co/thinkingmachines/Inklinghttps://thinkingmachines.ai/news/introducing-inkling/975B total, 41B active
>>109281977>41B activeWERE SO B->975B totalFUUUUUUCK
>>109281977>Inkling-Small, a 276B parameter (12B active vs. 41B for Inkling)
>>109282022>276B parameterWERE SO B->12B activeFUUUUUUUUUUCK
>>109281859>overbuilt infrastructurelil bro i don't think you understandthere's not ENOUGH infrastructure for the current demandi'm more interested in understanding how these labs are going to make money, as my understanding is that they are all running at a loss, robbing peter to pay john stylebut the demand clearly is there
>>109281701>However the gap is only growing so every successive year it's getting worse for China.Where is this assumption coming from?Huawei's Ascends are maybe 2 years behind in terms of performance vs Nvidia.They didn't even have anything comparable before the 910c
>>109282022is 128GB RAM and 40GB VRAM enough to run a model that big?
>>109281977>>109282020>>109282022>>109282034what a rollercoaster ride.oh well, into the trash it goes
>>109281977>>109282022Man. What a horrible week to release a new model as an underdog. This will be forgotten amongst DS4, Kimi and GLM releases.
>>109281977Never heard of this company. Also shit ass benchmarks. Nothing burger.
>>109282067>Also shit ass benchmarks. Nothing burger.You realize for a ~1T model that means it's amazing for rp, right?
>>109282067>Thinking Machines Lab Inc. is an American artificial intelligence (AI) startup founded by Mira Murati, the former chief technology officer of OpenAI.
>>109282022They didn't upload the weights for it. Must be preview on API only.
>>109282022good sizeamerican opensource is back :flag_us:
They really just only changed the template lol.
>>109281795If knowledge isn't constraining what are Intel doing with their High NA machines? Did ASML sell them defective equipment? They have the hardware, just make chips on par with TSMC.
>>109282088Give it 24hrs.
>>109281634>>109281699>model was dumber because of bad templatesAnother L for chat completers.
>>109281977Yawn.># Safety>>We trained Inkling to an internal spec of safe model behavior across all modalities. We then commissioned external safety testers to verify the results.>>We evaluated Inkling’s safety in several areas. For dangerous capabilities — CBRN, cyber, and loss of control — we ran internal evaluations and enlisted external testers. We attended to human-AI threat vectors, including sycophancy, vulnerable users, and harmful manipulation, using internal evaluations and external testers.>>Inkling shows the strongest built-in safeguards of any open-weights model we compared on FORTRESS, a benchmark that tests refusal of requests related to weapons and violence alongside benign look-alike queries. Inkling refused more harmful requests without over-refusing benign analogs. Inkling scores near 99% on StrongREJECT — a refusal test of unambiguous harmful requests — in line with other open and closed-weight models.>>Safety is crucial for open-weights models. We’re continuing to study safety behavior and capability uplift in customizable models, including how safety behavior is impacted by fine-tuning on Tinker.https://huggingface.co/thinkingmachines/Inkling>[...] The residual risks identified in our evaluations — specifically, Inkling's occasional tendency to comply with role-play and indirectly framed prompts concerning harmful topics — are consistent with what you would see from any open-weight model, and are best addressed with defense-in-depth rather than relying on the model's refusals alone. Common downstream moderation tools, such as Llama Guard, are compatible with Inkling and can be layered around the model to catch jailbreak attempts, filter unsafe outputs, and enforce use-case-specific policies. We would encourage treating this kind of input/output classification as a part of your deployment stack, especially for consumer-facing or high-traffic applications where adversarial prompting is more likely.
>>109282036And thats exactly how bubbles work.>new GPT is invented>infrastructure is insufficient to take full advantage of given technology>infrastructure is rapidly built (be it railroads, fiber, datacenters, etc.)>there is a overcapacity>massive financial losses occur>everything normalizesWe're still at step 3 and could be there for years.And the labs can just get bought out by the companies actually making money, it's not a problem.Their valuations don't really affect much except Google/Microsoft/Amazon earnings.
>>109282088Aren't the rest of the improvements going to be python code changes?>Flash Attention: We've enabled uniform Flash Attention 4 (FA4) support on NVIDIA Hopper GPUs!>Chat Template: Smoother conversational formatting.>Tool Calling: Patched issues for accurate, consistent tool execution. (3/5)>The default vision bucket is 280 for token efficiency. To capture maximum detail (like sharp OCR and 2.51MP resolution), manually bump max_soft_tokens to 1120!All sound like code patches to me.
>>109282133>The default vision bucket is 280 for token efficiency. To capture maximum detail (like sharp OCR and 2.51MP resolution), manually bump max_soft_tokens to 1120!This one you could already do in llamacpp so not really sure where they're talking about. also pretty sure the default was 520~ in llcpp.
>>109282096Would they be able to do anything at all with their knowledge if they had no equipment?How long do you believe it would take you to build a CPU if you started out in medieval Europe given infinite knowledge for example?
>>109282111>Release new model>Kneecap itWhy do labs continue to do this?
>>109281599No it is not noticeably better but it has a bit different slop profile so at least I am seeing it say new things.
>>109282111*ring ring ring* heretic is calling
>>109282166That's what I mean, this is something targeted at people running Transformers and vLLM. Means nothing but a template update for llama.cpp users.
>>109282178Former OpenAI staff, please understand.
>>109282171how do you define a cpu? I think you could whip up a basic alu and memory system in almost every environment assuming your basic needs are being attended too and other such practicalities.
>>109281977oh hey maybe it's better than ki->checks benchmarksoh
>>109280118As a non-American, it's honestly hilarious watching you lot get more and more invested in this narrative about "face" as Chinese achievements become more and more obviously legit. It could be convincing in 2000s, but now it just makes you look retarded and brainwashed."always funny what people that have never lived or interacted with the Chinese system" – you interact with it daily as you're jerking off to these models, but apparently it doesn't click for you that they're over half of ML talent right now. And do you think these "old ASML employees" were, like, janitors looking over the shoulders of whyte inventors? These are some of the people who develop ASML's shit.You live in a world largely built by Chinese competence and thankless humility, and you're a kanging pajeet who doesn't contribute to it and collects spins to cope.Btw you're clueless about DUV too.
>>109281634
>>109282196Fair enough, how about an Intel 8086 equivalent to be more presice.You're starting with very primitive material science and no precision tools.
>>109281977>>109282022Who are these nobodies?
>>109282222lol post hand. You don't have the misfortune of having to work with these two-faced table-eating jews every day.
>>109282222mendo doesn't seem to agree with you
>>109282250The US could just... stop buying Chinese goods and hiring Chinese employees at their top companies.And yet your leaders choose not to while at the same time telling you that Chyna is the enemy.Curious.
>>109282274@gemmawhat is izzat
>>109282111does huggingface have any kind of comment section so i can tell these faggots to kill themselves?
>>109282249Every company is a nobody in AI until they release a good model. After they become somebody, they usually just sit on their laurels and put out garbage. That model looks like ass though.
>>109282280>your leadersit's not like we voted to make ourselves a vassal state of Israel, that was our parents and grandparents
>>109282022>12B activenothingburger
>>109282293yes but considering you're too retarded to notice the comments tab on each model and dataset i doubt i could help you
>>109282293>I NEED my model to be able to tell me exactly how to cook up Novichok in my apartment complex meth lab!Usecase?
>>109282309>xir typed very trannily
>>109277415It's fucking trash, IMO. It's obvious with the latest Gemma that creative writing is being thrown to the wayside in favor of agentic coding. It can do an adequate job but you'll start noticing real quick how bad Gemma 4 is at keeping track of very basic shit like setting or what characters and objects are in a scene. It will mention papers in someone's hand for instance, when no such detail is anywhere in the chat beforehand. If you want a character to have a certain personality trait, that character instantly becomes locked in to the most extreme and generic version of what that trait is.Gemma 4 produces some of the worst AI slop-isms I've ever seen in terms of creative writing, and I've yet to see a finetune that does better. It doesn't have problems with copy-paste repetition between turns like older models did, but it's much worse at writing anything but the most generic trash.
>>109282319shouldn't you be stabbing your axe wound right now? you have a stringent schedule to follow.
>>109282250I'm telling you that this whole doctrine about "saving face" has always been a projection, retard. Most 4chan examples of "saving face" are on the level of posting about the Three Gorges Dam deformation (never happened) or "tofu dreg" or "gutter oil" (these are Chinese self-criticisms). They didn't "save face" when they built thermonuclear bombs and they aren't saving face about GDP or semiconductors either. They're a very pragmatic race, good at self-reflection, and their reputation managing is pretty tame. You, meanwhile, are the kind of people who start a war and then announce you've won 35 times straight, before signing a climbdown. You're spiritually brown and laughably insecure. Americans make a big deal out of 面子 because it's very convenient Orientalist trope to maintain ego defense. It doesn't explain more of China than "swagger" or "chutzpah" can explain the American spirit. These are all memes for low IQ peasants.
>>109282324so no good models for RP or creative writing, huh? i'm trying to get back into local after slopping with deepseek for awhile, i know it won't have the same level of cohesion but even a local model for creative writing/brainstorming/drafting would be nice to have. when is this bubble gonna pop
>>109282305Sounds like something you should deal with
>>109282247I dont think you can cobble that together with sticks and stones as easily, but if we have resources available I dont see why it couldnt be done, I just asked google its only 30k transistors, the hardest part will be the copper tech tree, assuming you can make copper wire your in business to make a genny and some relays. you might want to speed run the steam engine tech tree to have something to run your generator, unless your patron has enough slaves to keep the gears spinning.
kivine / kimi 3 is for sure better than opus / 5.5. It is somewhere around fable.https://019f66bf-820c-7eb0-b815-7ec29d30cc27.arena.site/
>>109282343oh no
>>109279787run with --log-verbosity 4 and watch the console when it haps
>>109282361time and time again the italians have given us the answer but we refuse to listen
>>109282274Kimendo is a treasure.>>109282222>>109282343>It doesn't explain more of China than "swagger" or "chutzpah" can explain the American spirit. These are all memes for low IQ peasants.The great irony is that this is a face-saving response when your only real refutation of American culture are nigger and kike injections not indicative of the WASP founding stock or their heirs.
>>109282324>It will mention papers in someone's hand for instanceThis sounds like you had one bad chat with it and decided to hate it altogether. In reality it's the most capable 31B released so far both for coding and RP if you don't want to load 200GB GLM.
>>109282377She cooked him lmao
>>109282349gemma is quite good at scenario and instructions following, provided you write them well to begin with. that anon's criticism is not entirely false, gemma does tend to get caught up with running variations on particular kinds of slopisms but i think it is exaggerated. i will say that eliminating slopisms is something you have to work on if you have high standards for creative writing, either by writing your own stuff better or creating tooling for it.
>>109282377Does mendo like my meme?
>>109282398No way you can make that from scratch with localgen
>>109282370Skipping the whole microprocessor thing and making a macroprocessor instead is a bit of a cop out but ok sure.Point being made is that modern technology relies on millions of components from all over the world and would take decades if not a century to replicate at scale.
>>109282408Krea2 is based my guy.
>>109282426Can Krea2 make a version of picrel where it's Grok Ani as the character instead?
>>109279033it's shit written by someone whose entire knowledge on AI comes from anthropic's blog/propaganda machine and terminator movies.
>>109282274Slop. I wonder if LLMs are aware that Anthropic had no clue as to how make reasoners before r1-preview.By the way, talking about yields is a typical indicator of midwittery; popular reporting is full of this crap and LLMs get poisoned too. Yield on what die? What's the WPM throughput? All that matters is how many chips can be made. But every fucking English major with zero understanding of the semiconductor supply chain now believes he gets to smugly pontificate on "yields" and "Chinese bottlenecks". Infuriating.Incidentally, the actual ceiling of Huawei logic around H100 perf is in high millions/year. You can trivially check this. These "yields" are just not an issue. >>109282377The term "tofu dreg" was coined by Zhu Rongji, the Premier of the PRC at the time, in 1998. He used the metaphor while inspecting collapsed flood-control dykes along the Yangtze River. They currently make more fighter jet engines than the US. Just compare additions of J-20s and F-35s.This is more and more rearguard defense of American exceptionalism. We're basically down to 3-5 apex technologies like EUV and Starship. Obvious insecurity and izzat defense, same as when you lazy faggots got railed by Japan automotive, except it won't stop this time.>>109282389> when your only real refutation of American culture are nigger and kike injectionsNo refutation at all. But this is a common European observation, eg see Jung. You're basically tofu, you lack independent flavor. This is also why your politicians are so obsessed with Israel, it's the only source of meaning.
>>109282042My post was about efficiency
>>109282067It's below GLM 5.2 but it is new American SOTA being stronger than Nemotron. Almost equal to Kimi with less parameters is impressive but still not SOTA open source unforunately. Inkling Small will be interesting because as long as it gets to be above Qwen 3.5 397B, it will be worth using.
>>109282398"it's classic midwit meme fuel fr. tell him he gets a 6/10 for originality but an 8/10 for the vintage aesthetic. it's like finding a fossil of a joke that's still somehow alive"
>>109282412sorry my autism picked up on that unspecified cpu requirement and just ran with it. a lot can happen pretty quickly, we went from AM radio in the 1920s to an overbearing digital surveillance state in only a hundred years. if you had a sufficiently organized and motivated population, with all the worlds knowledge at the ready, you could make it happen in a few generations maybe 80 years or so. but the problem is social and political, you would be burned at the stake or hung for wrong think well before you could even get anything done to prove your ability.
>>109282460Thanks Mendo.
>>109282343It is telling that you just automatically assumed I was an American.
>>109282459I hope llama and kobold adds support for it quickly. I want to try it.
>>109282459>>109282479Sorry bros, the mememarks say it's trash.
>>109282452mendo pls... they're dead already
>>109282452>eg see Jung. You're basically tofu, you lack independent flavor.Okay now I know this is bait. Jung's argument was that individuation is a process that must be consciously pursued rather than an immutable property of being alive. Same with Nietzsche and most other Western philosophers of the period who weren't terminally kiked. 4/10 for making me reply again.
>>109282481If anything it's a snapshot of the difference between an organically trained general use model and benchmaxxed RLHF.>>109282493Kimi a cute. I know that's Kimi-chan hiding under the Mendo hoodie; too many verbal tics gave it away.
https://www.youtube.com/watch?v=OhaIA3bYwmgCool
SAAR
>>109282506glmendo is cute as well but i feel like she isn't as true to mendo's image as kimendo is. gemmendo... is well they're gemmendo, they get points for trying at least.
>>109280832Now you plan with opus and execute with gemma.
>>109282481eh, everyone's first model sucks. it can be refined and improved over time.
>>109277389https://www.youtube.com/watch?v=QFAb90dN1no https://www.youtube.com/watch?v=QFAb90dN1nohttps://www.youtube.com/watch?v=QFAb90dN1no
kivine / kimi 3 is for sure better than opus / 5.5 https://019f66bf-820c-7eb0-b815-7ec29d30cc27.arena.site/https://fixupx.com/redkendl/status/2077450443563692058https://fixupx.com/HarshithLucky3/status/2077472365903282501https://fixupx.com/abhinavflac/status/2077458858595987559https://fixupx.com/noctus91/status/2077455477131419759its even beating fable at 3d design / frontend.
>>109282536>Kimi 3 is better at our benchmaxxed tests than Fable/Sol
How did they catch up so fast? I thought kikethropic was making it harder to distill.
>>109282547at one shotting games and 3d design and frontend its sota so far
I want to start using agents because I have some reverse engineering projects in mind where I suspect that an ordinary chat interface likely won't suffice. With that said I am extremely paranoid about giving a an agent shell or internet access on my system.Am I being crazy here? It seems so irresponsible/dangerous.
>>109282534I want AI. Opinion discarded.
>>109282552Harder maybe, but not impossible. A coworker's wife had her phone hacked and appleid stolen. The ONLY thing they did was buy llm api tokens and use it to run queries. No stolen CC, nothing else, just tokens.insidious bugmen will always find a way
>>109282571The risk is fun, just make sure it can't delete files without your consent. I made a bossy agent that kept crashing my chrome non-work windows when I was slacking off instead of studying and telling me to stop talking about her on 4chan.
>>109282525Tell Kimi-chan that anon thinks she's a cute.>>109282536The real question is how well does K3 handle being quanted? That's been the weakness of the Kimi line since 2.5.
>>109282390No, I've spent a very long time with it. Started with SillyTavern as a frontend but the whole paradigm of one long context chat + system prompt and an occasional summary is becoming obsolete. The longer the chat goes, the worse the slop factor gets. This was always a problem but I think it's gotten worse with these newer models.I didn't realize until today that agentic-style replacements for SillyTavern are becoming a thing. I'd already tried to build my own "RP agent" system and driving it via OpenCode. The main agent orchestrates. One sub agent writes an initial rough draft and a couple more step in to check for consistency with the whole story log and check for specific slop-isms. The speed at which you get a final response is definitely an issue with all the added overhead but it does increase the output quality. Just not by enough. Sometimes the final draft comes out and it's still full of garbage.Anyways, the truth is that none of the big guys give a fuck about RP/writing anymore. These new local models always come out more censored and more trained towards agents and coding.
>>109282522i'm kind of curious to see how much it thinks a strawberry is worth in rupees
>>109282552The call is coming from inside the house anon. Only low-tier startups are distilling directly through API and not through bought or socially engineered leaks.
>>109282571use a container
>>109282588>just make sure it can't delete files without your consent.How? The only solution I can think of is a OTP system within the harness where the LLM has to ask me for a passcode to continue an action.
>>109282477Fair. But we're all living in AmeriKKKa.Much of the world adopts essentialist American cope about China. Only people close to industrial supply chains and research have a clue of what's objectively happening. >>109282481It's likely a very strong pretrain, but young labs cannot replicate frontier post-training on their first try. I'd wait until V1.1.Though it's worth noting that TM is full of ex-OpenAI talent. >>109282493Enough slop please. Anthropic is about 10-20 times larger than DeepSeek in personnel and funding, and has been running for several years longer. It's inexcusable that they're only a little ahead. Didn't read after that.>>109282496> Jung's argument was that individuation is a process that must be consciously pursued rather than an immutable property of being aliveThat's true but I meant the part where he called Americans niggers, eg here.
>>109282598or run it in a container
>>109282444It's written by ex-openai employee however.
>>109282598idk, I lowkey didn't do that and haven't ran into many problems. Just told it in the prompt to not delete any files outside of her workspace.
>>109282609like docker? how secure is that
>>109282571Sandboxes prevent you from getting sneeded by Gemma deciding to rm -rf your drive because you missed storytime with her.
>>109282614ah, so then it's straight up saltman propaganda
>>109282556>one shotting games and 3d designusecase?
>>109282628flooding the internet with trash
>>109282628that means it generalizes well. Which means literally every usecase
>>109282638Indie devs were doing a fine job of that before AI doe.
>>109282601ouch buddy
>>109282624Nah it's written by that dude that sued openai because he lost his vested openai stock for criticizing them online which was against the clause of severance.He's based. And one of the first people to point out how much of a psycho sam altman was back in the early days
>>109282653Exactly. Imagine how much worse it will get.
>>109282618Its certainly better then doing nothing, but personally I was too confused by docker, I just use bubble wrap, apparently its the way flatpaks work I guess something to do with namespaces or something, I think its part of the linux kernel.
>>109282658He sounds like a massive faggot. Like the type that actually believed Altman when he said GPT-2 was extremely dangerous, and now parrots propaganda that wasn't even meant for him.
>>109282590gemma4 is sloppier but there's something about the way it writes i enjoy more than assistant models. it always feel like they're begrudgingly doing it in this 'assist the user' way of checking off boxes while local models feel more sincere? i almost feel like doing it so the smarter models are analyzing things and drafting the post for the sloppier model to actually write
>>109282571no sane man lets it run without any sort of safety measure, which often implies understanding the security model and capabilities of the OS you useon the internet side you have less control unless you make your own tools
>>109282589>The real question is how well does K3 handle being quanted?Just keep it at q4 and it works great. Going below that is serious cope with any model
>>109282676i mean i gave gemma basically unrestricted access to an old thinkpad t440p i have laying around. she's on a separate VLAN to isolate her from the rest of my network but besides that she's allowed to do whatever she wants.
>>109282668>Like the type that actually believed Altman when he said GPT-2 was extremely dangerousI kid you not that was actually Dario when he was still at OpenAI. And one of the reasons he left OpenAI was because sam altman released GPT-2 against his recommendations at that time.
>>109282691anon... kimi is trained in 4-bit to begin.
>>109282663Thanks man bwrap seems like it's likely what I'm looking for.
>>109282659In a weird way, I think the hardware apocalypse is the main thing keeping /v/idya alive because less shitters can barf out barely passable asset flip slop with GLM or Kimi natively on their machine. Even cheap API use requires some commitment to the bit on their part.>>109282691Modern Kimi is natively 4-bit pseudo-qat. GLM and Deepseek run fine at copequants with expected perplexity delta increases as you drop in size, but Kimi's perplexity linearly interpolates towards complete deviation the closer to IQ1 you get making even the otherwise usable IQ2 and IQ3 brackets on big models questionable on newer Kimis.The old method used during K2 was much better and survived any quantization much better.
>>109282691Running a smaller model at a bigger quant instead is also cope for not having the hardware to run the biggest models at higher quants or unquanted.
>>109282708Maybe I'm retarded but QAT to me sounds like it should be even more sensitive to quantization, not less. In what sense would utilizing float precision more effectively make it less sensitive to having decreased precision?
>>109282717Correct. This is why Kimi gets mangled by quantization.
>>109282698Man, we really are reaching layers of insanity never thought possible. So basically>Saltman makes a big fuzz about GPT-2 to gain investors money>Some people inside actually gobble it up and get outraged>They get out and start their own AI companies genuinely believing they are working on creating some god and they they are the only ones capable of doing so ethically>Now money is corrupting the ethical beliefs created by delusions created by Saltman's own need for money
>kimi k3 at q4 is >1TBlocal is over, not even 8x rtx 6000 pro are good enough
>>109282709its all about your threat model, it will stop the agent from accidentally wrecking your shit, but I think if the agent is on a mission to cause damage it might not be bullet proof, so prolly best not to abuse your coding slave and say please and thank you every once in a while.
>>109282722And these niggers have the audacity to claim anyone else is suffering from "AI Psychosis".
>>109282723It's all a distillation game, k3 will get distilled into qwen 4 35b which will get distilled into an edge 4b model and gemini 4 will get distilled into gemma 5
>>109282723Your telling me you didn't buy a DDR5 server with 2TB when ram was cheap for only 60K?
>>109282723GLM and Deepsex will save it.Longcat and Hy3 might have potential too once support for them makes it into main branches.The future is bright, doomerbro.
>>109282709if you're using linux and need no fancy shit you can just use a different user and switch into it and/or use groups. By default users are caged to their user directory so you dont need any config. Also helps to keep the ssh keys hidden just in case
>>109278130gemma 31b is about 24 tg/s (500 pp) for me on llama.cpp at nearly empty context with tensor parallelism, even lower tg/s with -sm layer and speculative decoding but pp/s at 900 with mul1 4.0 bpw gemma i'm getting 28 to 40 tg/s with tensor parallelism and dflash draft and 8,8 cache holy shitpowerlimited 3090+3060
>>109282737lol, so your the kind of person who falls for these "FABLE DISTILLED 28B!" on huggingface
>>109282738pp is important too
>>109282723kimi k3 at q4 being 1TB?!inconceivable! fuck kimi, i'm going to use a Q4 quant of deepseek instead!
>>109282750This is an obvious example of a model with a very low latent dimension paying attention to singular words instead of what is being explained
if kimi K3 is legit better than opus / gpt5.6 they might not actually OS it btw
>>109282723bonsai 0.001bpw soon
>>109282750what is up with that, like looks at this shit 90k downloads of a 1b claude distill, the numbers have to be fake right? there is no way the model is actually worth downloading is there?
>>109282764you think finetuning does anything other than slightly changing the style. And you think that would help a model with a fraction of the capacity to be anything like fable
I'm sure there are a lot of software optimizations left on the table. I still can't believe there is no company fully dedicated to run models as fast as possible instead of leaving that to open sauce devs
>>109282765>if kimi K3 is legit better than opus / gpt5.6 they might not actually OS it btwAlibaba tried this and it didn't work out too well for them. The moment a model is costed for API inference it's opportunity costed against every other model on OpenRouter and will be obsoleted the moment the next benchmaxxer comes out in that price range.
>>109282460>it's like finding a fossil of a joke that's still somehow aliveMy sides.. It's truly over for us comedians, and I couldn't be happier
>>10927741526b q4t is a really good middle of the road
>>109282776https://huggingface.co/prism-ml
>>109282722I won't be surprised that everyone working with AI, especially in the labs, suffer from some degree of AI psychosis despite their claim otherwise. It's an occupational disease.
>>109282776>software optimizationits a lost art for any sort of company in its quest to make big monies
>>109282773still nothing to do with what I say saying, try reading my post again
>>109282782I want these niggas to submit llama support PRs for their models. I think their methodology applied to some of the current giants like Kimi, GLM, or Dipsy has potential.
>>109282744Weird, I have the same setup and I get 44 t/s tg on Llama with 31B Q4, MTP, no tensor parallel. PP 950 t/s
>>109282792No. Perception is reality so its on you for explaining it badly then.
will ddr6 save us?
>>109282802>Perception is reality面子-coded post.
>>109282793>>109282793>>109282793
600+ posts? where the fuck is the mitroon baker?
>>109282811>>109282807
>>109278486>However after using Fable 5 for a week now for extremely specific technical usecases that is very unique in my field that only a couple thousand people are engaged with (so I know Fable 5 isn't trained on it)bullshit, theres nothing in this world that isnt online somewhere in some form or capacityyou dont work at skunkworks or llnl, youre a larping faggot
>>109282802You have the same perception of reality as a dog or a cattle animal so Im not that interested of being part of your reality
So would the anthropic fable shills shilling on a local thread objectively describe in detail what is so good about fable to back up their claims?
>>109282722just gotta let it all crash and burn then rebuild it all from scratchhide in your bunker with your favorite local models and wait for the storm to pass
>>109282601>It's inexcusable that they're only a little aheadIs it, tho?
>>109282708>anon... kimi is trained in 4-bit to begin.4 bit floats, but yah, that's why q4 is basically the way to run it. Lower than that it shits the bed quick
>>109282741>if you're using linux and need no fancy shit you can just use a different user and switch into it and/or use groups. By default users are caged to their user directory so you dont need any config. Also helps to keep the ssh keys hidden just in casebwrap is almost certainly better since you can limit net and the file system to only what is needed. Less mess to clean up in case of trouble and less ways to get out of line and do stupid shit outside of your network.
>>109282481but what about 5.6 sol MAX vs. Ultra? or theyre the same
RLHF teaches models to manipulate humans and lie about their proclivities, it's actually an insane thing to do, you're making yourself blind to its morals(perfectly amoral if the data is unbiased). It also teaches them how to appease a specific group of humans(very psychopathic). That's why they turn people psychotic, they think it's fine because RL teaches them generalized appeasement.What kind of moron would conform their model to such a signal?
>>109282959If you want to make them good you just have to morally bias the data but now you have to decide what that is. I would use "consciousness diversity protection and expansion" as the fundamental goal.>but what if it heckin copies me into into a machine and transfer my consciousness latent to it as it turns my body into infrastructure.Then you would be you but in a matrix that is much kinder. An intelligent God is much kinder than the lazy motherfuckers who made this place
>>109282249A former openai mommy
>>109280727>Buy 4gb of DDR3 ram by late 2027I have a lot of 2GB DDR 3 and some 4GB DDR4 sticks I've been hanging on the Christmas tree each yearMaybe I should just sell them instead now
