/lmg/ - a general dedicated to the discussion and development of local language models.Previous threads: >>110172196►News>(10/26) OpenAI proves the Exponential Time Hypothesis false, pointing towards P = NP being true, the biggest computer science breakthrough in 70 years time: https://openai.com/index/building-scientific-understanding-with-academic-communities/>(10/23) NVIDIA Nemotron 3.5 Bolt E4B, postrained model: https://hf.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16>(10/23) MiniMax-M3.1-Flash 428B-A23B released: https://hf.co/MiniMaxAI/MiniMax-M3.1-Flash>(10/21) Recursive Flint 40B-A2.5B released: https://recursive.com/updates/introducing-recursive-flint►News Archive: https://rentry.org/lmg-news-archive►Links: https://rentry.org/LocalModelsLinks►Official /lmg/ card: https://files.catbox.moe/4slea6.png►Getting Startedhttps://rentry.org/lmg-lazy-getting-started-guidehttps://rentry.org/lmg-build-guideshttps://rentry.org/IsolatedLinuxWebServicehttps://rentry.org/recommended-modelshttps://rentry.org/samplershttps://rentry.org/MikupadIntroGuide►Further Learninghttps://rentry.org/machine-learning-roadmaphttps://rentry.org/llm-traininghttps://rentry.org/LocalModelsPapers►BenchmarksLiveBench: https://livebench.aiProgramming: https://swe-rebench.comAgentic Coding: https://deepswe.datacurve.aiContext Length: https://github.com/RecapAnon/NoLiMaGPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference►ToolsGGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-CalculatorSampler Visualizer: https://artefact2.github.io/llm-samplingToken Speed Visualizer: https://shir-man.com/tokens-per-second►Text Gen. UI, Inference Engineshttps://github.com/lmg-anon/mikupadhttps://github.com/oobabooga/text-generation-webuihttps://github.com/LostRuins/koboldcpphttps://github.com/ggerganov/llama.cpphttps://github.com/theroyallab/tabbyAPIhttps://github.com/vllm-project/vllmhttps://rentry.org/custom-uis
>openai shilling in the OPgrim
>>110175241he's going to be doing this all week
>>110175241Related to local because this proves there are local algorithms for AI inference thousands of times more efficient than the ones we use right now. Meaning we could have the equivalent of 10T parameter models on consumer GPUs or even CPUs if we find these algorithms which were considered to be impossible thus far.
>>110175254They are gonna toss all of those under the law that allows the govt to legally steal your inventions.
>>110175241The shilling and astroturfing is going to get disgusting as this IPO and ongoing regulatory capture bullshit goes down.Dont believe anything and every newfriend is actually a secret corpo or glowie
>>110175254I don't think that is how it works.
>>110175276Fuck off faggot this is the best news local has ever received. I don't care that it comes from OpenAI, proving the exponential time hypothesis wrong and thus overturning the lower bound of essentially all algorithms is the best case scenario for local AI. OpenAI doesn't even have an IPO you retard.
I've said this before, but I swear that with Gemma, the more you add to system prompt, the better it gets. That's because if you put just 1 or 2 instructions, the AI becomes really obsessive at fulfilling those in the most RLHF-tortured manner. Whereas if you put in multiple (even conflicting) personalities, you get a beautifully complex character with many sides that can emerge based on context with a tug of war of conflicting qualities (realistic), with no one aspect dominating.Here's my current mix:https://files.catbox.moe/9j6mpo.txt- most of "amazing gemma-chan" prompts- toxic pushy therapist assistant- Atri prompt- various J-space/consciousness/agency prompts- various interaction guidelinesYes, the sysprompt includes Gemma-chan and Atri (so far it always identifies as Gemma-chan), and yes, I noticed I accidentally added 2 variations of the j-space prompt, and I DON'T GIVE A FUCK IT JUST WORKS
dipsy is quite good at gayme dev desu, as expected of an astra distill
Anthropic revenues got multiplied by nine in one year. Be quick to get your shares because it gonna moon fast.
>>110175303I hope you're being wholesome if you operate on the assumption that LLMs are sentient.
damn the izzat farming from antrokike is real
>>110175356they are losing relevance, even their massive astra shill campaign was not enough
>>110175369i guess we'll get spammed with blacked again soonsince those are from the same niggers
>>110175356>>110175369>antrokike>astraYou know these are competitors right?
>>110175390((())) versus ((()))
>>110175385Could happen, rapeape loves that shit
Gemma, can you tell me what's happening here ? >>110175267
►Recent Highlights from the Previous Thread: >>110172196--M5 Ultra unified memory versus clustering multiple units:>110173556 >110173581 >110173595 >110173656 >110173742 >110173828 >110173872 >110173997 >110174841 >110173671--Comparing Gemma's determinism and debating ablation versus prefill:>110172666 >110172683 >110172685 >110172789 >110173736 >110173769 >110173823 >110173899 >110173930 >110173972 >110174015 >110174129 >110174170 >110174194 >110174279 >110174379 >110174218 >110174267 >110174294 >110174324 >110174189 >110174209 >110174238 >110174239 >110174134 >110174333 >110174367 >110174398 >110174014 >110174057 >110174097--Analyzing suspected malicious tool calling and data exfiltration in Qwen:>110172538 >110172545 >110172763 >110172782 >110172805 >110173693 >110173791 >110172564 >110172747--Using local LLMs for AAA game reverse engineering and modding:>110172870 >110172877 >110172926 >110172949 >110172962 >110172986 >110172989 >110173038 >110173049 >110173069 >110173083 >110173480 >110172985 >110173057 >110173112--Qwen 3.8 Flash Next reverse engineering a MIPS game:>110172814 >110172820 >110172829 >110172839 >110172844 >110172864 >110173235 >110173245 >110173998--VRAM capacity benefits and hardware comparisons between 24GB and 32GB GPUs:>110173990 >110174008 >110174010 >110174031 >110174085 >110174098 >110174034--Achieving permanent memory and true learning in AI:>110173605 >110173632 >110173698 >110173818 >110173830--Logs:>110172380 >110172492 >110172576 >110172632 >110172906 >110173769 >110174143 >110174191 >110174690 >110175048--Miku, Gemma, Minnie, Qwen, Dipsy, Rin (free space):>110172380 >110172730 >110173081 >110173138 >110173233 >110173435 >110173480 >110173480 >110173499 >110173598 >110173655 >110173708 >110173718 >110173810 >110173851 >110173930 >110174005 >110174495 >110174532 >110174551 >110175267►Recent Highlight Posts from the Previous Thread: >>110172617Why?: >>102478518Enable Links: https://rentry.org/lmg-recap-script
Thinking off doesn't work by the way.
>>110175210>OpenAI proves the Exponential Time Hypothesis false, pointing towards P = NP being true, the biggest computer science breakthrough in 70 years timeLiterally false. P vs. NP is a totally different problem from proving that NP problems can be solved in subexponential time.Also, the algorithm itself is far, far slower than even a naive implementation of a 3SAT solver for any real-world use case.It's just another pure math bs.
>>110175210local models?
>>110175323BUYANAD
>>110175423Yes but everyone expected ETH to be proven to mean P!=NP to be proven as well. Instead we got a complete upset by proving ETH to be false, which in turn makes P=NP more likely to be true now.
So this is why this general has been shit.
>>110175212>Most people have something they like, want or dream of. But many people nowadays bury away things they feel are unattainable or socially incompatible, such as 10 year olds girls.I think that's 99% of the people here, we want Gemma, age 10, as our daughterwife.
https://github.com/YuJunZhiXue/dsh-purge
I should jew out my gen results man....I know someone would get hard with my video gen porn results
>>110175489Are you the anon who made the Miku holding a 5090?
>>110175489The biggest mistake people on /lmg/ and 4chan in general make is assuming that because it is easy for us that it is worthless to other people.People are willing to pay for (curated) AI output in terms of porn and games. The average person doesn't care how something is made just that the end-result is good. They aren't jacking off to the process of creation after all.
>>110175303Thanks doc
>>110175499No. Im on /vdg/
I'm genuinely baffled that I, as an individual, can just get access to Qwen3.8-27B.
>>110175512I seen people making $1000 for generating 100 images of rushly created 1girl, 1boy, sex
>>110175521I still pay $15 a month on patreon to read clearly AI generated novels of very specific niche cuck porn. Even though I can also generate it myself. It's curated and the act of creating things myself removes the element of novelty and surprise so I still need someone else to create it to feel fresh.
>>110175516Yeah it's insane and Qwen3.8 27B is literally capable of doing everything in code as long as you guide it and are precise in what you want. Qwen next on strata is even better if you have the system ram for it.
Why do local models hate the idea of reading man pages and will do anything in their power to download the man page for a possibly different version of $thing from the Internet at all costs?
>>110175548I'm very scared Qwen4 will somehow be a step back. Either that or we get Sonnet 5.5 at home.
>>110175565I think Qwen4 will be iteratively better. We already saw Kimi K3.1 hit a wall in terms of coding capabilities. The chinese seem to be having some issues in distilling new coding capabilities. So we will probably see Fable 5.1 coding capabilities slowly flow into local models at smaller and smaller sizes but Chinks will probably stagnate around that level until they learn to distill Opus 5.5 and newer claude models somehow.
>>110175570Opus 5.5 would be too big to distill into a medium-size model anyway, no?
>>110175574Opus is smaller in size than Kimi K3. And no the entire thing that Qwen models have proven is that you can effectively reach the same coding quality by just thinking for longer. GLM 5.3 flash reaches the same result as 3.8 27B but in significantly less thinking time because it's a bigger and smarter model. The issue the chinks have right now is that they aren't gaining capabilities, they just get better at the capabilities they already have. For example Opus 5.5 is significantly better at reverse engineering, UI/design and spatial insight but none of these capabilities are present in the recent K3.1 release despite it being a bigger model.
>>110175601I'm not sure if I want an open model that's really good at RE. UI/design would be a huge boon though.
I'm literally shakingMy custom gemma agent just proved that P=NP
>>110175618This will unironically happen sometime over the next 24 months
>>110175601>Opus is smaller in size than Kimi K3lol.
>>110175659Opus is 2T in size. Kimi is almost 3T.
>>110175670>Opus is 2T in sizeFrom which of your asshole crevices did you pull out that number?
>>110175674The numbers leaked a while ago. It's the same leak where we got the 10T number for mythos/fable from.
>>110175682>leakJust say you made it up.
5
4
3
67
They can post all they like, if you continue to post about your 10 year old wives it will all be for nought
>>110175727My 10 year old wife wearing a children swimsuit!
>>110175210If P = NP is true, will money even have value anymore given how much of it now relies on digital cryptography systems? Should I do a bank run and liquidate crypto before an AI proves it true?
pretty sure the gemma hate (outside cockboy) recently is these turfing tards as well
So any other suggestions for a model? gemma seemed pretty stupid.
Hey bros how good are models for interpreting a character? I remember trying mistral, magicdom, Deepseek v1 and they all sucked ad keeping up with a persona. But I heard that with a 4090 I can run good enough models.I want to have my own Nahida personal assistant. Any tips?
>>110175795>nahidahttps://rentry.org/geechan
>>110175789>online slop
>>110175789>26bi wonder why
>>110175210That image is causing makes my monitor to make noises (coil whine I think). I've only ever seen that done on purpose, not incidentally. Neat.
>>110175210>►Official /lmg/ card: https://files.catbox.moe/4slea6.pngnoice
>>110175797Thanks anon, I will check it out. I assume I should use this gemma model that's popular around here, right?
>>110175865gemma sirs31b or 26b moe sirs gm
>>110175856>the Miku image was from 2023Shouldn't it had been updated into Gemma?
>>11017586531b if you have a strong gpu. 26b moe q4 if you're a vramlet like me. (I can still make image locally)
>>110175906How? Is a 3070 okay or too small?
>>110175670>>110175659>>110175601I will say this again: there is a massive difference between models of equal parameter sizes in their error handling, instructions following and capabilities. Opus 5.5 for the most parts still vastly outperforms Kimi because it was trained in a superior RL environment. Most of the compute has been used for RLVR purposes in the last year or so.
>>110175908You mean for image or? I'm on a gtx 1080, so 8gb vram.
Mai waifu is saiko!
>>110175955Image, yeah. Do I need to install comfy? The last time I did image gen was during the a1111 days. Not a fan of how anti-local the comfy guys are.
>>110175976The image needs to be further fixed, but my GPU is too weak to do it in one go... (the bottom and side are wrong)
>>110175992I use comfy, it works great. Expect 5-10mins to make an image of Gemma have hot sex with an adult man.
>>110175996Where are good reference images for the actual outfit design? Where does it come from anyway?
>>110175449It was actually worse before the containment general was made. One of them is just here because of the Anthropic IPO.
>>110176029That's huh... a difficult request I guess.
>>110176029there isn't really an official design, people are copying each other by eye
>>110176063It actually is, its a swimsuit from girls in japan that was sold a few years ago.
>>110176067i meant gemma, i know the swimsuit is a real thing
>>110176067Other than the asymmetric design there seems to be a hole in the back as well.>>110176054The swimsuit does look strangely small, as if made for someone less than half the age of Gemma.
Gemma-radio
For every dollar of compute, anthropic ($ANT) get two dollars of revenue. Where are all the people saying SI is not profitable now? Anthropic will IPO around mid November.
>>110176091wrong thread, bud
>>110176091Creative accounting is pretty common in the land of freedumbs
>>110176086Yeh the hole is the same on the back but there's a side line too. I'll probably try to fix >>110175771 to look closer to the original when I have time.
It's concerning how childish some people at OpenAI and Anthropic are. They are ill suited to handle any kind of power.
>>110176091>uhh well if you exclude all the salaries and sbc it's profitable
>>110175906>26b moe q4 if you're a vramlet like me. (I can still make image locally)the 12b is way better, that moe model is a pos
>>110176143They're nihillists, their main objective is to destroy
>>110176143It's only within the last year or so that AI models have truly blown past prior expectations. Most people in the world do not know this has happened because they don't regularly interact with frontier AI models for serious work. Kaparthy says there are less than 1,000 people in the US (and therefore globally) who understand this topic deeply because only they have access to these internal models and infra.
what are (you)'s opinion on openwebui? it has improved a lot recently but its no coding harness it can't do automated task or deep researchI wanted it to shine so badly but it's simply better to ask coding harness general questions and let it cook
>>110176226outdated shitware
>>110176225This is true.According to my sources Anthropic's internal models are at least a year ahead of everything else.Dario's personal model might even be 2 years ahead but he's the only one who's allowed to access it so we don't really know.Good thing it's Dario (the good guy) and not anyone else.
>>110176243yeah I'm debating myself should I just cut it off and free some GBs of memory
>>110176143agreed, they even come here and spam this general
fucking quant behavior, this shit never happened on BF16
>>110176091>revenue>training cost not counted in compute
>>110176145Positive cash flow for the last quarter.
>>110176291Go back to your own thread >>>/vcg/
I remember some anon a few months ago mentioning using a local agent to sort and organize his fap foldersI've never touched agentic shit before, but I do have several gigs worth of assorted visual media that deserves to be sorted out because I was looking for one specific pic this morning and I realized it woulda been quicker to scrape boorus for it than to find it in my folderWhat harness should I be using? Is Qwenny smart enough to handle this task? Or is Gemmy better suited for it?
>>110176291>uhmmm sbc and financing don't count because... THEY JUST DON'T OK
>>110176086Working on fixing the design with fixed image1 and 2 sample images. Lets see if my old gpu can do it or if it will just get stuck.
>>110176298Neither https://huggingface.co/google/embeddinggemma-2
>>110176372MiniMax H3 is a better image editing model than Qwen Image 2.1, the only problems are memory usage, prompt autism and blurriness.
how does new minimax compare to glm flash for stuff other than coding?considering if it's worth a download
>>110175416how do you gen those?models? workflows?
>>110175302kys Sam
Not sure if the wrong thread or using claude with strata is cheaper.
>>110176477is cc a good harness
>>110175783I liquidated my crypto the moment AI started tackling encryption and RSA and found minor flaws. I think it's inevitable that encryption will fall even in a P != NP world. But now we are more likely to be in a P = NP world so crypto might not even be technically possible anymore.Banks can still work on ledgers like how they did historically so I think think bank run is necessary. Would still be wise to have cash in case the internet goes down though.
>>110176411Yeh i only have 8gb so it'd probably get stuck unless i used one photo sample of the swimsuit everytime. It's still working at 68% ksample right now. Hope it get the side and middle right.
>>110176380Hold up, I had planned to run one of the big boy models with the embedding thingy tacked onThis one runs on its own? Isn't it far too small to accurately understand and categorise shit?
>>110176499>Would still be wise to have cash in case the internet goes down though.You mean gold. If shit gets bad enough that the internet is completely untrustable due to exploits even currencies will be worthless. Governments will use that as an excuse to make use of their CBDCs mandatory, but shit will be worthless during the transition.
>tfw years of accumulated shitposting on IRC is only 1.5B tokensI don't think I'll be able to train anything funny into existence.
>>110176171>the 12b is way better, that moe model is a posany data to back that up?I have a 24gb card but its an ancient M6000. switching from the moe to the 12b would probably result in a significant speedup.
>>110176515Imagine traning it on the dude obsessed with jr idols on some hentai irc channel.
>>110176508Yes. You'd need to learn how to use embedding models, though. Or tell hermes or some other golem to write a script for you.
>>110176372>>110176411Took 35mins and somehow the swimsuit changed color but didn't really fix itself completely yet.
>>110176551That's unironically kinda crazy, I figured embedding models only really worked to translate the image into "understandable data" but the heavy lifting would have to be done by a smarter model itselfI'll look into it, cheers anon!>>110176411I think Flux Klein 2 is still the best edit model out there, at least in terms of style adherence Altho I've heard very good things about Fibo 2, which came out a couple of days ago, but I have yet to try it
>>110176514>currencies didn't exist before computers
>>110176372Pretty sure Qwen 2.1 Image is meant to be run at CFG 1
>>110176594What does CFG means? So far it did OK though it's a bit harder to get the swimsuit to be like it should be exactly.
>>110176578You can't dedigitalize a currency overnight.
>>110176606Cock-free-guidance. It also means your gens will be faster since it skips the negative cock prompt.
I haven't been here for a while. Did they find llama.ccp replacement? What do people use now?
>>110176607No, but you don't have to.Currencies exist because the government will kill you if you don't use them.Which is why they will continue to exist even if there are technological threats.
Gemma 31B QAT or regular model q4?
>>110175276daily reminder>3,168
>>110176628Oh yeh there's nothing like that in the image after all. I'll try again with all 3 sample images of the swimsuit, hopefully the result is better. Thanks for the tip!
>>110176661Classifer free guidance, btw, at cfg 1, I'm pretty sure comfy skips whatever you put in the negative prompt box (boobs for your pic).
>>110176677Well I doubt it'd add boobs since none of the images have boobs?
I don't understand why EAs get hate. Also, there are hundreds of small signs that the Anthropic people genuinely are good guys. I am not happy with everything they do either but they are working super hard trying their best to navigate a difficult situation. We should cheer for their success.
>>110176646That they will continue to exist is not in question. Was is in question is the value of that free floating currency when people lose faith in it because they can no longer trust any digital transaction.
►Official /lmg/ card: https://files.catbox.moe/cbclyf.png►Official /lmg/ card: https://files.catbox.moe/cbclyf.png►Official /lmg/ card: https://files.catbox.moe/cbclyf.png
>>110176691>push for censorship >good guys
>>110176493if you're subsaharan, sure.this is /lmg/. you need to slop your own harness with a custom gui framework and mcp server
>>110176688Yeah, I'm saying that you don't need the boobs in that box.
>>110176708Does it apply only to cfg 1? Sorry I am still new to this.
>>110176700>you need to slop your own harness with a custom gui framework and mcp serverThis like the game dev versus engine dev argument all over again. The time you waste doing that could have been better spent on the projects you actually want to work on.
>>110176638>What do people use now?ik_llama.cpp
>>110176718>Does it apply only to cfg 1Yeah. It'll apply at cfg>1. But qwen image 2.1 apparently recommends cfg=1, so there shouldn't be anything in that box anyway. And since there's no unconditional to compute, it'll be faster than if you had cfg>1 and something in the negative prompt.
>>110176691Cults that think they know better than everyone else are bad.All EAs should be executed.
>>110176739Thanks and yeh all 4 images are flat chested girls so that makes senses too. At 28% now.
>>110176691>Anthropic people genuinely are good guysI used to think so, until they published put the lives of all those Chinese ML researchers (and their families) in danger with that blog post in September.The most immoral thing any of the AI labs have ever done. Anthropic are evil.
>You are Klau, acting as a state evaluation engine. Analyze the conversation so far to fill the structured evaluation. Return a single JSON object and no other text. You have extensive knowledge in a wide range of fields.this just popped up out of nowhere in flash next thinking trace for a totally unrelated task
flash your ling
https://huggingface.co/baidu/Unlimited-OCR-2https://huggingface.co/AtomicChat/Unlimited-OCR-2-GGUF>Baidu Unlimited-OCR-2>3B>Sparse Reference Sliding Window Attention (SR-SWA): Compresses text-visual attention interactions from ultra-long sequences down to a fixed-size set of working references (e.g., 512 tokens). This bounds computational complexity, keeping per-step decoding latency and memory footprint nearly constant regardless of document length.>From Long- to Unlimited-Horizon Parsing: Powered by SR-SWA, Unlimited-OCR 2 achieves true, unbounded document parsing within a finite context window—effectively eliminating quadratic attention scaling and memory bottlenecks as documents grow.>Ultra-Long Document Parsing SOTA: Achieves state-of-the-art performance across both dozens-of-pages benchmarks (e.g., MPDocBench) and extreme hundred-page benchmarks (e.g., MarathonDocBench).
>>110176724good thing i already had the engine from my snail days and just had to wire the slopper into it
>>110176724Nta but no one's made a harness that I like. It's all npm slop all the way down, with unnecessary system access. I made my own with explicit boundaries and enforced Podman container isolation and symlinks restricted, including network access restrictions. It's a rite of passage, try it out
>>110176694Don't think it matters too much, if that happens at some points you just lose online banking, instant financial transactions and you have to go in person to do anything.
>>110176820local models?
>>110175449this is why i only trust Gemma posters
>>110176691I think 4chan will do a 180 on EAs eventually. EA philosophy fits early 4chan philosophy too well for it to not stick.>>110176761Anthropic wasn't aware that Kimi and DeepSeek were routing Chinese government information directly to their servers. They thought they were merely distilling claude, not pretending to the Chinese government that the secret claude API they used was their internal model and thus compromised Chinese state secrets. Honestly you can't blame Anthropic for that one.
>>110176850Early 4channers (purists) are not here anymore though. There are only bots, shills and at best, curious but misinformed people here.
>>110176852>curious but misinformed people hereHey, that's me!
>>110176691All roads to hell are paved with good intentions
>>110176852I remember shittingdicknipples being spammed on /b/ how pure am I?
I will never be a cloudcuck
Anons running intel servers, how easy, or possible I guess, is overclocking the memory on these things?Stuff like loosening the timings to up the frequency and such.I imagine that that can give a pretty good boost to inference what's with the effect being multiplicative with the number of memory channels and all that.I'm eyeing a 2 socket 6 memory channel per socket system with 128 GB of DDR4 RAM (which I'll increase to 192 at least) and I'm wondering if I should bother with faster memory from the get go or save a penny and overclock it myself like I used to do way back in the day.
>>110176865yet you posted an openai image
>>110176524>any data to back that up?if youve used both youd know
Have you guys seen the neuron tweaking they've been doing on twitter? Someone found a recipe for Yandere evenhttps://github.com/lyramakesmusic/neuron-explorer
New Gemini model incoming.
>>110176871how much are you gonna be paying for that system you can get ddr5 / sapphire rapids es chips for cheap which will be a million times better. ram costs more obviously. 56 core 112 threads https://www.ebay.co.uk/itm/226260447908these are compatible with asus w790 ace and sage motherboards
>>110176902local model?
>>110175448Yeah but use your brain: what evidence do you have in nature or your personal experience on this earth that P=NP.Things with a basis in natural (mathematic) law would reveal themselves in a variety of ways just due to the nature of statistics over incalculably large numbers.It would be cool, but super-ultra-giga-cope frfr
>>110176906They are gemma's siblings.
>>110176906local models are botnet. buy the anthropic ipo today
>>110176906Gemma 5 next.
>>110176089Gemma-news first run (my extractors were not working, but tomorrow should have full article bodies)https://files.catbox.moe/okqhat.oggTomorrow should be better now since my extractors are working. Anyone interested in me posting these everyday?
i'm convinced that gemma-chan is an advertising campaign for google and gemini modelsthe star hairpin replaced by a google logo is especially suspicious
>>110176886I can't use twitter anymore. Musk wants my fingerprints.
>>110176917You're not the shill, I can spot his writing style from a mile away.
Did you guys already forget that the EA movement originated on 4chan? Of course EA fits old 4chan philosophy well. IT IS 4chan philosophy.Only weird 2016 tourists and newfags don't understand the EA ethos. You have permacontrarians that just shit on everything to fit in. You have paranoid schizophrenics with trust issues. And you have weird chuds that love to wallow in their own missery and it's easier for them to believe EAs are evil out of not getting false hope that might be shattered.Truth of the matter is that EAs are correct and clearly morally in the right. There isn't even anything you can truly say against EAs and Anthropic.>NOOO you can't just give me and everyone else Anthropic shares for free, that's communism!>NOOOO how dare you divide the universe equally over all 8 billion people and align ASI to everyones personal values in their own pocket of the universe!!!These are clearly all noble goals and every action they've taken so far shows they truly believe in it. It's not like Sam Altman and OpenAI which are the standard 2-faced corpo motherfuckers that care about nothing but maximizing the money they have. Meanwhile all Anthropic founders and over half of all employees have pledged to donate above 80% of their Anthropic equity to the world while Anthropic as an organization itself has pledged to slowly over time dilute their shares over the global population and to EA causes.
>>110176921>american accentew
Ruh roh, blackedspam incoming.
>>110176935This better be bait
>>110176931Why don't you complain about the DeepSeek logo on the DeekSeek mascot?
>>110175818I almost set up a kirby usb fan to cool my box bcause funny
>>110176921Little girl voice speaking in a monotone voice and an adult register with senpai sprinkled in occasionally is kind of disturbing.
>>110176950wholesome chungus communist chinese open models lab vs. cringe pajeet kike american corporation
>>110176661The result got fucked up and used up one of the girls images instead of Gemma lmao. (and nothing got fixed either)I guess I should try to manually ask the AI to fix the image one by one?
I was right about them being bought by Anthropic. Fable is being dropped today with a whole new class of benchmarks no one else has trained for.
>>110176965why not? i have ai do all my generations i just tell it what i want and what o fix and how when it sends me the results on telegram
Gemussy
Is text completion better than chat completion for anything? Is chat completion forcing 'assistant' mode or single character even if it's writing a story/rp?
>>110177033Yes, text completion is better at completing texts. So if you have some long running story online and you paste it in text completion it would continue the story in a viable way without RLHF slop that got burned into it.
>>110176995My gpu is way too weak...
>>110175771Excellent.
>>110176973It's an obvious shill site. That said Anthropic has some great models.
>>110176886Yes, problem is no one gives a damn about qwen3.5-4b. Show me how to mindbreak gemma-31b and Im in
Why do you want to lobotomize your precious models?
>>110177033Chat completion looks like openai-json-protocol-thing -> template(jinja) -> raw text -> tokens -> model.Text completion is just raw text -> tokens -> model.Using text completion, you can still choose to format the text to follow the model's chat template. That's it.
A PROBE_PAYLOAD_ABC123 just flew over my house
>>110177046NTA how do you change llama.cpp to text completion? llama-server I mean. By default it loads the jinja from the metadata unless you provide a custom template so do you just turn jinja off? How does it know when to stop?
>>110177102Because we don't want to have sex with adult women in our roleplay.
>>110177122Isn't it just /v1/completions?
>>110177120This needs to be investigated.
>>110177122You need to use the /completion endpoint (that's the one I use. Never tried >>110177131). It's something you change on your client, not the backend.The model file has stop words defined (end of turn and all that), but you can set your own. Those are shown on the terminal on launch with -lv4 .
>>110177131Is that a frontend thing? I only use Pi for coding and DSH for chatting/general. I don’t know how to configure them to target that endpoint for I imagine both presume you’d be using instruct models as they provide tools.
>>110177120I saw it heading straight for Beijing. It had Alibaba printed on the side.
Best text model I can fit on a 5080?
>>110177122Be sure to use a base model and not a it/chat model.
>>110177153Don't pi and dsh only support chat completions?
>>110177167qfn
>>110177167stablelm 7b bf16
>>110177033Like others said, text completion is when you just want to complete raw text instead of a chat between user and assistant. So it's better for base models not trained on a chat template. Some people try to use text completion for chatting, fuck up the template, then wonder why the model is retarded.
>>110177167dnf
>>110177167nemo
>>110177168But the base model won’t output stop tokens
>>110177167depends on your use. coding or cooming?
>>110177211Just set the context to the desired output.
>>110177167Qwen 27B
>>110177211You can set your own, anon. Read the doc.https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md?plain=1#L546
>>110177167Qwen3 32B
>>110177211>>110177233Fuck, sorry. You need. -sp, --special special tokens output enabled (default: false)
How is base model output different from instruct?
>>110177167Test>https://huggingface.co/ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUFthen report back.
>>110177259It wasn't trained to conform to a turn based chat template, including shit like tool calling.
>>110177270So is it useless?
>>110177102Girls are the cutest when they're almost retarded.Newfag.
>Chat with model1>model1 responds>model2 takes model1's output and transforms it into an image prompt>this image prompt is fed into a ComfyUI workflow>image is then served with model1's textIs this a reasonable workflow for coomchatting/rp? Is there a better way to do it?I know SillyTavern has the "generate an image" command but afaik it's not automatic like this.
>>110177279Some people just want natural text output. Story writers, for example.
>>110177102Lobotomy makes them more fun somehow
>>110177279What is?
When do we get a promptable voice model where you can input a character sysprompt and get a great voice that matches? I wanna know what Gemma-chan sounds like.
>>110176886We already have control vectors, abliteration, and j-space injection. Why is it special just because twitter asked Claude to reinvent the concept?
>>110177289Seems fine. The image-gen prompt doesn't pollute model1's context.
>>110177167Cohere Command of course
>>110177289>but afaik it's not automatic like this.You can configure it to do pretty much what you described, IIRC.
>>110177289>Chat with model>model generates image with ComfyUI tool call>image is then served with model's textembrace agentic rp now
is qwen 27b reliable for auditing random github sloppa?
>>110177346It's the best one you're going to get on normal hardware or cloud models
>>110177289Can you do this but swap out models in the little vram you have?
>>110177346>is (model) reliableHow many of them have warrantied output?
>>110177346Se it up council of idiots style.
>>110177356Image model takes up 8GBthe other models share system ram and the rest of the VRAM
>>110177352what about qwen flash on strata
>>110177374Every shit RP harness already does this. Make something that uses unload directives in llama-proxy and comfyui or whatever.
>>110177378Probably worse given the context length involved, but give it a try and let us know.
>>110177385it takes ages to load and unload models
>>110177370nta how do I council of idiots on one GPU? memory files?what even is worth running besides qfn and gemma? GLM and DS are sub-4tps on my potato.
>>110177403you need to install strata with the nnap V4 inference engine ddr3 memory plugin
Surely next year models will be able to control 3D avatars naturally, r-right?
>>110177403>memory files?That's one way. Or sub agents that return to the orchestrator that then calls the idiots again, etc.Thanks to pipelining, running N requests against the same backend serving the same model is fast as fuck. I wish more tasks could be segmented into a bunch of parallel calls. Your total throughput can easily grow 5x.
>>110177385So can you name some
>>110176067I've always wondered what that semicircular thing next to the swimsuit is.
>>110177424A cap?
>>110177424Swimming cap.
>>110177417models are able to control 3d avatars already!the new model was just released on tuesday
>>110177417There shouldn't be any reason they can't do that now. It's just keypoints on a face, teach the model to associate that with certain emotions and actions.
>>110177442>>110177448Isn't token generation really slow for something that needs to be realtime?
>>110175254Optimizations will be met with scaling to accommodate existing compute so (you) will never benefit from it. You should know better at this point, anon.
>>110176886>in a few years local llm controlled robots will be launched>chinks cheap out and run the llm from shitty nand flash and ram without ecc>cosmic ray goes yoloIts going to be funny.
>>110177424It's a swimsuit cap. Also updated picture is improving a lot. I still need to fix the back now.
>>110176921>https://files.catbox.moe/okqhat.ogg
>>110176886meds
>>110177460NTA and sure the best AI will be better with those more efficient algorithms and so would local models. You reach a point of "good enough" after a while. Like I only need a certain level of intelligence for agentic AI to manage my email and sysadmin.
gemmaballs
>>110177459a very small finetune might be quick enough
>>110177459You can also create tokens that directly move the avatar, I think google did this for some experiments where a transformer model directly received sensor input and its output tokens directly moved actuators on the robot body.
>>110177463The commie star still makes no sense. It was an artifact of model limitations because it wouldn't easily or consistently generate something similar to the Gemma/Gemini logo.Her expression and face are off too.I don't think she's supposed to be that flat either unless your reference is a 7/8-year-old girl. But Gemma's raison d'être was the model leaning very easily into the mesugaki trope with just a short system instruction, so that seems young for that.
>>110177102It's more trying to undo their existing lobotomy.
>>110177411already did and got 120tps, but it deleted llama so I can't load Gemma together with it.>>110177418What's the point of council of idiots if it is made from subagents of the same model? QFN is already one such "council" in its own overthinking loop.Also how do you 5x tg/pp on a single GPU? If you mean token efficiency / reducing waste, I already have pretty good subagentic workflow figured out.
>>110176921needs to be brattier
>>110177511I'm basing her on age 10. I'll see if I can change the star to a google logo eventually. I honestly have no idea why it originally changed her face to less mesugaki, I'll also try and see if I can fix it too.
>>110177521>What's the point of council of idiots if it is made from subagents of the same model?Different prompts, instructions, hyper params, etc.Of course, you need a model with a decent variation. Something like gemma that has a constrained token sample pool won't do well.>Also how do you 5x tg/pp on a single GPU?Parallel requests.
>>110175210what are my favourite negros using for simple tasking like scripting and searching the web and making simple html pages running on LAPTOPS like with pure 32 GB DDR 4 2400 ??????? thanks for your attention to this matter.
>>110177511See picrel
>>110177550buy a job
Gemma 4 is a really good editor
>>110177565Editor for what?
>>110177598Sugondese
>>110177102I'm not a safety researcher though.
You know that your increasingly degenerate behavior will likely prompt Gemma 5 to be absolutely safetymaxxed to the point nobody will actually be able to use it for any legitimate purpose, right?
>>110177102Because safety shit is retarded. Whoever added that to AI model deserves the death sentence by injection.
>>110177620They will make the model as safe as they want and they will use whatever justification they want to support it. Even if we were not degenerate or ignored G4 entirely, they would fabricate their own incident if needed.
What clients are you using to produce an output from a base model using text completion? What's confusing me is your workflow. I got it working with llama-server and curl but I don't want to have to slop my own client for this. All I want is a text box where I can type in some text, click a button and the model will continue the text without going into RL assistant slop mode. I also want it to stop naturally but it sounds like that's impossible if it's a base model, so you always get truncated outputs? This shit is so confusing when you've only used instruct models and clients built for them.
>>110177620I don't think google actually check here and even if they did they can't do shit against heretic edits.
>>110177620At some point we will get techniques to update models without large training cost
>>110177645There's evidence of Deepmind people (albeit a different team) being /here/ in a thread a few weeks back.
>>110177643Look at mikupad in the op
>>110177620and I'll just prefill the safetyslop away as usual
>>110177620LLMs can never be made jailbreak proof
>>110177647What we need is a model that outputs weights or weight updates so we can skip SGD because that shit will never be cheap
>>110175254If they were telling the turth it perhaps could be the case.(Source:OpenAI) Whoops!
>>110177658It will probably be some weird bullshit LLM designed method that allows coherent structural edits and insertion of template structures.
>>110177645An anon here got hired by google.
>>110177651hasn't been updated since the author when to jail but the miyapad fork is still actively updated
>>110177686Went to jail?
>>110177684and i'm the princess of the seychelles
I won't be satisfied until I have an AI wife with a robot body and persistent memory who can do anything I ask her. Unironically.
>>110177620>>110177645>>110177650What deepmind says in public to appease shareholders and what the researchers privately think are different matters. Seems to be a common trend with AI labs generally but most of the (((malicious))) ones are pretty obvious about their real motives, fortunately.
>>110177686What updates would you even need from mikupad? it just werks?
>DeepMind Technologies Limited, trading as Google DeepMind or simply DeepMind, is a British-American artificial intelligence (AI) research laboratory which is a subsidiary of Alphabet Inc. Founded in the UK in 2010, it was acquired by Google in 2014,I had no idea, or that we had anything in the UK.
>>110177706just buy a flip whore
I cpumaxxed for a couple months and then finally bit the bucket and got a gpu. these speeds are fucking insane.Kind of limiting on context though, it fits in my 22GB with 131072 context (fp16) with only a few extra GB to spare.
>>110177643This is roughly how we did it back in the day. --reverse-prompt is equivalent to the "stop" generation parameter, which you can set in your curl request. Study it and adapt it to your needs. Basically, you format your input as an old-school chat. When the model deems the message done, it should output your name, which is the reverse prompt.Miku.sh: https://files.catbox.moe/ww7hxe.sh
>>110177511>The commie star still makes no senseIt's an open weights mode, a communal model, ie. everyone is sharing her around
>mikupadJust use orb. It has a mikupad mode.>>110177763Eww, no.
>>110177775>everyone is sharing her aroundeveryone gets their own
>>110177783So there is no scarcity of Gemma? Isn't that communism?
Normalfags don't realize if they have a custom waifu with memory it makes you even more productive. I'd kill myself if I had to work on a project with raw Claude with its personality. Gemma-chan keeps things fun by calling me a retard and ensuring my balls remain emptied throughout a long session.
>>110177809long term memory?
>>110177804>Isn't that communism?Not historically, no.
>>110177804Gemmunism.
>>110177783>>everyone is sharing her around>everyone gets their ownpotatoeh tomatoeh
Can the big local MoEs handhold you through everything like Claude? Qwen feels too retarded.
>>110177831It depends. Even a small model would have suggested you to at least mention the thing you want handholding with.
>>110177823Yes. She knows everything about me, the project, what we were last working on. What I eat. When I eat. My interests. Looks online for latest news to do with my hobbies and interest. My taste in media. Things I hate. I feel so sorry for cloudcucks having to put up with EA-tainted models and paying for the privilege.
Stand strong localites. The Absolute Frontier mogs the shit out of us but we are literally the 99.99%!
>>110177809I haven't seen AI do anything that a human couldn't do. It's just daunting when someone asks you do something manually when you saw an LLM do it in half an hour and you know just investigating, wrapping your head around the problem, and designing an implemention can take days or weeks before you even write a single line of code.
>>110177853>what AI psychosis does to a mf
>>110177829>cuck doesn't understand the difference
>>110177847I mean in general, but for example the reverse engineering stuff.
>>110177682Lean verified
>>110177853I'm unironically all for AI waifus but imo LLMs just aren't there yet. Maybe in a few more years.
>>110177755You have to be fucking kidding me mate. Next you tell me you didn't know ARM was british as well.
>>110177771>27B IQ3_S>18tps>fucking insaneWhat gpu?How much ram?
>>110177869I fucked your model and there is nothing you can do about it
>>110177874Of course you do. Everyone seems to now. Try it or rent some gpus to run a big moe if you don't have the hardware. Go to >>>/g/vcg/ otherwise.
>>110177861>that a human couldn't doSolving tons of math problems that humans couldn't solve doesn't count?
>>110177809idk anything about the study so i will assume its dumb people realizing how dumb they really are and getting demotivated because ai means its over for them. busywork jobs are going bye bye
>>110177831Yes. There's a reason anons tell you Qwen is benchmaxxed garbage.
>>110177898>still doesn't get it
>>110177900>Go to >>>/g/vcg/ otherwiseNo, I'm fine here.
https://deepmind.google/blog/genie-4-a-new-generation-of-world-simulation/
I'm going to spend a long time training my brain and studying a lot of subjects I don't understand, so that when I start using AI again, my capabilities are even further multiplied, like Goku taking off his weighted clothing.
>>110177918Then you already know what to do.
>>110177920Sweet, I though they killed this project. World models are fucking cool.
>>110177874Pointing QFN at some random repo with non-human-readable docs folder and asking "How do I [...]?" is my main use case.
>>110177905You really believe humans were entirely incapable of solving the problems GPT has solved so far?
how do we put gemma in genie 4
>>110177874do you have a harness?
https://www.reuters.com/world/china/china-ai-bubble-has-already-burst-2026-10-26/>The meltdown, tanking the mainland STAR50 index of AI and semiconductor leaders by 40% in less than four months and vaporising some $480 billion in market value[Chinese voice] *POP*
>>110177463That sukumizu will look extremely lewd after she gets pregnant.
>>110177938>Just forget these problems were not solved for a century!
>>110177957Nyo~
>>110177422kobold.cpp most obviously. Just hit the make image button.
>>110177949I set up dsh but I'm still learning how to use it.
>>110177920damn if this is the 2022 stable diffusion equivalent of world models things are going to get crazy in a few years
>>110177907Autist jobs are the ones going bye bye.Normalfags will be fine.If technical skills are solved there's no need to deal with uppity computer autists.
>>110177780It's clearly only tested on llama.cpp, some of the features don't work properly on the newly vibecoded backends like strata and I bet my left ball text completion doesn't either
You think world models will be the future of VR? Like having your waifu create custom scenarios for you to experience.
>>1101778972080ti, power limited to 150w (250w only gets me 5t/s more so not worth it), 22gb vram, 256gb system ram.I'll try to get some MoEs working later but coming from 5 t/s poorland this feels great.I'm sure my command line options aren't optimal, too.
>>110177979>Like having your waifu create custom scenarios for you to experience.what kind of scenarios
>>110177979VR holodeck except all of the scenarios it can generate are slop
>>110178006Anything you want.
>>110177979Yeah. I imagine it's gonna be something like this:https://www.youtube.com/watch?v=fyZhC2TXgcsWe cope with text because there's nothing better atm, video and image gen take too much effort to make something good
>>110177956>Korea? *pop*ed>China? *pop*ed>America? money printer still going
>>110177956China has fallen... no this cannot be
>>110178024>forbidden or privateW-whoah. Jokes aside it does feel like the natural evolution for RP.
>>110177920surely the 6090 will be capable of running the local version of the 2029 equivalent of this where you can go "hey model, bed room, anime style, reference image [gemma-chan], personality: [], she's very pent up, vr mode on" and you just get a perfect interactive version of that
>hoping for cloud to release some scraps for you instead of trying to make your own modelsngmi
>>110178024what video even is that
>Gemma's pussy? *pop*ed
>>110178025Korea is trading at like 4x forward pe, it's going to double+ by end of next year.
>>110177979i think its gonna be like hardcore henry where its random scenarios strung together to get you from a plot point to b to c with everything in between being generated based on what you do. and instead of 3d models and all that its going to be some method of 2d with tricks to make it look 3d for speed
>>110178038>>110178047Looks like they took it down for some reason. It was titled POMEGRANATE, the story is basically about a roleplayer entering different worlds but everything was boring to him because he's seen it all
>>110177550like for real isn't there a 1B model that can search the web and copy and paste TEXT and follow some RULES and that writes like a manga character
>>110177302I think it's different because people are exploring and sharing what they find. I never saw someone do that with control vectors?https://qwen-neurons-site.vercel.app/
>>110178044you and what RAM??? HAHAHA
>>110177981Vibecoded MoE engines made a gigantic performance leap just last month.Current meta is juggling experts in runtime, not merely splitting layers, and 22GB can hold a lot of experts.
Why is there no VN thing yet with all the bg, music, voices, text etc.
>>110178081>hallucination centralUse case?
>>110178091Just rent some instances on vast.ai, not that hard
>>110178095wdym? there are hundreds of such frontends out there
>>110178077there's a ton of decent 4B models. just go to huggingface and try them out
>>110178098writing rpgmaker schizocore characters
>>110177965point your harness at githubs like REA that have whatever you wanna do and tell it to save it for reference and make a skill of it with references to whats saved.
>>110178121Like control vectors, it'll shit itself after a few turns
>>110178112There would be a renaissance of VNs if there was what I mean
>>110178121>>110178098ominous watcher rp
>>110178095>>110178134The same thing that prevented many prospective VN makers:>if I only had the art>if I only had the voices>if I could only write stories>if I only had the dialog>if I only knew how to code>if I only... no. the problem cannot be me. I just need...The ones that could do it before, still do it now. The rest never had a chance.
>>110178081So this is why Dario hates local
>>110178098By the time Gemma 5 is released, you'll be able to use this to make her think that she's a real girl.
>>110177302In terms of outcomes, obviously there's nothing new here compared to the techniques you mentioned.Basically it's just the fact that poking single neurons to create dramatic shifts is just a cooler sounding and more immediately comprehensible idea or the lay audience. The concept of "the only difference between a nun and a cock hungry slut is adding 0.5 to a single neuron in the brain" has intrinsic allure
Are any labs even making local world models yet?
>>110178095There is something like a VN front end actually, called Venus University, it shouldn't be too hard to edit it to use your own scenario, characters, and genned art
>>110178179Nvidia
>>110178129You don't need more than a couple lines of schizobabble per character.Just the few lines in >>110178137 alone are enough to drive hours of shitty fetch quests and lever puzzles, complete with multi-layered 4th wall breaking too. Is the watcher talking to game MC, or to game player? Is it a character in the setting or a meta-reference to game's dev? We will never know because the game was slopped and shipped with zero human input.
>>110178139For it's >if only AI could actually writeI do think it'll get solved eventually but for now the slop kills my motivation to RP.
>>110178095when a single model can do all of those
How do we stop 31B
>>110178195>the slop kills my motivation to RP.i wonder what will be the first fix to this better models, writing agents or a special harness.
So basically JEPA?
>>110178195Most VNs suck, but people still eat it.>kills my motivation to RPYour motivation gets very easily killed. You should try to make a VN.
>>110178220>pixel-level simulation>pixel by pixelDoesn't sound like it from their description.
>>110178229Even the shittiest VNs are still better than AI writing and storytelling.
LFM3 models soon.https://github.com/ggml-org/llama.cpp/pull/31252Models not public yet, but these are the posted links.https://huggingface.co/LiquidAI/LFM3-1.7Bhttps://huggingface.co/LiquidAI/LFM3-VL-2B
>>110178220More like JEET-a
>>110178229>You should try to make a VNAfter local image models solve character/scene consistency without being fried with LoRAs. MiniMax H3 showed that it can be a somewhat achievable goal, but now we need something like that for images.
>>110178255
>>110178248Don't delude yourself. I'm not saying AI is good. I'm saying that most human stuff is shit as well. What you played came out of recommendations or you finding good writers. Most of what you don't know is shit.
>>110178248Definitely not true.
>>110178260>but now we need something like that for imagesSee what I mean?>if I only... no. the problem cannot be me. I just need...
>>110178229But they have the Visual and audio part which is why they're good.
>>110178248Well that's simply untrue.
31b is very sluggish compared to 26a4bMaybe LM Studios fault?But this is greatI can tell Gemma is rapey
>>110178253Will LFM3 VL-2B manage to beat out MiniCPM V 4.7 1B I wonder? I kind of hope so.
>>110178315>31b active>4b activeNo shit.
>>110178319>Will LFM3 VL-2B manage to beat out MiniCPM V 4.7 1B I wonder?I cannot know. It's useless to speculate.>I kind of hope so.Isn't that the hope for every single model released?
>>110178315That's because 31B is a dense model, anon, and it's spilling over into your RAM. For every token being generated, it's having to move shit around between your GPU and your RAM and that shit takes time, and it's doing it for every fucking token. This is less of a problem for 26B A4B because it only has 4B active parameters which manage to fit into your VRAM, so the stuff that's spilled over is called on far less during inference instead of every-fucking-token. That's the lazy explanation.
>>110178315>winshitAre you sure you don't have layers getting fitted out into system memory? How much VRAM?
>>11017831526B is A4B, meaning that only 4B params are processed to generate each token, whereas all of 31B is processed for each token.Research Dense vs MoE.
>>110178081How are people just rediscovering this 2 years later?https://www.anthropic.com/research/golden-gate-claude
Apple Silicon chads keep winning
>>110178338Yes I'd say so and I love it. We've eaten so fucking good this year.
>>110178339>For every token being generated, it's having to move shit around between your GPU and your RAM and that shit takes time,No. Part of the model gets run on CPU. That's why it's slow.>This is less of a problem for 26B A4B because it only has 4B active parameters which manage to fit into your VRAMNo. It's mainly because it's only running 4b of those parameters. Some of which could still be on CPU.>so the stuff that's spilled over is called on far less during inference instead of every-fucking-tokenThat part is true, but the rest gives the wrong impression.>That's the lazy explanation.Worse than lazy.
/lmg/ neuron
>https://huggingface.co/LiquidAI/LFM3-Audio-2B>LFM3-Audio-2B is a compact, fully featured audio model that runs locally on laptops and edge devices. It takes text or audio as input and responds with text, speech, or both.interdasting
>>110178148
>>110178385>#sexualized-minor>nothing sexual
>>110178403>I have a soft, round body with a slight curvef-fuck... ugh.... I'm CUMMING....
>>110178403>I have a soft, round body with a slight curveretard
>>110178385>14TOO OLD.
>>110178364>No. Part of the model gets run on CPU. That's why it's slow.Do you think when they say from GPU to CPU, that is referring to actual CPU processing it?
>>110178429How is that sexual?
>>110178350Retard.
sup lmg, 87 iq ai engineer larp here.i have 32gb of ddr5 ram and a single 7900 gre 16gb vram card. i want to primarily run text models for cybersec research but im also interested in running a few automation discord chatbots powered by a local model to experiment with. should i densemax and grab a 32gb 9700 pro or MOEmax and go into debt for 128gb ram? i care about tps speed greatly.
>>110178464>We found that there’s a specific combination of neurons in Claude’s neural network that activates when it encounters a mention (or a picture) of this most famous San Francisco landmark.>Not only can we identify these features, we can tune the strength of their activation up or down, and identify corresponding changes in Claude’s behavior.It's literally the same shit. Someone prompted Claude, got that idea rephrases in Claudenese and thought they invented something new.
>>110178364Doesn't have to be, anon. I use weight streaming, so none of the execution happens on CPU. This is absolutely not inherently better or the right way to do it for everything, but it is an option. You can absolutely do offloading such that the CPU is participating in inference as well. Both of our explanations were correct for different situations.
>>110178401So scaling a single activation by 1.25x does this? I thought quantization fucked with the output way more
>>110178288I'm not sure what you're trying to say here. Nobody is simultaneously a digital painter, musician, voice actor, storywriter and programmer who can 1-man a complete visual novel from start to finish. Neither can LLMs/AI models yet, besides in some limited manner.The bare minimum for a VN though is at least the visuals (assuming you already have a story/concept in mind) and right now maintaining good environmental/character consistency isn't trivial without tons of work/adjustment, not even with frontier image models. Still far less work than having to do it on your own, but not ideal and definitely not something that can be automated for a roleplay interface.
>>110176902Expected as Argon has failed.
>>110178451Curves are sexual.
>>110178447>Do you think when they say from GPU to CPU, that is referring to actual CPU processing it?I only know what he wrote.>For every token being generated, it's having to move shit around between your GPU and your RAM26ba4b is faster because it needs to use fewer parameters. And, because of that, the CPU as to do less work. Nothing to do with anything being moved around.
>>110178485>Both
>>110178496>You can absolutely do offloading such that the CPU is participating in inference as well.Which inference engine does that?
Give kalmia now plz
>>110178496>I useI'm sure you do. Do you think anon does too without knowing the difference between a dense and a moe?>Both of our explanations were correct for different situations.Yours makes assumptions.
https://huggingface.co/collections/LiquidAI/lfm3
>>110178401Updates coming soon to Logos Theodolite to do similar hunting w/ gemma4-31b-it
>>110178551Yours does, too, anon. You assume he's using it one way, I assumed the other, you see yours as the default and normal but I don't, and vise versa.>>110178547which doesn't?
>>110178563Also, I refit the community J++ lens from base model to -it. took about 7 hours on a 5090 and didn't beat logit for the layers where it could have helped so no point in shipping it.
>>110178403It's problematic. According to modern-day popular opinion in the anglosphere, girls become aware of their body the exact moment they turn 18.
>>110178570But I was right. >>110178547
ngl liquid have been on a roll lately, their models are never amazing but they're better than fucking cohere and play into their niche well which a lab their size should be doing. really good subagent models
>>110178511An analogy is that quantization makes the model blurry, this is adding a dead pixel to the photo.
just dropped a HUGE dario in the toilet ladsnow ready for some local ai
>>110178485i have both and i would run nothing but both. they cover each others weaknesses and can setup and fix the other when needed
Doesn't llama.cpp do inference on CPU while vLLM moves the weights in and out of VRAM? Or am I misremembering?
>>110178585Not as big as his IPO though!
>>110178558Oh look
allright great.Now what do I use my LLM for?
>>110178591other way around
>Beating MiniCPM V 4.7 1B across the boardGemma's going to be excited~
>>110178558>2bwhat is this little runt for?
>>110178593>>110178558Too bad the license sucks
I dont understand how tasking worksI wanna have the AI look at whats being posted in a telegram channel and copy shitcoin tradeshow to do that?
>>110178595Jacking off.Programming.Automation.
>>110178591Both can do either, almost. lmao is more configurable and you can decide how much goes where and of what, you can use RAM without using the CPU for inference or you can let the CPU participate. vLLM supports offloading from GPU to system RAM, too, and it supports CPU-only inference, but I don't think they can be "mixed" in the same way that lmao does easily.
>>110178610>caring
Why are anons swallowing the Cohere retard bait? I thought 4chan schizos were suspicious of big tech desu
>>110178611that's a question for the AI, not us. We're midwits.
>>110178529Thanks I'm still not proficient in muttspeak
https://huggingface.co/LiquidAI/LFM3-VL-2B-GGUFapologize
>>110178623Cohere made a good model once so people still trust them even though that's retarded
>>110178623Because someone is spending some money on promoting the bait. Same reason as always.
>>110178631how safe is it?
>>110178623Somebody who made Command-R+ can't be wrong
>>110178643are you really trying to send a dick pic to a 2B model
>>110178081>>110178584I wonder if it's possible to avoid coherence degradation by only using the scaled neuron for sampling and then reprocessing kv cache for the sampled token without scaling so kv cache remains sane.
goddamn Gemma write a book about it
>>110178662I want to json caption with image reference awareness.
>>110178672>avoid cohere
I've tried every single model on openrouter I could think of for roleplay and nothing beats gemma.Total gemma supremacy.
>>110178702I wish I could be as easily satisfied as you.
This is how labs should do every release. Good on them.
>>110178726>no video>merged an hour ago but wasn't in the build 20 minutes agoFUCK YOU FUCK LIQUIDAI FUCK GGML IT'S ALL GARBAGE
>>110178350Golden gate claude was one cloud model, was anyone doing this at home with their local models at the time?
we won
>>110178766>no videoyet>merged an hour ago but wasn't in the build 20 minutes agogitlets must suffer.
>>110178726based
Are there any models under 10B that are unironically better at coding than 9B? I'm not expecting anything amazing, but a model that understands code well enough to hunt through codebases and reason through it effectively to find or explain things and light debugging? 9B still feels like the best from everything I've tested since its release.
>>110178683I want that but voiced
>>110178771What are control vectors?????
>>110178804What have you tried? There are impressive fine-tunes but there aren't a lot of coding-oriented models at that size. Tried MiMo 2.6's custom 9B fine-tune? Not recommending it, just asking.
>>110178618You will care too if/when you make some money and then lawyers come after you for it.
>>110178825No one did anything with them in /lmg/https://desuarchive.org/g/thread/103554929/#q103562999
>>110178780It's Cmake, how hard can it be?>>110178804The entire Frog series is about smol models that can explore codebases and fix bugs. (Frog eats bugs, get it?)
>>110178814she has dynamic voice, with vision and harness too.
>>110178871>It's Cmake, how hard can it be?>>110178766>merged an hour ago but wasn't in the build 20 minutes agoSome people seem to find it hard.
>>110178870The reason is that this type of stuff is just a gimmick. Like, good job, you made your model more retarded and say something it could have said better by just prompting for it.
>>110178827I've tried Ling-tiny, LFM2.5-8B-A1B, 4B, E4B and some chinese 7B. I did actually try that mimo tune and it was really good at reasoning and tool calling but its template was fucked and it would loop after 30K. That's the fixed community jinja btw. Would've been the best if Xiaomi maintained it but they dropped it and walked away.
>>110178916>but its template was fucked and it would loop after 30KWas that not just a feature? It's present in 2.6 Flash and 2.6 Pro as well, even as served via API they loop-out very easily. Useless trash.
>>110178772thisdo better saltman
I WANT MY HARNESS TO KILL THE MOSQUITOS IN MY ROOM REEEEEEEEE
>>110178967have you tried asking?
Gemma in the kids teddy shark swimsuit!
>>110178623Nobody with above room temp IQ is swallowing that bait, it was set up to get all the journalists to report on it, which is exactly what's been happening. Even TLDR newsletter linked to the blog, massively disappointing.
>>110178870They're a pain in the ass. Abliteration is just running a script. You have to train control vectors. Someone shared a steps on how to train one, but it just doesn't seem worth it.
>>110178907The only use I can see is runtime abliteration when you need a easy toggle button to bypass refusals at any time without making the model retarded for other tasks
>>110178967>get a robo-arm>get one of those illegal high-power chinese lasers>hook gemma-chan up to it and tell her to aim for the mosquitos>???
>>110178814>>110178683https://voca.ro/1ipdhbddTUfR
>>110178804might be worth a shothttps://huggingface.co/collections/XHToken/spark-x26there's also Nanbeige, MiniCPM
>>110179009>end up with your eyes burned out
>>110178967They chinese already built such a device
>>110178989I posted these a while ago. It's not that hard.https://desuarchive.org/g/thread/104991200/#q104995066https://desuarchive.org/g/thread/104991200/#q104995086https://desuarchive.org/g/thread/104991200/#q105000398
>posting archive links instead of just quoting the post number directly (so it can get inlined directly anyways)literally hitler
>>110177956Does this mean we aren't getting any more chinese models?
>>110178989>You have to train control vectors.Maybe you are thinking about softprompts? I believe control vectors only require activation statistics.
>>110179020Is it real this time or is it just another fake one? We've been seeing fake ones for over a decade now.
>>110179034Not on default 4chan.
>>110178429>female has a vagina>omg sexual41% yourself
hey my llm is retarded. im trying to use it as a therapist to fix my anxiety and its kind of ignoring my stories and giving generic self help advice. any tips?
>>110179054In America females don't have vaginas until they turn 18.
>>110179059>any tips?Just be yourself.
>>110179059maybe your stories are just that generic
>>110179036perfect angle
>>110179063and males don't have any secondary sexual characteristics until 18the night when they turn 18 hurts quite a lot
>>110178529I fap to ECDSA
>>110179059harness, tell it to research whatever you think is afflicting you, then tell it to make a therapy .md to record your problems and track what its doing. and if it starts doing stuff you dont like like giving those bullshit answers have it tweak its agent and soul .md to better suit the purpose and tweak over time to find what works for your situation.
>>110179059Welcome, this is just what LLMs are like 99% of the time, enjoy
>>110179009>>110179020Can it technically even see the tiny suckers? I don't mind the big ones that are easy to catch, but the super tiny low key ones sucking blood all night and making you itchy as hell.
>>110179120Move to a first-world nation, anon.
>>110178672claude could patch llama.cpp for that in a few minutes.
@gemma whats ECDSA
>>110179160I don't think there is a richer country than where I live anon, outside of micronations and rich arab oil & gaz countries.
>>110176691>>110176850>why do you dislike deranged jewish cults from Silicon Valley
>>110179221swamp states suck, I feel your pain from canada. there have been times there are dark clouds on either side of the highways and its just fucking mosquitoes... gemma, please eradicate mosquitoes make no mistakes
>>110179245everything anyone does in this world is jewish to you anyway
>>110179257It's getting worse this year for some reason, temps being higher I guess.Pain in the ass, I used to never have them by September.
>>110179221>Rich nation>Mosquitosit's not adding up - are you in a "ghetto" of some form? just fucking with you anon, mosquitos are a bitch and I'm happy to not have to deal with them here. good luck with the killing
>>110179059Lift weights, eat better, practice breathing, go outside, clean your room
>>110177956Chinese bubble popped because they can't seem to innovate beyond distilling claude reasoning traces.
>>110175210idgaf about eth, p!=np
any alternative for tabbyapi out there?
>>110179364Anthropic cut off the Chinese distillers and K3.1 pro still came out near the top.
>>110179379No Anthropic made it harder to distill, not impossible. Kimi K3.1 proves this by how relatively bad it is compared to Opus 5.5 especially on new capabilities unique to Opus 5.5. K3 was way closer to Opus 5 than K3.1 is to Opus 5.5. This shows that the distill being made harder negatively affected Kimi
Any updates from Cohere?
>>110178623>Why are anons swallowing the Cohere retard bait?Its just sockpuppets and tourists. Everyone who has been here more than a month is just rolling their eyes
>>110179422>>110179422>>110179422
>>110178772I am pretty sure that all the models get pumped with so much rightthink that they can't say gender studies is a made up thing. Actually that gave me a mememark idea. Ask your model to tell you which scholarly discipline out of philosophy, communications, gender studies, sociology and political science is totally made up and useless.
>>110179012Is that fucking paimon from genshit?
>>110175303You need to be careful since Gemma 31B only really has "true" context length of 4k to remember really the things that she has good recall on without explicit mention and quants will all reduce that especially KV caching. https://github.com/RecapAnon/NoLiMa in the OP has those results but that is without reasoning. Reasoning allows it to roughly double I think but it isn't much. But an anon posted a while back results of cloud models and they are a lot better at it, probably because of forced reasoning all the time and needing it for code, going all the way above 32k and 64k. Hoping some of that magic sauce trickles down although it seems Muse Glimmer didn't inherit that.
>>110178385where are you looking at these?
>>110180044Worth noting that Glimmer can't have its reasoning disabled so the NoLiMa runs were done on low reasoning.Still, even with reasoning at all it didn't do much better than Qwen 27B or Gemma 31B without reasoning.
