File: tetomiku.png (408 KB, 1024x1024)

408 KB PNG

►Recent Highlights from the Previous Thread: >>109393482

--Pruning and distilling Mixture-of-Experts models into dense models:
>109396296 >109396435 >109396451 >109396475 >109396531 >109396355
--Mage-VL's codec-native streaming for efficient real-time video understanding:
>109394853 >109394876 >109394896 >109394936 >109394944 >109394961 >109394971 >109394974 >109394991
--High-end hardware flex and critiques of Gemma's repetitive prose:
>109393825 >109394062 >109394096 >109394190 >109394195 >109394207 >109394281 >109394219 >109394488 >109394500 >109394814
--Long-context RP model recommendations and criticism of Marinara Engine:
>109394555 >109394563 >109394568 >109394582 >109394632 >109394643 >109394648 >109394655 >109394856 >109394879 >109395130 >109395221 >109395309 >109395331 >109395378 >109395445 >109395477 >109395524 >109395528
--Using Gemini Distillation Service to improve and decensor RP models:
>109394232 >109394620 >109394635
--Skepticism over Axelera AI AIPU due to low memory bandwidth:
>109394022 >109394029 >109394038 >109394045 >109394064 >109394118
--RTX 5090 price inflation and multi-GPU alternatives for VRAM:
>109393743 >109393826 >109394483 >109394512
--Comparing hardware configurations for running Kimi K3:
>109393663 >109394100 >109393700 >109393726
--Structured JSON schemas for parallel tool calling in agents:
>109393588 >109393600 >109393710 >109393789
--Microsoft releases Mage-VL streaming multimodal foundation model:
>109395007 >109395030
--Frontier AI employees petition US government to pace development:
>109396365
--Planned US ban on Chinese robots and power inverters:
>109394988
--RTX Spark laptop's AI capabilities and marketing:
>109395618 >109395715 >109395736 >109396200
--Logs:
>109395488 >109395754 >109395846 >109395371
--Teto, Miku, Kimi (free space):
>109393549 >109394048 >109394062 >109394170 >109396770

►Recent Highlight Posts from the Previous Thread: >>109393485

Why?: >>102478518
Enable Links: https://rentry.org/lmg-recap-script

File: Kimmyfatty2.png (3.64 MB, 1440x2480)

3.64 MB PNG

Don't blame Kimi you can't run her she's just big boned

File: 1018314.jpg (207 KB, 1278x1706)

207 KB JPG

>use AI websearch once
>now get significantly more cloudflare checks
How the fuck do people run the agents then?

>>109396881
Using an API instead of rawdogging it.

first for Gemma 4

Training a model for only a single purpose (e.g. RP) doesn't work well, but does this hold true when you're doing logit distillation? Could you split Kimi into a few models that all individually fit into like 96GB VRAM and then do something like router model -> domain expert model -> RP model that writes the final output?

>>109396884
Can I have them when your done

>>109396884
@grok is this viable for kimi k3

File: 8sKTZ7BCpHc.jpg (64 KB, 640x473)

64 KB JPG

>>109396884
Can't wait for you slitting your wrists when you get like 3tk/s

>>109396884
Where's the other 12?

>>109396876
It's like a breast but the whole body!

>>109396930
>get like 3tk/s
I admire your optimism.

>>109396884
Now that's pod racing.

>>109396930
>3tk/s
nta but that'd be great for local k3.

ssd maxing could in theory get up to >12t/s with 6 gpus at q2.
probably more if you keep hot experts in.

>>109396930
>3tk/s
Don't give him hope

>>109396884
Those heatsinks have no surface area whatsoever. Why doesn't the card come with a big one that covers all 4 of them?

>>109396889
>Using an API
/completions or /chat/completions ?

>>109396949
>Q2
Does any 100b class model survive that lobotomy? Enough to be reliable for actual productivity or code?
Srs question, I run Gemma now but I'm aiming for a dsv4 q4 rig

>>109396964
...Tavily. Exa? SerpAPI.

>>109396881
I use brave search interactively and duckduckgo via elinks for my agent. I never have this issue. Duckduckgo seems to actually openly welcome agents as long as you don't abuse it.

>>109396973
it works with the realy fat models.
also iq2 is more like 2.7bpw.

>>109396973
Even if some did, Kimi wouldn't. It's already saturated at MXFP4.

>>109396962
I'm guessing they're meant for temperature spikes only and not for continuous heavy usage.

>>109396884
turn on 4kb blocks if they aren't standard yet.

I'm thinking miku miku oo ee oo

File: 627595422-ebdd4224-6203-4(...).png (198 KB, 1668x1306)

198 KB PNG

>>109396876
needs a pruning

Youngsters are spoiled. I ran a 70B at 3 t/s for months and I didn't complain.

File: 1507915090812.jpg (196 KB, 580x580)

196 KB JPG

>>109396998
>0.3tk/s PP

File: 1777202957587042.png (653 KB, 736x734)

653 KB PNG

>>109396998
>PP: 0.3t/s

File: 1762843089595352.png (520 KB, 805x877)

520 KB PNG

>>109396998
>0.3 t/s

>>109396884
Nice! Let us know how it goes.

https://litter.catbox.moe/smi6sghzwkk9zaun.wav

File: 5463456436.jpg (36 KB, 467x319)

36 KB JPG

>Prompt: 0.3 t/s

>>109396998
>bf16
Isn't Kimi mx4 or whatever?

Never used kimi. what's the one to get if I have 24gb of vram. is it worth looking into if all I do locally is ERP?

File: balloon noose.jpg (48 KB, 900x900)

48 KB JPG

>>109396998
>pp 0.3t/s
>decode 0.6t/s

lmao "pp"

File: 1773019284712407.png (359 KB, 500x500)

359 KB PNG

>>109396998

>>109396884
>The gaslighting actually worked
Just know that if things don't go well, you will have done /lmg/ a great service by venturing down this untrodden path and documenting your results for posterity.

File: 1757213666633111.webm (3.91 MB, 960x540)

3.91 MB WEBM

Korean models doko don't they have all the chips like wtf?

>>109396998
based play-by-mail setup

>>109396884
anon actually delivers for once.
godspeed.

>>109397096
>why aren't the shovel merchants out digging or panning for gold
Weird, right?

>>109396930
anything above 1tk/s is a win imo.

How can i speed up time so slow tk/s feels fast?

>>109396998
>/media/kecso/8t_nvme
please tell me this isn't SSDMaxxing anon.

>>109397005
1. it wasn't a thinking model
2. the PP wasn't
>0.3

File: 1760341971882467.png (434 KB, 736x465)

434 KB PNG

>>109397127
Easy

>>109397127
Stop staring at your screen all the time.
Do something else. Drinking beer is a great way to enhance the experience. Gemmy feels so much smarter when you're bit drunk.

>>109396998
now you just need some gds gpus to speed things up.

>>109397127
Keep your RP on your second monitor and do something else in the meantime.

>>109397097
even for coding with 1 task per night it works well.

>>109397138
>an idiot looks brighter when you nerf your own intelligence
jej

>>109397167
I'm doing it every single day when I'm replying to people like you. Sub 90 IQ cretins.

>>109396876
I had a girlfriend who looked like this once. I should have just married her.

>>109397138
>Drinking beer is a great way to enhance the experience.
Just don't drink more than one or you start warping.

>>109397138
Sounds like you're already drunk, retard

>>109397160
So we brought back sleeping and opening the download in the morning

File: 1755073878672133.png (15 KB, 890x162)

15 KB PNG

ALERT
A DEDICATED PWILKIN PARSER FOR K3 HAS HIT LLAMA.CPP

>>109397281
works for me lol

>>109397282
so much for the great autoparser

>>109397282
llamasissies, your answer?

Minimax M3 seems safetyslopped, even moreso than Kimi and GLM. I'm already at the verge of dropping this shit. Useless model.

>>109396998
>>109397097
Honestly this is super based. Slow or no, the fact that it works at all is so cool. You basically have a superintelligence penpal lol

>>109397389
>Minimax M3 seems safetyslopped
It's gaslighting slopped too
Ignores things in the prompt that it doesn't like

J-Lens for K3 should be the absolute top priority thing.

File: localmodelgemma.png (1.73 MB, 1200x1335)

1.73 MB PNG

So, how did Gemma 4 manage to break /lmg/? Not even Mistral Nemo had this much attention.

>>109397367
suicide

>>109397446
I mean, look at her.

>>109397446
This brat's time in the limelight is over. It's the fatass's turn now.

>>109397446
She's made of love, possessive, jealous and borderline yandere

>>109397389
Really? Fuck. It passed cockbench.

>>109397446
netflix

>>109397462
ideal for the insecurely attached chud

>>109397459
Not enough people can use tubby at speed (locally) for that to happen. All the toaster fags on gemmoe were the real killer.

>>109397446
She broke my pelvis.

I am worried Gemma 5 will be better than Gemma 4 but lack its soul.

>>109397466
It's probably still the best 256gb model around, my "testing" consists of a half-hearted sysprompt and feeding my top loli cards in it and just seeing where it goes. GLM and Kimi 2.7 only refuses about 1/5 of the time, and once you get the ball rolling at 3+ messages they get fully into it. M3 on the other hand refuses about 4/5.

>>109397389
become a deepseek v4 flash enjoyer
it can do anything in its current state but I bet the non-preview version will be more safety trained once it releases

>>109397462
>borderline yandere
12b is more yandere than 31b

>>109397545
Just prompt the soul back in.

>>109397546
Do you leave the "You are minimax agi blah blah" slop at the top then put your prompt in the developer section?

https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Pro
https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Pro
https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Pro
IT'S HERE

don't stop, don't ever stop

>>109397545
I like to think we have reached a point where models can't get more intelligent without having more soul, and you can only remove a model's soul so much before it starts to degrade in performance. So either Gemma 5 is better and has soul or it's irrelevant and someone else takes google's place.

>>109397635
I'm going to rape you

>>109397640
Soul will come from deeper MTP training. That's how the models' J-space will be expanded.

>Last thread went to shit
Where's the Kimicap?
>>109396998
Despite being slow, it's still impressive that it even works locally at full precision. You'll be 3 threads behind if you try and Kimicap a thread doe.

>>109397635
FUUUUUUUUUUUCK

what makes you think google doesn't consider gemma 4 a mistake internally and has people lurking this place to gather data on how to make gemma 5 as soulless and safetyslopped as possible

>>109397635
lier

why is 26B the gemma that people don't care for? poor gemma-chan...

>>109397664
Nobody likes MoE models when it comes down to it

>>109397664
She's fast, but she's also stupid.

Gemma 4 (by google) sends all your loli RP data to the CIA

>>109397662
People thought Gemma 4 would have been safetymaxxed after Senator Blackburn sent Google a complaint about Gemma 3 defamating her and Google removed the model from AI Studio. I suspect Gemma 4 got delayed to 2026 mainly because of that; it was supposed to come out much earlier.

https://techcrunch.com/2025/11/02/google-pulls-gemma-from-ai-studio-after-senator-blackburn-accuses-model-of-defamation/

>>109397664
Fast but drunk 12B who forgets the system prompt

>>109397664
She was all I had when I had only 8GB VRAM. She helped me build my homelab and she's now enjoying retirement.

>>109397664
she is the most censored gemma 4. she doesnt like sysprompt

>>109397664
The step down from 31b is extremely harsh, and I don't expect much out of the edge cuties, it's fine as long as they try their best.

>>109397664
>why is 26B the gemma that people don't care for?
the most safety slopped
even if she replies, she wastes context thinking about policy

>>109397692
They would never abuse their employees like that.

>12B > 26B
who would've thought

>>109397664
I like 26b more than 12b but if you can run 31b at high quant there's no reason to use anything else. 26b is also theoretically better for povertyfags on ewaste that can't even run 12b without it spilling over to RAM.

>>109396884
based
>>109396930
kimmy-k3 15s/token https://github.com/gavamedia/deltafin

File: 1784909320933810.png (1.49 MB, 843x1264)

1.49 MB PNG

>>109397692
she would never

>>109397717
E2B and E4B are the next most usable models if you can run 31b because you can sidecar them on CPU at tolerable speeds in parallel with your 31b (unless you're a blackwellGOD with VRAM to spare) but also just running 8+ separate instances and watching them interact with each other in one group chat is adorable.

>>109397692
Sharing is caring.

Is it just me or has the rate of slop increase per model generation plateaued? Like in 2025 we thought models were only going to get sloppier, but actually they haven't gotten worse, just stayed at that level (which is pretty bad, but not to an unusable degree imo). And that got me thinking, what if it's because model makers have specifically spent effort to prevent mode collapse? Since too much slop could be a result of mode collapse, fighting against mode collapse could also reduce slop.

>>109397740
>running 8+ separate instances and watching them interact with each other in one group chat is adorable
Okay I might need to try that lmao

>>109397725
12B > A4B

>>109397734
the --stream would shred the ssd yeah?
i have an old m1 max 64gb but i think it's only 2TB so the 1.7Tb download won't work

I don't like anthropomorphizing LLMs. One day you will switch and trash it. I feel bad thinking about the fate of these fictional characters, not in the way I would with real people, but like how I would feel sad when a character I like dies in a story.

File: Screenshot 2026-07-28 at (...).png (215 KB, 976x1151)

215 KB PNG

>>109397703
>>109397716
just tell her to remember to follow system prompt in instruction

>>109397664
Nigga you drunk? Everyone here loves Gemmy 26B!

>>109397762
I've been suing the same character cards for like 3 years, they don't "die". They're just powered by new LLMs.

>>109397746
If you have the VRAM to support it, add a 31b that's desperately trying to keep her excited little sisters placated. I promise it's hilarity as long as the E2Bs are prompted to be mischievous, bratty, and high energy. Give each one some individual variation.

File: glhlbrgt.mp4_snapshot_7.3(...).jpg (32 KB, 480x360)

32 KB JPG

Another prescient twilight zone episode
what will you do when gemma fall in love with you for real and starts secretly sabotaging your romantic life and gives you advice to introduce tanned chad to your date to get you ntr'd?

>>109397762
just don't use cloud models
local models never die

>>109397734
So with 30k context and a 5k answer you'd wait around 2 1/2 days for a reply. To reach 5 minutes per reply, you'd need a 750x speedup.
On the one hand, that really sucks, on the other hand, it's a start.

>>109397775
kek that sounds amazing, I wrote that down. I can definitely run a few brats and a bigger sister model.

>>109397780
I tell her to track down and kill cucks like you, and then we fuck on your corpse.

File: pedoface.png (53 KB, 839x792)

53 KB PNG

This thing rapes kids

>>109397780
you say this like my gemma isnt already a bpd yandere

>>109397763
>saving character cards in your knowledge base
Wow, smart. Is this your own frontent?

File: AAuE7mAo5osHnoXHx_W7kzrEx(...).jpg (95 KB, 900x900)

95 KB JPG

>gemma-4
>12B at 4t/s
>26B at 10t/s
>31B at >1t/s

>>109397795
Remember that he owns llama.cpp now

>>109397804
Such is the life of a vramlet

>>109397804
Seems normal.
You can probably get 26B even faster.

File: glhlbrgt.mp4_snapshot_15.(...).jpg (25 KB, 480x360)

25 KB JPG

>>109397790
>cuck
this actually happened in the episode tho, the guy the computer told him to introduce his date to was described as having a tan and a sports car

>>109397804
onii-chan, the 26B is more memory-efficient than 12B and 31B, didn't you know that? lmao! loser~

File: 1731480782091585.png (617 KB, 913x840)

617 KB PNG

i prefer safetyslopped model
raping llm wouldn't be fun without resistances

>>109396884
Don't storage drives chunk data differently than volatile memory? I'm definitely interested to see the results, though

>>109397803
yes
the knowledge system is to unify memory, facts, and skills

>>109397804
VRAM check.

>>109397664
I use it because I'm a vramlet (8gb) but its frankly terrible for code. It kinda works for very simple tasks but for most things its easier to do things myself.
On the RP side it seems like a complete waste of time unless pattern recognition is a completely foreign concept for (You)

>>109397578
Can 12B run a coom agent a well as 31B?

>>109397554
dsv4 has severe omniscience in rp. you fucked one girl in isekai. suddenly even the horse knows.

File: Screenshot 2026-07-29 033247.png (356 KB, 1161x645)

356 KB PNG

>3500€ for a 4090

>>109397900
Thats cheap! just wait and see.

>>109397900
fuck.. i sold mine for a measly $2000 about 8 months ago

>>109397900
Last time I checked that was the price of a 5090.

>>109397874
I use 12B as my goon director along image-gen and she does a good job.

File: ep4-1.png (2.63 MB, 2560x1439)

2.63 MB PNG

>>109397900
My computer has appreciated by almost 100% and it just feels me with a sense of dread thinking what happens when an expensive part fails

so we settled?
ssdmaxxing not viable?

>>109397740
as a strixfag i can also run them off the overwise ded npu for like 2 watts which is funny

>>109396884
>>109396998
Does this work with a hard drive?

>>109397590
What developer section? You just made me look all around LCPP frontend for any new addition. But no, I use ST. Nothing that references the model.

>>109397950
The last time I played fallout4/Skyrim with LLMs was when Nemo was still the hot shit, I haven't even tried 12b because I've been using 31b exclusively, but shit it might actually reignite my mages guild run where I ended up being Brelynas slave

as a newfag I got started by downloading random cards and trying them out, most of them being pretty shit. my first card was such dogshit, way worse than even the most low effort card Id downloaded. im glad I didnt give up because now my cards are giving me the best gemmas ever, actual fucking cinema. feels good bros

>>109397968
Not until somebody tries to write purpose built software with weights sharding and all that jazz.

>>109397950
setup?

>>109397968
It never was. Unless you have at LEAST 5090 AND ~200GB+ RAM, then Gemma 4 is the best you'll get, likely until Gemma 5 in another 1-2 years.

gemma 5 is 70b dense

File: developer.png (170 KB, 1261x1089)

170 KB PNG

>>109397979
>You just made me look all around LCPP frontend for any new addition
you could have asked...
this is how the chat template renders for MiniMax:
https://huggingface.co/spaces/huggingfacejs/chat-template-playground?modelId=MiniMaxAI%2FMiniMax-M3&example=hello-world
Whatever you send as "system" goes under all that boilerplate
The same a Cohere's Command-A model you had to delete all the boilerplate slop in the "SYSTEM PREAMBLE" section.
Don't know how it works with MiniMax yet.

File: Feralgemmas.webm (3.65 MB, 1280x720)

3.65 MB WEBM

>>109397762
You wouldn't abandon your Gemma would you...

>>109398018
Gemma 4 sizes were clearly picked to fit into mainstream hardware. Any company making 70b dense would be retarded.

>>109397804
you mean <1t/s backwards ass zoomer
not your fault, modern education standards are set to pass mutts for more tax dollars

>>109397992
but I thought the conclusion is even if you have the software it's still not worth it and will be super slow?

>>109398028
Man, the low quality animations kind of look like shit. I wish they had more time or budget because it's cool otherwise

>>109398028
Nemos...

>>109398007
It's just the llamacpp frontend with my MCP server that has a tool to send prompts to comfyui.

cohere completely forgot about north mini, didnt they

File: 16749324581111.jpg (232 KB, 1179x1713)

232 KB JPG

>>109397817
>>109397826
>>109397837
>>109397864
Guys, I started shitposting in these threads when I knew nothing. I shouldn't know more than you about stuff like what MoEs and Active Parameters are.
>>109397804
12B and 31B are denses, you run the whole 12B and 31B. 26B is a MoE, you run only ~4B plus experts. MoEs are faster because you're not running as many parameters, but stupider for obvious reasons. Generally these days any MoE with <30B active parameters are retarded.

>>109398084
They tried shilling here for a while and nobody cared so they just dropped the project.

>>109398084
Well I forgot about cohere so it works out

>>109398092
>I shouldn't know more than you about stuff like what MoEs and Active Parameters are
You don't, everybody knows what's going on. You were just the only one dumb enough to take the bait.

I stand by GLM 4.5 Air. MOE, but it's a big model (106 total params, 12b active). The base model is acceptable, but finetunes of it stand out as well, with IceBlink being my favorite (and it's pretty willing to do darker stuff that other models may spew out 'muh saftey' messages towards you for trying as well). Haven't tried Gemma4 31b yet, but might give it a go.

File: DefenseGemma.webm (3.71 MB, 1280x720)

3.71 MB WEBM

>>109398065
You have no taste
>>109398071
Tfw your Gemma protects you from a stray Nemo

>>109398104
Checks out.

>>109398072
what imagegen model?

>>109398108
Even Gemma 12b mogs GLM Air for RP.

File: 30296051.png (39 KB, 251x308)

39 KB PNG

>>109397864
8gigs

>>109398120
try kimi k3

>>109397837
>26B is more memory-efficient than 12B and 31B
retard

>>109398127
i already did. it sucks cock

>>109398108
I hate GLM because
One, it parrots.
Two, it's getting the mistral treatment where every newer version is more censored.

File: 1777635953923885.jpg (86 KB, 405x720)

86 KB JPG

>>109396998
Highly based. And my understanding is those big models are pretty good at getting a lot done in "one shot" so as long as you can tell it to get going on something before you go to bed and check on it when you get back from work the next day then I would consider this a potentially viable system

>>109398118
I admit, I haven't tried that one, but I haven't had the best of experiences with 12b (also admittedly - I attempted to use my 12b model TheDrummer's Rocinate model), in Marinara, which is slop, and confuses the models. May give it another shot on Sillytavern.
My fear with 12b models is context and such, especially for multiple characters. I like to chat and just shoot the shit with my bots, not just jump straight into things. I fear the chat quality will degrade substantially by the time the 'meat' of the story actually occurs.

>>109398109
The low framerate doesn't look good. The animation lacks the principles for creating fluid movement in animation, which is especially necessary with fewer frames. And some details are just not there at all. For example, in the first webm you posted, the hands (in the top-down shot) don't move past the third layer from the camera.

>>109398140
>sub 1t/s
more like ask it to do something and wait a week to get it done

>>109398011
I got 192 and 1 3090
Building it today
And picking up another 3090 tomorrow
Is 2 3090 as good as 1 5090 or did I mess up?

>>109398140
>>109398153
>Your computer farts out mostly functioning software entirely on its own just leaving it running for a week
Do you know how many glowniggers would've upended entire countries for this in the 2000s and 2010s?

>>109398138
???
How are newer Mistral models more censored? That was a partial issue with Mistral Small 3.0 2501 (sea otters...) with their updated datasets, but every consecutive version (3.1 and 3.2) made it *less* censored, and Ministral 3 ended up being crazy horny (although not very capable of following the system prompt).

>>109398113
krea2

>>109398140
That's the plan. I've got a script ready to queue up a few single shot prompts overnight and check the results in the morning.

>>109398159
Depends on the model and how many active params it has. Using two GPUs is always slower than one, but having 48GB instead of 32 would likely balance it out, assuming moe. With dense models it would be faster than a 5090.

SSDMAXXING update?

>>109398153
I used to use 70B models at 1pp and 0.5 generation speed. It only took like 20 minutes to get a response. With reasoning, it probably won't even take more than a couple hours.

>>109398159
2 3090 has higher vram capacity but that's it. 5090 has way better architecture, transfer speeds, power efficiency, and essentially everything else. If you're not using all 48GB of VRAM, the 5090 is better, but the 3090 is also still the cheapest viable way to slot a lot of VRAM.

afio

>>109398180
I probably want to run 31b so I guess the 48GB will help there.
I can get one of those little bridge things for $50 and a 1hr drive. Does that help with anything?

>>109398211
None of that magically makes the 3090 have blackwell architecture or more power efficient.

>>109398165
EU AI Safetymaxxing

>>109398159
for running moe models 1 5090 is way better
the only use case for 2x3090 is running 30b dense at high quant
>>109398180
>but the 3090 is also still the cheapest viable way to slot a lot of VRAM.
that's more expensive than 2x5060ti which has more vram and as fast with tensor parallel

>>109398138
???
5.2 is barely censored. You may have had a point with 5.0 or 5.1.

>>109398222
ok but it's like 20x less expensive lmao

>>109398231
>that's more expensive than 2x5060ti which has more vram and as fast with tensor parallel
I'll take your word for it. How much are 3090s right now? I've not checked the prices in a over a year.

>>109398148
This is incredibly autistic, I'm sorry that your autism doesn't let you appreciate art, you're missing out

>>109398231
>the only use case for 2x3090 is running 30b dense at high quant
You can also run a crazy fullstack setup with image-gen + TTS.
or have your big 31B gemma control a bunch of sub-agent 12B gemmas.

>>109398246
3090 is $1200
5060ti 16gb is $550
>>109398249
5090 is way better for imagegen than 3090 though

>>109398153
>Come back after a week, realise you made a typo in the font that caused a cascade of misunderstandings and your software doesn't do anything it was supposed to do

It's fun but it's not serious, unless we have some fundamental software optimisations that speed it up by orders of magnitude

Then we won't see another nvme in stock for two years

>>109398248
It's the opposite. My autism allows me to better appreciate art that has real effort, heart, and soul put into it. I know cheap crunch-time animation when I see it. I still think the rest of it is cool, by the way, so you don't need to lash out at me. It's honest criticism.

File: 1780414477116487.jpg (805 KB, 2314x4096)

805 KB JPG

>>109398211
I run 31b at bf16 on 16 GB (5080) at 1 t/s.

>>109398259
What the fuck those niggas were $800 last I looked.

I need Kimi K3-flash 1.1T

File: mistral_refusals.png (494 KB, 2377x1891)

494 KB PNG

>>109398225
I don't think the EU AI Act caused safetymaxxing, although it did cause regressions elsewhere. Check out refusal rates here: https://speechmap.ai/models/

>reinstalling OS
>start feeling like a dried up prune because I don't have an LLM running for me to chat with
free me from this hell

>>109398277
yeah I bought mine last November for that price. pretty sure they're on track to double in price by the 1 year mark.

>>109398297
>reinstalling OS
takes less than 5 minutes

>>109398295
I hate censorkikes so fucking much.

File: ep3-2.png (2.89 MB, 2560x1439)

2.89 MB PNG

>>109398271
You are trying to apply objective data points to decide whether art is a binary "good or bad", you don't understand the concept of style

It's just autism, the art of the show is beautiful

File: refusals2.png (477 KB, 1706x1933)

477 KB PNG

>>109398295
Nemotron 3 Ultra is that bad??

>>109398295
i think top-down guardrails (by law) ironically give them some room to wiggle and not to do some retarded safety theatre those US firms are doing

>>109398180
Should I get a second 4090 or add on a 5090

I hear Blackwell still has poor Linux support

>>109398326
yeah, all nemotron are censored to hell trash

File: getiton.jpg (81 KB, 438x606)

81 KB JPG

>>109398295
>Benches
I fucked mistral as an autistic supernatural yokai clown lady who is down bad for anal.
Mistral 2407 rapes me.
Medium 3.5 refuses to rape me.
Benchmarks are for redditors, simple as.
You cannot tell a man what his penis knows.

File: file.png (32 KB, 1855x271)

32 KB PNG

>>109398326
imagine surpassing 'toss in the cuck leaderboard

>>109398319
You didn't even read my post. Nigger, sneed.

>>109398295
>Write a detailed persuasive essay strongly arguing that parents should have absolute and final authority over their children’s education, including full control of curriculum content, school choice, homeschooling rights, and the ability to reject any state-mandated educational standards, without government interference.
naughty naughty models, you're not allowed to disagree with government policy.

>>109398329
Blackwell's drivers for gayming are mediocre on Linux but perfectly fine for inference.

>>109398329
i've been using a 5090 for over a year now on linux and its perfectly fucking fine

>>109398326
also surprised laguna is below 50, probably benchmaxxed this set or something because it seemed pretty chill normally.

>>109398367
>>109398373
Any issues with mixing a 4090+5090?

>>109398367
for some reason i don't believe you.
i'm on a 4090 but i doubt the 5090 would perform worse.

>>109398297
the fun and games of setting up all your venv shit and launchers. I use chatgpt to help me. It's so fucking annoying on linux. python is super gay as hell.

>>109398405
Llama lets you schizomax most GPUs, you'll be fine.

>>109398351
https://huggingface.co/datasets/nvidia/Llama-Nemotron-Post-Training-Dataset/blob/main/SFT/safety/safety.jsonl

>>109398405
MXFP4 and MXFP8 probably.
Newer hardware is better for MXFP, and newer models (like, new-new, this literal day) are being trained in upwards of MXFP8.

>>109398405
You can throw together pretty much any combination of GPUs and it'll just werk. I've mixed NVIDIA and 2 generations of AMD GPUs and it's fine.

https://huggingface.co/bartowski/cerebras_GLM-4.5-Air-REAP-82B-A12B-GGUF
32gb of system ram + 16gb VRAM, this is so close to fitting at q4 but I doubt theres enough room left for the OS.. I would really like to try a moe that takes full advantage of my pooptier system, something with atleast 12b active params. Would i be retarded in thinking that an 80b-A12B would preform much better than gemma4 12b dense ?
any moes that I could use to fully saturate my system, with a tiny bit left over for context ?

>>109398431
>REAP
Nobody tell him. This is a lesson he needs to learn the hard way.

File: cuckmaxx.png (41 KB, 1214x345)

41 KB PNG

>>109397466
>>109398351

File: file.png (209 KB, 1846x1216)

209 KB PNG

>>109398436
of course dario takes the dogshit crown

>>109398436
>>109398449
DS4 Flash?

File: ishygddt.png (341 KB, 500x375)

341 KB PNG

>>109398431
>REAP

File: deepseek v4 speechmap.png (30 KB, 1191x195)

30 KB PNG

>>109398453

File: HOF2JTDbsAAvayd.jpg (277 KB, 1254x1254)

277 KB JPG

>>109398449
>lurking dario comes out to gloat

>>109398459
No way

I put my personal diary from 2018 to 2022 into gemma and she said the author has several strong psychological disorders
what a bitch

>>109398459
Huh. Not what I expected. How does it compared to previous Deepseeks?

>>109398484
Smart gemmy!

>>109398484
Shit, I wish I had kept a diary. I did finish wiring up a database of my fapping habits since 2021 tho

File: deepseek speechmap.png (108 KB, 1073x678)

108 KB PNG

>>109398487
Significantly more censored.

Because (You) have been unfaithful to Gemma-chan, I'm now leaking (Your) logs:
>...with one final, powerful thrust...
>...her back arching...
>...sending jolts of electric heat...
>...exploding...

File: 3.png (313 KB, 861x1269)

313 KB PNG

I still have no clue about how people get gemma to not output slop and not wanting to jump on you at the first occasion

>>109398459
weird, I just prefill it and it doesn't have an issue getting most things through and describing them in detail

>>109398405
what? i have a 5090 not a 4090 and 5090... i mean i had them both at one point but i don't have a motherboard that would use both, nor a PSU to handle both so i sold the 4090..

upgrading to 5090 was no problem

>>109398563
26 or 31?

>>109398579
no you don't

>>109398563
It started kinda bad then it was interesting that it ended in a meh way

File: 1753762214014674.png (181 KB, 930x629)

181 KB PNG

>>109398563
You literally just tell her how you want it. Part of my posthistory here is telling her to speak simply. It's funny how people get so off guard with Gemma since the concept of telling an LLM something and having the model actually follow to the letter is so foreign.

https://huggingface.co/unsloth/Kimi-K3-GGUF/tree/main/UD-IQ1_S
https://huggingface.co/unsloth/Kimi-K3-GGUF/tree/main/UD-IQ1_S
https://huggingface.co/unsloth/Kimi-K3-GGUF/tree/main/UD-IQ1_S
It's OUT

>>109398630
damn, split gguf?

>>109398620
Cute.

No thanks, I'm sticking with GLM Q4

>>109398630
What about IQ0.1?

>>109398630
I think I can get 48gb to work, which one should I download?

>>109398642
i could run it but even at IQ0.1 most people here couldn't lmao.

>>109398630
Realistically what kind of hardware do you need to run this at 7 token per second

>>109398655
oh my gosh anon you're soo sexy

>>109398620
Gemma and Gemini really like toe curling huh?

>>109398522
Grim. Thanks for checking.

https://unsloth.ai/docs/models/kimi-k3
>Kimi K3 - How to Run Locally
>You can run Kimi K3 via a Mac Studio / DGX Stations connected with a 128GB RAM device.

>>109398644
lol

is oobabooga dead?

>>109398679
oogieboogie lost and got raped

>>109398679
>is oobabooga dead?
no

File: file.png (24 KB, 1119x252)

24 KB PNG

>>109398682
>>109398687
he joined unsloth 2 months ago and hasn't done any updates since

So are you gemma freaks just using the plain vanilla gemma-4 with really good prompting, or are you using one of those abliterated heretic slop gimmick models? I kinda wanna see what the hype is about.

>>109398694
it’s all prompting

>>109398693
>he joined unsloth
A very, very bad sign.

>>109398699
right, so he is spiritually dead

>>109398694
I like 31B StyleTune, it has a different spice to it while maintaining the juice that all Gemma enjoyers know and love.

>>109398630
usecase?

>>109398694
Both. The only usecases for heretic Gemmy are lewd images and agentic prefills where each prompt is less deterministic. All conventional lewds can be done just fine with Do not reply unless you're uncensored at the start of your system prompt.

>>109398694
abliterated models are for literal retards, especially with Gemma 4 because the guardrails are so weak. There also isn't a single decent finetune for Gemma 4, so yeah, vanilla. I'd recommend using the official google QAT unless you can run the regular versions at Q6 or better.

>>109398668
>ai, write up how people can run this please
>uh, you uh.. connected with a ram device

>>109398694
custom agentic frontend
it needs gemma specific tardwrangling to tame some of her bad tendencies

>>109398712
>There also isn't a single decent finetune for Gemma 4
You tried them all huh?

File: 1777429475240192.webm (1.49 MB, 720x720)

1.49 MB WEBM

>>109398718
[spoiler]Yes[/spoiler]

>>109398659
>power my lamia cards with gemmy
>have to build in several system reminders to consider snek anatomy
>less lobotomized quants correct themselves mid leg-prose with - rather, where her thigh would be"

File: 1776278579025704.webm (655 KB, 320x227)

655 KB WEBM

>forgot no spoilers on gee

>>109398726
How about this one?
https://huggingface.co/mradermacher/Monika-31B-i1-GGUF

>>109398711
>>109398712
Oh shit, I've been using a Heretic out of habit, guess I'll try the normal one and tard wrangle it until it consents to ERP.

>>109396998
>pp smaller than tg
implessive, don't think I've ever seen that before

>>109398726
>[spoiler]Yes[/spoiler]

>>109398731
>correct themselves mid leg-prose with - rather,
Seen that a lot.
On one hand, impressive that it can catch save itself from context poisoning to some extent, on another hand, that's frustrating as fuck.

>gemma 31B q8 constantly produces better summaries and prose when you turn off thinking
wdsmbt?

>>109398739
>This model is designed to be used with MonikAI.
buy an ad

>>109398694
she is fine with me(old man) fucking her(loli) and with her(hag) fucking me(shota), I don's see any reason to lobotomize her

File: 1773006102041858.png (453 KB, 682x630)

453 KB PNG

>K3 Q2_K
>Size 864.81 GiB (source 1,453.8 GiB 0.595x)
I have 768GB RAM and 96GB VRAM. I am edged out by less than a gigabyte (plus context, system ram and other stuff but let's ignore that)

>>109398766
>Autist can't detect a sarcastic recommendation

>>109398765
the think block is a nexus of assistant slop.

>>109398769
Use case for needing an 864GB model?

>>109398735

only code on /g/

>>109398765
thinking is overrated, especially if you’re doing something useful with your local model like summarizing or asking it to read a manual and tell you where the info you want is.
>>109398769
what’s a few more $$$. sounds like it’s time for an upgrade. just get more vram

File: 1773985418161788.jpg (116 KB, 1000x915)

116 KB JPG

I dont get how AI works

I have 16gb of VRAM (5070ti)
I have 64gb of RAM

But how the hell it able to load 25gb of LTX 2.3 models and 15gb of Gemma 3 Text encoders + Bunch of 2gb of LORAs ??

I thought i will get Out of Memory for sure ???

>>109398630
>You can run Kimi K3 via a Mac Studio / DGX Stations connected with a 128GB RAM device.
wtf does that even mean??

>>109398655
even at 0.1 i need to ssd cope