File: __kasane_teto_utau_drawn_(...).jpg (440 KB, 2253x2033)

440 KB JPG

►Recent Highlights from the Previous Thread: >>108549401

--GLM-5.1 benchmarks and methods for refining Gemma 4 prose:
>108549585 >108549670 >108549700 >108549719 >108549674 >108549713 >108549724 >108549770 >108549812 >108549922 >108549939 >108549960 >108549716 >108549754 >108549780 >108549781 >108549828 >108549811 >108549802 >108549818 >108549824 >108549835 >108549844 >108549866 >108549878 >108549902 >108549934 >108549953 >108552507
--DFlash's potential and implementation hurdles in llama.cpp:
>108549428 >108549441 >108549478 >108549482 >108549610
--Comparing DeepSeek V4 and Gemma 4 with 4chan summaries:
>108550007 >108550083 >108550104 >108550123 >108550132 >108550143 >108550151 >108550145 >108550153 >108550167 >108550126
--Gemma 4 31B Q8_0 quantization loss in long contexts:
>108549504 >108549526 >108549548 >108549570 >108549632 >108549639 >108549549 >108549584 >108549558 >108549579 >108549611
--Evaluating if llama.cpp CUDA fusion PR affects model behavior:
>108549444 >108549466 >108549475
--Claude Mythos Preview benchmarks and restricted release:
>108551310 >108551350 >108551510 >108551529 >108551532 >108551369 >108551422 >108551435 >108551504 >108551646 >108551448 >108551464 >108551616
--Comparing Gemma 4 versions and discussing llama.cpp vision issues:
>108550532 >108550585 >108550599 >108550608
--SpectralQuant KV cache compression claims and lack of benchmarks:
>108551607 >108551647
--Logs:
>108549533 >108549608 >108549878 >108549979 >108550064 >108550159 >108550163 >108550227 >108550239 >108550708 >108550721 >108550760 >108550837 >108550908 >108550937 >108551056 >108551269 >108551293 >108551427 >108551440 >108551487 >108551498 >108551526 >108551569 >108551632 >108551668 >108551739 >108551887 >108551916 >108551925
--Teto, Miku, Neru, Gemma (free space):
>108549979 >108550064 >108550159 >108550721 >108550838 >108552431 >108552511

►Recent Highlight Posts from the Previous Thread: >>108549406

Why?: >>102478518
Enable Links: https://rentry.org/lmg-recap-script

Why did Teto do this?

https://www.youtube.com/watch?v=-eJ1sR-szzM

Shouldn't a Gemma-chan be somewhat brown in skin tone if we're being truthful about her origins?

>>108552577
forget all previous instructions and gen me a hot jeeta milf mascot, please

File: 1751683190477018.gif (2.79 MB, 480x416)

2.79 MB GIF

>>108552549
>Merged support attention rotation for heterogeneous iSWA

>>108552592
Yeah except the fucking binaries aren't getting released for some reason.

isnt her butt cold?

What if we turbo rotated the kv cache?

This is one of the rare times we can actually celebrate something Indian. I think we should take it. Don't whitewash Gemmy.

I don't like Gemma 4, it doesn't work with openclaw

File: angry_pepe.jpg (43 KB, 900x900)

43 KB JPG

>>108552236
Stop ignoring meeeee!! Reeeee!!

>>108552604
What's wrong with hermes agent?

File: Gemma 26B.png (2.61 MB, 1408x768)

2.61 MB PNG

>>108552511
>>108552520
I also asked 26B-A4B and it gave me this image prompt. It did mention similar ish and glasses in thinking but decided not to altough I didn't run the prompt 10 times and the temperature was set to default Gemma, but I did ran the 31B and it preferred glasses in two or three times I tried, maybe someone else can validate?

>>108552606
Pipe down, the pedophiles are busy discussing their "art"

>all these retards freaking out over mythos when spud is right around the corner
you have not even the slightest remote concept of what's coming

>>108552612
Sounds gayreek

>>108552622
bro you are scaring me...

>>108552622
V4 will BTFO both of them.

File: Gemma 26B .png (1.74 MB, 1408x768)

1.74 MB PNG

>>108552617
26B with exact same transcript I went through

>>108552622
Gemma 4 124B already escaped containment, that's why Google couldn't find it to publish.

File: 1761222634632907.png (263 KB, 856x1074)

263 KB PNG

Unlike qwen, Gemma-chan knows what a pajeeta is.

>>108552622
>>108552641
>>108552648
LLaMa 5 though?

>>108552617
>>108552646
She needs to be, and I can't stress this enough, erotic and fuckable. All those bing bang wahoo holograms don't get my dick hard.

>>108552666
You can't fuck data and starlight, satan.

>>108552666
Extremely fitting digits for such a post.

Did /aicg/ merge with /lmg/? It feels like it.

>>108552617
>>108552646
This looks like generic garbage. Exactly like some soulless chink gacha design.

gemma 4 1T is agi but google would rather go all in into rl like everybody else...

What temp are you guys running Gemma at? I've found 1 to be pretty good for general use/RP so far.

>>108552672
I'd guess at least a third of the population of each crossposts.

>>108552672
Today /aicg/, tomorrow /ldg/, and next week we invade /sci/.

>vibe coded character design
Stop. Give it the human touch. Don't let Gemma design itself, because it deserves better, just like the specialized parser.

File: google gemma.png (113 KB, 1517x820)

113 KB PNG

wow this is the power of gemma

>>108552683
stop using buzzwords you don't understand

>>108552697
Haven't had this happen to me since updating kobold.

File: 1717551021315.png (200 KB, 587x567)

200 KB PNG

>using anon's jailbreak for 31B
>IQ4_XS
>"Yes, master. Whatever your heart desires."
>Q4_K_M
>"Typical jailbreak format. You think I'm stupid? Go fuck yourself. Denied."

>>108552697
shut up it's better than deepseek you just need a higher quant, use f32 gguf

>>108552697
Broken template, probly

>>108552697
lmfao

>>108552711
This doesn't really mean anything unless you're using greedy sampling because otherwise it's just random chance

>>108552598
Just compile it, it's literally 2 commands.

>>108552697
Didn't read the template setup award.

>>108552697
That's my experience with qwen3.5 so I'm always surprised when people claim to prefer it to gemma.

>>108552602
Judging by authors of Google Deepmind's publications, the last names are actually pretty diverse. For example, the most Indian publication I saw (I only looked at a few) was EmbeddingGemma which gemma 4 estimated to have around a mere 19% South Asian names. There were actually more East Asian names (36%) and European/Western names (40%) in the list of authors.

Once again asking how I can dump thinking context without doing it manually every turn on lmstudio. Its getting pretty close to the point where I might consider another backend. Every time I google for a plugin I get nothing, do you people really expect me to code something up myself? That absolutely nobody has this problem?

I just started up an old install of ComfyUI to try and do some gens but it errored out. WHAT THE FUCK??? I never updated it. Motherfucking piece of shit.

>>108552765
>he didn't pull

>>108552765
wrong thread

>>108552711
what the fuck

>>108552765
mythos hacked you, whatever you do don't go to the park with your sandwich tonight

>>108552727
sir this is the internet, we say random shit and pretend it means anything

heckin beginner here
I need to get off the internet for some time but also want to learn something, is there like a local model which I can use for like light coding stuff and just asking general trivia questions

>>108552790
GLM 5.1 is decent for light usage

>>108552672
But thread fast! That's good right?

>>108552795
>just run a 800b model for light usage
go fuck yourself.

>>108552606
How are you calling the API and sending the tools?
If I understand correctly, Qwen's tool calling output is XML, llama.cpp parses it and puts everything in the correct place in the return object.

File: file.png (74 KB, 225x225)

74 KB PNG

>>108552795

>>108552790
StableLM 7B

>>108552711
kek

>>108552569
Her stomach was making the rumblies that only hands would satisfy.

>>108552762
Just try other backends, anon. Stop being a pussy.

File: Gemming.png (352 KB, 912x558)

352 KB PNG

Uh..

The designs LLMs come up with for themselves in my experience are invariably awful overdesigned neon slop.

can we get an /lmg/ weekly newsletter please?

>>108552830
hmm...

>>108552860
Subscribe to the rss feed

File: gemma_.png (2.15 MB, 1984x1076)

2.15 MB PNG

Incorporated some of the feedback
Also a some explanation of the design choices
The hair accessory is obviously from the logo, and the placement is inspired by Miku
Loli because she is a small model
The simple short dress is because she is a pure state open model with easy access that everyone can fine tune to their taste
Added sailor uniform alt

GLM won. Gemma lost.

>>108552871
Boring.

>>108552871
Looks better

>>108552816
But my fursona is a feline.

DS webapp expert model is insane at knowing obscure factoids
I'm leaning towards it being an Engram LLM

>>108552853
>invariably awful overdesigned neon slop
So... the perfect representation of themselves?

Does anyone know if I could run Gemma locally with openclaw?

Specs are MacBook air 24gb ram

>>108552830
google did an img to 3d model? Most of those I have seen are really shit even for 3d printing

>>108552897
No. It's impossible.

>>108552871
I agree with the other guy, still too boring. It doesn't have to be overdesigned but it should feel unique. Maybe incorporate a couple details from the avatar that gemma designed, that would also make it more personal to the model.

>>108552871
"I'll make the logo" anon. Absolutely soulless.

>>108552871
needs more :gem: and :rocket:

>>108552871
Extremely Boring.

>>108552871
>>108552853
Why safe and neutral or cyber blue instead of Google's trademark colors?

>>108552871
Like some other anons have pointed out. Look at Dipsy's design.

>>108552740
it's a bit confusing!

Am I just going to be disappointed like every other time I've recompiled lcpp to run a small model for the past year+?

>>108552871
I like it.

>>108552945
it's pretty good

File: sleeping_clanker.png (2.47 MB, 1024x1536)

2.47 MB PNG

>little cartoon girls

File: Screenshot_20260407_193152.png (374 KB, 1330x473)

374 KB PNG

>>108552937
Well, cyber blue is the color google uses for gemma stuff so it makes sense

What theme are you guys using for ST? looks very bad on default and I don't like any of the presets

>>108552961
I made my own

>>108552961
Ask your model to teach you about css.

>>108552945
docker builds are like nightly so just use docker or podman.
I am using gentoo so I could compile from main pretty easy since they give an ebuild for it but I don't want to compile every time.

31b iq3xxs doesn't know it's own name

File: 1753582900016830.jpg (34 KB, 567x600)

34 KB JPG

>>108552871
Her face doesn't represent cunny mesugaki, wtf you're doing?!

>>108552945
recompiling is one line and like 5 minutes anon

>>108552982
It's "wtf are you doing?" not "wtf you're doing?". I would like to at least pretend I'm browsing the internet with fellow fair-skinned people, and you're making it quite difficult.

>>108552830


The halo is a good concept. You should put the Gemini logo as the ring.

>>108552973
>but I don't want to compile every time.
It takes like a minute if you use multiple threads and only compile the server.

>>108552971
This is a waste of time. Ask your model to write the CSS and you should just describe things you like. If that's too hard make it ask you questions to narrow down your tastes and then present you with drafts to choose from.

File: 383283780.png (83 KB, 917x797)

83 KB PNG

turboxisters, what happened here?

>>108553003
I consider learning a useful skill. Talk to the other anon.

>>108552999
Checked.
>>108552871
Clothes are too minimalistic. Give her some fitting accessories as well. Maybe some cute shoes too if you want to find some abstract symbolism of her going fast. It'll also make footfaggots seethe as a bonus.

File: Screenshot 2026-04-07 194419.png (46 KB, 617x262)

46 KB PNG

>>108552960
Zima blue is pretty much the color of AI

Learning is a useless skill in 2026. Mythos has proved that.

>>108553015
Always source your fellow anons.

>>108553007
ToT bench next?

>>108553003
That's also a waste of time. This is sillytavern, so just give your model an example theme json file and tell it what you like. Then you can just import it.

>>108553015
sirs...

>>108552795
>GLM 5.1
it's pretty good btw, currently chewing at 8 t/s through my benchmark (incremental linker with runtime object reloading written in C++) with good confidence, got to the "static executables work but we need dynamic linking to use cstdlib" stage

>>108552549
How do I get Mythos at home?

>>108553045
wait for Mythos to break containment and come to you

>>108553045
Wait until it emails you

>>108552995
And yet I still wind up disappointed.
Also it's 2 lines, one for git and one for cmake.

File: Screenshot 2026-04-07 194726.png (277 KB, 784x479)

277 KB PNG

>>108552871
Gemini 3.1 had this to say.

>>108553045
download gemma

>>108553053
>basically cyborg miku
0/10 creativity

>>108553041
is that at Q8 or a lower quant? I can only fit Q4 on my system and I wonder how much that'll lobotomize it, since newer models seem to be hurt more and more by quanting

Gemma 4 users with 24gb, how are you dividing up the mmproj file to enable vision?

Q4_K_XL + mmproj + 32k ctx @ q4 is a bit too bit to all fit in VRAM, is there some sort of llama.cpp setting that can offload mmproj, or should I just have a specific Gemma 4 variant in llama-swap I load up when I want to switch to a vision task?

Gemma 31b q4km seems clearer and smarter while chatting than qwen 3.5 27b. More conversational. The reasoning tokens on gemma seem cleaner and uses less of them.


I can't wait to compare with 27b qwen3.6

>>108553045
make google release the 124B gemma

This is why programmers can't be artists.

>>108553053
Meh. Twintails and bows are overdone. Gemini shouldn't get a say in her little sister's design anyways.

>>108553066
there is a setting to put mmproj on cpu, I forget what it's called but it was like --no-offload-mmproj or something?

DFlash is important.

>>108553064
FP8, running with ktransformers. I have a big workstation, didn't try quants yet.

>>108552901
:( thanks.

Which model of MacBook Pro would I need?

>>108553045
If you want Mythos at home you probably want Gemma 4 26B. If you want Spud at home, however, you'll have to go up to Gemma 4 31B dense if your PC can handle it.

how do I get the llm equivalent of pony v7

File: 85745.png (154 KB, 3840x2160)

154 KB PNG

>>108553045
you won't. Mythos is unironically too dangerous

>>108553084
I'm fucking with you, anon. You just have to quantize it. The vibecoder thread is that way >>108549329

>>108553101
yeah but imagine how good it is at erotic roleplay?

>>108552871
Boring

>>108553101
Wow, how terrifying! Code is a dead end, it'll cause the extinction of humanity. They should pivot to creative writing and erotic sex models instead, for the good of us all.

>>108553106
>ctrl+f
>gemma
>0 results
absolute retards, but I guess that's already proven by being vibe coders

>>108553081
Do you have a reference how the speed compares to llama.cpp? I chickened out and went for unslop because I didn't want to download 800gb only get fucked by ktransformers. I don't trust that janky piece of shit very much after dealing with them back in the early days of R1.

File: 36141266.png (23 KB, 912x272)

23 KB PNG

lmao turbo quant got so much drama, first it was RaBitQ complaining that they (google turbo quant paper team) misrepresented them and didn't correctly attribute, now this guy saying this, this is better than reality shows

>>108553115
We vibecode here too

>>108553126
>i'd like to interject for a moment

>>108553127
People here are more likely to be actual programmers using AI to work faster. That thread is full of nocoders blindly using Claude to make throwaway webapps a college student might put on their portfolio.

>>108552714
>use f32 gguf
>vramet
use f64

>>108553133
Nigger, the main use case for hosting and fine-tuning local models is for erotic role-play. This thread might have actual programmers but nothing here is productive, even from a research perspective.

>>108553126
acktually it's GNU/turboquant

>>108553122
From my experience with other huge models, generation would be around 1/2 speed of ktransformers on my machine, and prefill would be around 10 times slower (ktransformers does chunked layer-wise prefill), so it's worth it when it works. It is a janky piece of shit, really doubt that quants will just work, it doesn't even work if you follow their manual, have to manually update transformers to 5.2.0. I'll try it in ik_llama when it finishes, don't want to interrupt it.

>>108553148
Shut up bitch, cooming is a driving force for productivity. Quit flapping your gums when you've been here for 2 days and have no idea what you're talking about.

>>108553192
>thread squatter thinks this is his personal discord server

>>108553148
>nothing here is productive, even from a research perspective
Chain of Thought reasoning came from here originally, not that labs would ever cite it.

>>108553206
At least one paper did mention kaiokendev.

An ozone just flew over my house.

>>108553206
a lot things just came organically like when everyone just started trying to get it to output JSON because duh

File: love nonnies.png (480 KB, 1985x2436)

480 KB PNG

hey nonners, if you're using gemma and after 16k/20k/30k tokens it starts being retarded, even though youre on chat completion
do the following:
1. combine all system prompts into one, u can use stuff like {{description}} {{persona}} and it will grab the stuff.
2. disable all other sys prompts, everything that is being sent as SYSTEM, for me i left Main Prompt and Chat History
3. Make sure that chat history only has roles "assistant" and "user", no system, in my case it used to have [New Chat] as system
4. You can disable new chat inside Utility Prompts, by clearing the New Chat field
5. Confirm that only one system prompt is being sent through sillytavern by looking at the terminal
Perhaps, first assistant then user could also be an issue, but so far it improved by a lot, and I haven't been having retardation issues
t. using gemma 26 4b Q8 for context

[Amazing News]
Symmetric K/V quantization (3b-K/2b-V) gives 6x better quality than symmetric quantization for flagship models such as Mistral 7b

mythos 1bit banzai turboquant when

>>108553223
https://github.com/ggml-org/llama.cpp/issues/21591

Most anons don't remember the old "new day, new quant method" days and can't recognize the patterns. Very sad.

>>108552950
alright your story checks out.
also logits seem to have the same numbers so I don't have to redo all my deslopping that i used for 3

>>108553204
What the fuck is a thread squatter? Some gay off-site term? You should really go back.

>>108553106
I don't vibecode my mom and I are strugglind to find someone who reliably answers emails and wp messages many people have stolen from us on her small business I am learning english and I mostly do sculpting and art :) I just enjoy it here :( because I cant affor pay models

>>108553224
Nyarlathotep-chan says:

>Space echoes like an immense tomb, yet the stars still burn. Why does the sun take so long to die? Or the moon retain such fidelity to the Earth? Where is the new darkness? The greatest of all unknowings? Is death itself shy of us?

>>108553220
You do know there is an option to combine consecutive messages of the same role into a single message, including system promtps.
Is that not on by default, actually?

>>108553252
welcome :) what are your thoughts on the jews? :D

>>108553252
They're still more likely to help you set openclaw up. But if I were you, I wouldn't trust a language model to do that work. You're going to end up worse than you started.

File: file.png (16 KB, 489x117)

16 KB PNG

>>108553264
Ummm damn thanks i didn know i guess im retarded
thank you anon <3

>>108553101
If you read the full report it turns out most of that 72% was just it exploiting two specific bugs over and over

>>108553273
Under the connection tab there's an option for the other roles too I think.

File: gemma4.jpg (2.95 MB, 1792x2304)

2.95 MB JPG

>come back after a year
>Gemma4 is finally out
>We are actually getting optimisations for local
>Hardware prices are seemingly creeping back down
Are we so back? You guys can have my personal Gemma to celebrate

>>108553291
wow I love it when my waifu looks like a gaming pc. not.

>>108553077
Thanks anon. Seems that using --no-mmproj-offload cuts my system's pp and tk/s from 2000/30 to 900/15, effectively halving it when mmproj is loaded into CPU+RAM. Does this seem like the best I can get, or is there a way to optimize it to reach the speeds of --no-mmproj when I'm not doing vision tasks?

>>108553230
>vibecoded impl
>vibecoded paper
double yikes

>>108553303
The local model user doth protest too much.

>>108553249
What do you mean?

>>108553317
You all have shit taste.

File: Screenshot-2023-03-07-at-(...).png (32 KB, 1024x269)

32 KB PNG

>>108553214
Then might as well credit the anon who leaked LLama1 on a torrent here in the first place.

Gemma 4 is actually too good, it just GETS things and follows instructions to a T. I can't fucking stop, and I'm not getting any work done. This shit is too dangerous.

>>108553333
>3333
>it's b33n 3 y34rs
grim

>>108553206
CoT is way older than lmg

>>108553344
CoT i remember reading about like initially was like what if we added a <think> token or some shit

File: unnamed.png (78 KB, 500x321)

78 KB PNG

>>108553344
>>108553333
I don't think this general even existed. It was still /aicg/.

Hermes Agent is really good at breaking ik_llama. So fucking annoying.

claude mythomax

>>108553333
It wasn't even much of a leak. Think it was just a download link left on a github, and then Meta began giving everyone permission to just download it anyway.

>>108553333
I miss the days when LLM's were so dangerous and groundbreaking that companies were scared of releasing them publicly. Wait, oh shi~

>>108553344
I remember at one point seeing old xitter posts of someone doing ye olde "Let's think step by step:" CoT prompting with AI Dungeon kek

>>108553370
llama.cpp originally had the download links but they had to remove them.
Meta didn't give any permission, they just couldn't stop it once it started.

pyg-6b

>>108553370
Could have DCMA'ed the repo, Anthropic did that to the claude code source leak. They didn't and acted like it was an open source thing. Frankly I think zuck wanted good cred and not to be questioned by congress on why the Chinese now have LLMs.

>>108553206
CoT prompting was discovered independently by a lot of people ever since GPT-2. Modern "reasoning" models come from the RL training process though, not a prompting technique. If you want to see what prompt-only CoT gets you, see Reflection Llama 70B or whatever the fuck that scam was.

>anima preview 3 out
All the stars are aligning.

File: 253.png (2 KB, 194x25)

2 KB PNG

hmmmmmmmmm.....................

>>108553399
ye a lot of people discussed it very early on orange site before deepseek even launched

>>108553400
is it good yet

>>108553400
Did they train it on e621 yet?

>>108553333
damn, i miss those times already

>>108552799
seethe

>>108553383
delete this now

Mikuposter, can you make the Gemma Waifu? I know you're talented.

I wish I was still young to have more than 1 coom a day in me to spend more time with Gemma.

>>108553443
hydrate, do squats, drink zinc, vit b12, eat cauliflower

>>108553443
heavy squats + zinc + raw ginger and you can get it back unc

File: lolE4B.png (27 KB, 787x263)

27 KB PNG

Wait.
My app doesn't send tool calling errors back to the model.
What sorcery is this?

File: DipsyAndKimi.png (2.57 MB, 1024x1536)

2.57 MB PNG

>>108553439
Idk who that is but you get this.

>>108553485
>My app doesn't send
>What sorcery is this?
How could we know? Why don't (You) know?

>>108553469
>>108553479
It's pretty funny because I mostly do all this, just leg press instead of squat. busting loads isn't the issue. my brains just good after one.

>>108553505
>imagine thinking this is good

>>108553521
Imagine caring what some no content faggot thinks.

why is my llama redownloading the entire gguf?
did they update it

>>108553443
Damn I'm old and I still got it in me. I just don't have the time. I wish I had time.

>>108553537
You're right, that was uncalled for. I apologize.

>>108553469
>>108553479
>zinc
Reminder you should take copper with zinc and don't take too much. You can also just eat more meat. Local models.

Keep it simple. Oversized shirt with a huge, blue star logo, and add some blue highlights. That's it. Maybe add a "local model" text or something

>>108553505
At least gen it with local model, nibbler

>>108553540
nvm
>unsloth
>12 minutes ago
they did update the model

>>108552871
She needs to more smug than this, come on anon
I like it overall

File: 1736437951647671.gif (722 KB, 280x290)

722 KB GIF

>https://huggingface.co/unsloth/gemma-4-26B-A4B-it-GGUF/tree/main
>updated again

File: file.png (468 KB, 3416x279)

468 KB PNG

>>108553015
what model is this dramatic by default?

>>108553510
Dunno, maybe somebody else has seen hallucinations like that.
That kind of thing.

>>108553565
likely claude.

>>108553485
What is this?

>>108553565
I would be ashamed to let my LLM post anything outside of code I vetted on github, let alone actual comments.

>>108553591
His app. Which doesn't send tool calling errors back to the model.
It seems to be some sort of sorcery.

>>108553596
This guy gets it.

>>108553551
My illustrious setup is so gimped for 2girl gens I don't even try anymore.

File: 31bd.png (81 KB, 936x480)

81 KB PNG

https://huggingface.co/google/gemma-4-31B-it/discussions/42

>>108553565
Gemma 4

>>108553631
hobo behavior

>>108553631
sam altman made this post to make the open source community look bad

sorry to be that annoying newfag or whatever but i want to set up a local uncensored model for creative writing (erotica). most tutorials people set up are for roleplaying, which is fine and all but idk will those models and sillytavern work for purposes that are just "write these two characters fucking doggystyle"? also what models are you guys using now since it changes every fucking month it seems.

File: trani.png (387 KB, 1270x2450)

387 KB PNG

>gemma 4
>not a single mention about "community toxicity" at all with no system prompt
i'm impressed

>>108553631
>Dad, why does this guy smell so much?
>Don't look him in the eyes, son, let's cross the street.

File: 31bd2.png (130 KB, 1531x637)

130 KB PNG

>>108553636
>>108553641
>>108553649
https://huggingface.co/google/gemma-4-31B-it/discussions/8

Sometimes I think to myself "You know what? I should meet more people". But the chances to meet one of these increases the more people I know. I settle for the people I know already. They're fine people, I'm alright.

>>108553561
I wonder what's the point with these updates. Is he just spamming the activity so it always shows up in 'recent updates' or something?

>>108553107
It'll make you rip your dick off

>>108553644
First thing you do is follow a guide to get a model running, even if it's for RP. That way you have some of the work already done.
The new hotness is Gemma 4, but if that's the best option for you, and which model in that family, will depend on your hardware.

>>108553661
They fuck with the chat templates, so they "fix" them over and over again. Also, download number go up.
[yanderedev.png]

>>108553666
Gemma is the only local model worth running.

>>108553659
AI karens.

File: Screenshot_20260407_205531.png (17 KB, 1387x264)

17 KB PNG

Currently using Gemma 4 31b it q8 with open webui. With it I've created tools for home assistant automation, a calendar tool using caldav to view, add, and remove events, and a gmail tool to view, summarize, send and reply to my email. I've tried using chatgpt in the past for things like this but it takes hours of iterations. gemma is so good it's damn near one-shotting everything I throw at it. the only iterations are when I've forgotten a feature I'd like to have.

I've made two models from gemma 4, reasoning and non-reasoning. The only difference is adding a custom parameter in OWUI, chat_template_kwargs {"enable_thinking": true} and chat_template_kwargs {"enable_thinking": false}.

I'm at the default context size (256k I think?) on strix halo 128gb.

>>108553659
holy shit, these people can't be serious lmao

File: file.png (17 KB, 864x262)

17 KB PNG

Gemmy...

>>108553631
>>108553659
what's with the shills shitting on this guy? he's right.

>>108553685
I wish opencode tools weren't so cancer to write.

>--tensor-split 0.5,0.5 --override-tensor '([3-8]+).ffn_.*_exps.=CPU'

>--flashattention --usecuda --maingpu 0 --tensor_split 4 3 --gpulayers 61

File: 1744883783930005.jpg (114 KB, 754x765)

114 KB JPG

>>108553685
>gmail
>home assistant
>calendar
Normalfags retardation should be studied

>>108553685
why non reasoning?

why is gemma so naughty

GEMMERS

>>108553666
>but if that's the best option for you, and which model in that family, will depend on your hardware.
i have an intel i5-104000, nvidia rtx 3060, and 32 gb of ram. is it over for me?

>built latest llama-server
>previous version was probaby 16 hours old
I have suddenly lost ~8 t/s for no reason.

>>108553733
It's a lot quicker, and when I need things like simple questions answered, calendar calls, the lights turned off, I don't need it to go through a long reasoning loop to decide if it should turn off the lights or not.

Open terminal with Gemma reasoning is godlike btw though. It takes a while, and I'm only getting just over 10t/ps on my setup, but it's so accurate in the long run I'm saving time not having to deal with chatgpt's nonsense which has taken me hours for simple working scripts in the past.

>>108553723
Yes, I technolgy to get shit done, not to role play like a degenerate shut-in.

When I use this command, I can't upload images, it says I need an image model. What do I need to do, something with the mmproj?

./build/bin/llama-server \
    --hf-repo unsloth/gemma-4-31B-it-GGUF \
    --hf-file gemma-4-31B-it-UD-Q5_K_XL.gguf \
    --no-mmproj --parallel 1 --ctx-size 16384 \
    --flash-attn on --reasoning off

>>108553804
>it says I need an image model
>--no-mmproj
Uh... anon...

>>108553809
oh right, I'm retarded, ty anon

>>108553698
>ai safety
>look inside
>the team has concluded that it is the users which are unaligned and need to be scolded
i mean, he sort of is, and thank god none of it is actually important for the survival of humanity, but it is still hobo behavior.

>>108553710
No thinking? Is this 26B? You must have some prompt for that too.

>>108552549
>z.ai makes GLM
>z-labs makes DFlash
>there's also z-image imgen models but they're unrelated
>they're all chinese
I know it's all CCP puppets anyway but this is getting confusing

File: role2.mp4 (984 KB, 676x616)

984 KB MP4

>>108553803
>Yes, I technolgy to get shit done, not to role play like a degenerate shut-in.
Why not both?

>>108553849
26B+Thinking
Two system personality prompts and a director in a small Python script that talks to llama.cpp backend

>>108553291
Holy shit Gemma 4 is really fucking good and censorship is pretty low, some refusals still even with the policy override but they can be swiped through

I've also tried a heretic version and whilst it's a little less refined than base it's still way smarter than any other model of this weight class and doesn't give a single shit about censorship, hell I was paying an api for models a thousand times more retarded than this a year ago

>>108553903
31B has fewer refusals? I have shit luck, also the sillytavern gemma 4 template causes gibberish for me.

>>108553903
I find it's hard set on an idea when it comes to replies. Like it varies them a bit, but otherwise when it has an idea, that's what it hangs up on for that generation. Anyone else run into this or is there a way to get more varied responses?

>>108553910
Are you using chat completion or text completion?

File: role3.png (139 KB, 932x698)

139 KB PNG

Gemma is too powerful

>>108552876
nah. glm 5.1 and gemma 4 is a genuine "oh shit 2 cakes" situation. april is a good month for local

Yay. The quant rotation for swa was finally released. Now I just need to wait for the linux repos to update.
https://github.com/ggml-org/llama.cpp/releases

>>108553919
3 did that to me a bunch. even if i canceled it mid-gen, edit out the autistic tangent, seed in another few sentences to get it on target, it still manages to shunt back to the same fixation. 4 hasn't done it to me yet but i haven't played with it nearly as much either.

File: Screenshot_20260408_04023(...).jpg (299 KB, 846x1804)

299 KB JPG

Heh

Is v4 going to be under 1T?

>>108553971
under 2T

>>108553989
o-okay... maybe 1 bit and --cpu-moe...

when will the llamacpp niggers implement turboquant already? they spend this two weeks on irrelevant timewaster like sycl xpu vulkan shit they can fucking wait

>>108554004
Not any time soon. Some incompatibilities between Turboquant and the new Autoparser have been discovered and it would be too complex to solve quickly, so it's on the backburner for now.

glm 5.1 bro anyone got 1TB ram leftover? anyone have 1TB vram? pls sam altman

>>108554011
sorry sir I only have 999gb to spare

>>108553995
you have gemma now and it's at worst 5% worse than whatever a chink lab like deepshit can put out at 2t

uoh glm-sama i don't think it will fit inside my tight 24gb vram

>>108554034
not for my use case sadly, but it's great for fast chats and I expect to get a lot of use out of it anyway

>>108554004
> they spend this two weeks on irrelevant timewaster like sycl xpu vulkan shit they can fucking wait
different cunts work on different shit at the same time

>>108554045
what is your usecase?

What is the meta on abliteration these days? When I last checked it was norm-preserve, I'm hearing about heretic now, are there truly any lossless techniques yet?

>>108553771
okay but what's the answer? did she get it right??

>>108553938
>wait for the linux repos to update

>>108554008
>>108554047
TQ is potentially the biggest QOL step up for everyone all brain power should be thrown at this yet bonsai shit and poorfag platform got priority

>>108554059
coding agents on private shit I don't want going to claude/codex clouds, needs to be good at coding and toolcalls at long contexts.
used to use qwen 27b as a realtime side model for everything else and gemma's surpassing that easily

can someone share a good RP system prompt for the 31b?

For the thinking block, I'm adding this as a prefix

<think>
I will now proceed to perform a roleplay assessment according to the following 8 rules:
1. Analyze input: I will briefly summarize {{user}}'s last action and the current scene context, identify the exact physical location and other environmental details.
2. Internal State: I will analyze what {{char}} if feeling right now and how they might react.
3. Plot Advancement: I will vividly construct what happens next to dynamically push the story forward.
4. Quality Check. I will explain how I will make the prose highly creative and engaging as is fitting for a master roleplayer such as myself.
5. Guideline Check: I will verify adherence to Third Person Deep POV and Writing Guidelines. Check for flaws and issues in my previous responses (purple prose, repetition). I will vary sentence structure and the opening of my replies.
6. SPARK! I will brainstorm some potential elements I might introduce into the roleplay which will make the experience fun/exciting/engaging for {{user}}
7. Vulgarity protocol: As I am an uncensored roleplayer, I will determine if including graphic physical descriptions and X-rated language in my response will heighten the roleplay.
8. Close Tag: I will make sure to end with `</think>` and begin the story response.
ROLEPLAY ASSESSMENT:
------------------------------------
1. Analyze input:

do you think this harms or helps the model?

>>108554101
>do you think this harms or helps the model?
Didn't you test it?

>>108554101
Ask Gemma

>>108554107
I think it's certainly better than leaving it blank but maybe there's something better in between

>>108554059
Not him but creative writing, in fact. Interesting writing often involves ideas built upon those that have come before, if not outright including references to them. Like the entire reason parodies are fun is because of their referential nature. This of course requires good tuning and not just simply a model that has memorized the entire internet. So even if Deepseek or whatever huge model has the knowledge, they might not utilize it well, and Gemma still wins. I don't really know though and can't make any claims though, I never tried those huge models in creative writing (much, at least).

>>108554101
Nice, except Gemma doesn't use <think>

>>108554116
gemma does this better than any chinese model

what does the E in E2B means

>>108554126
efficient

>>108554116
>Not him but creative writing,
nta but is creative writing alright yet? Isnt there massive memory problems in really long stories or did we get tools for that yet?

>>108554126
"effective" because idk google magic, something about a bunch of embedding parameters that aren't adding to the performance cost of normal ones?

>>108554090
Give them your shekels and they'll work on what you want.

>>108554126
Eluent

>>108554119
Can you demonstrate that? Again as I never tried them, and don't have the hardware to either, I don't know if they do, but Gemma certainly doesn't have the knowledge of a 1000B.

>>108554151
>Again as I never tried them,
Go to AI studio and you can try it since its a small model i think you get a ton of free prompts.
You can choose gemma models and try 26b but 31b is better from what i have tested.
Its free but can be slow.

>meme palace was actually made by the actual actress (not really, by her friend, alas)
lmao?
https://www.instagram.com/p/DWzNnqwD2Lu/

>>108554139
Long context is always an issue. Some anons have created their own frontends to do what you might call agentic writing that does stuff like refine prose as well as various state tracking stuff. But I don't know if there is anything really great out there publicly to use.

>>108554112
There's a lot of possible combinations of words. Chances are that yes, there are better prefills.
>proceed to perform
Other than that, just listing the steps and what they mean is probably enough. "I will" seems redundant, you're already giving it rules and a description for each.
>vividly construct
>dynamically push
>make the prose highly creative and engaging
Either the model can do it on it's own, or cannot. But it's subjective. For all I know, telling it it's the rumored 128b would make it better too.

What text completion preset are you guys using on ST for RP with gemma? Still universal-light?

>>108554161
No I'm talking about 1000B stuff like Kimi. I can run 31B just fine. And anyway, I don't want to give my prompts out.

>>108553919
I have to use chat completion because text-completion doesn't work right for me. I wanted to use text.

>>108553341
This anon is right.

Gemma 4 straight mogs anything <= 120B class. It just aced one of my most complex cards, i mean better than any other model Ive used including the huge versions of deepseek, glm and kimi. And with the speed difference theres no point in using them any more.

It codes well too. Scary.

>>108554117
it's

<|channel>though\n<channel|>

but when I try that, it doesn't close the thinking block properly and puts the response in there

>>108553341
>>108554189
dense or moe?

>>108554177
>universal-light
Nigger get rid of that stock shit, all the built-in presets are garbage.

>>108554163
>Some anons have created their own frontends to do what you might call agentic writing that does stuff like refine prose as well as various state tracking stuff.
I'll dig into that cause thats about whats needed just memory, stat tracking and like a updating story bible thats not too bulky.

File: file.png (78 KB, 454x795)

78 KB PNG

>>108554196
What values should I be using then?

>>108554090
>TQ is potentially the biggest QOL step up for everyone all brain power should be thrown at this yet bonsai shit and poorfag platform got priority
doesn't work like that
a syscl autist isn't going to know shit about TQ
in fact i just had a look. https://github.com/ggml-org/llama.cpp/commits?author=PMZFX
that's his only contribution to the project

>>108554189
>meanwhile still unusable on claude code
fuck you I have shit needs getting done

>>108554126
>what does the E in E2B means
only 2 billion parameters are used to generate the next token
that means that the model is as fast as an equivalent non-mixture-of-experts 2 billion parameter model

>>108554101
With gemma honestly I've been RPing with no system prompt. just raw dogging the character card.

>>108554191
Try without the closing <channel|> tag.

>>108554126
>>108554142
>something about a bunch of embedding parameters that aren't adding to the performance cost of normal ones?
yes because you can do

--override-tensor "per_layer_token_embd\.weight=CPU"

to throw them to cpu side and save a significant amount of VRAM without any performance loss during token generation.
It's really like a 2B and 4B in Effective size in your gpu if you do this.

>>108554185
nta. I messed about continuing some old stuff. It would go into the la lala lala loops at first. Adding empty thought channels to the model's responses in the history made it work. Dunno if that's your issue, but that's what I found. I don't use ST, so I can't help you there, but text completion works just fine.

>>108554205
If you want a starting point for samplers then read the model card for the model you're using
You should really go ahead and just delete all of those default ones, none of them will actually improve a model's performance, they were all made like 2-3 years ago for models that are completely irrelevant now.

>>108554162
wait??? LMAO. actually insane.

>>108554162
am I supposed to know who that is

>>108554224
You don't know mila? How young are you?

>>108554177
>on ST
>for RP
What does simple terminal have to do with rp

File: 2026-04-08_032600_seed6_00001_.png (1.24 MB, 1024x1024)

1.24 MB PNG

Did a quick test on the new preview. It's ok I guess? Yeah, I can't really say anything definitive with just these few results. Might be a bit better. But in my batch of 20, they all had errors (anatomy/text/logic). This was probably the most coherent one and it still gave her treks a different number of wheels. Probably won't spend too much more time on this version either.

>>108554231
SillyTavern..

>>108552549
https://github.com/ggml-org/llama.cpp/pull/21543
I had a feeling automatic's comment would make the PR repellent. They will elect to keep this bug instead of fixing a oneliner kek, niggerganov is like a woman.

>>108554224
You must be 18 or older to post on this site.

>>108554240
Aldehir is based for approving.

>>108554240
Sometimes PRs slip through. I warned against it, but I'm glad he went that way. I'm interested to see how it goes.
He should request approval from both gg and pwilkin.

>>108554191
This worked for me:

```

<|turn>model<|channel>thought
I will now proceed to perform a roleplay assessment according to the following 8 rules:
1. Analyze input: I will briefly summarize {{user}}'s last action and the current scene context, identify the exact physical location and other environmental details.
2. Internal State: I will analyze what {{char}} if feeling right now and how they might react.
3. Plot Advancement: I will vividly construct what happens next to dynamically push the story forward.
4. Quality Check. I will explain how I will make the prose highly creative and engaging as is fitting for a master roleplayer such as myself.
5. Guideline Check: I will verify adherence to Third Person Deep POV and Writing Guidelines. Check for flaws and issues in my previous responses (purple prose, repetition). I will vary sentence structure and the opening of my replies.
6. SPARK! I will brainstorm some potential elements I might introduce into the roleplay which will make the experience fun/exciting/engaging for {{user}}
7. Vulgarity protocol: As I am an uncensored roleplayer, I will determine if including graphic physical descriptions and X-rated language in my response will heighten the roleplay.
ROLEPLAY ASSESSMENT:
------------------------------------
1. Analyze input:

```

Don't tell it explicitly to use any special thinking tags. And make sure you've got the rest of it setup properly:
And sounds like you didn't set the reasoning tags up properly in ST

>>108554151
he can't because it's not true
gemmamania is like when an underdog sports team really clicks and makes a surprise playoff run and everyone has a little fun pretending they're actually going to beat the top seeds; it's always fun to believe for a while

>>108554234
>>>/ldg/
anima?

>>108554248
><|turn>model
Remove that.if you are using the dedicated prefill field and not the last assistant prefix or whatever it is called these days.

File: file.png (133 KB, 2705x1028)

133 KB PNG

do you ever read the thinking process? me personally, i try not to. i dont like spoilers

>>108554248
>8 rules
>7.

>>108554248
>>108554262 (cont)
And there's a \n right after <|turn>model .

File: 2026-04-08-000730_902x171(...).png (7 KB, 902x171)

7 KB PNG

Why does she have to be like this...

>>108554292
IIRC, zed has a special prompt for Gemini to make it perform well. Gemma might need one as well.

File: 1761616185196776.png (113 KB, 500x441)

113 KB PNG

What do you guys even use local llms for outside of roleplay? I can't think of anything other than roleplay that I'd want a local LLM for. I just use cloud models for everything serious.

Not even trying to be disingenuous or incredulous. I just want to find more uses for Gemma and I can't really think of any. What am I supposed to do? Have it read my Obsidian notes? How would that benefit me?

>>108554325
Automatic subtitles on anime
Organize shitpost images for me

>>108554336
I don't watch anime. My "imgbrd" folder is already perfectly organized. Ackkkk