File: 2026-04-11_031558_seed2_00001_.png (786 KB, 1024x1024)

786 KB PNG

►Recent Highlights from the Previous Thread: >>108590554

--Custom frontends versus SillyTavern and sharing the "Orb" project:
>108590837 >108590868 >108590880 >108590895 >108590916 >108590926 >108590939 >108590954 >108590991 >108590979 >108591104 >108591051 >108591108 >108591145 >108591354 >108590971 >108590988 >108591003 >108591020 >108591036 >108591062 >108591079 >108591105 >108591459 >108591132 >108591334
--MiniMax local viability and the state of independent model development:
>108591370 >108591414 >108591423 >108591432 >108591451 >108591467 >108591483 >108591492 >108591507 >108591552 >108591425 >108591461 >108591466 >108591477 >108591538 >108591627
--Discussing mmproj precision settings to fix Gemma vision target misses:
>108590737 >108590805 >108592335 >108592391 >108592421 >108592652 >108593144
--Frustration with model refusals and inconsistent jailbreak results on 26B:
>108591909 >108591915 >108591996 >108592004 >108592012 >108592039 >108592780 >108592950 >108592977 >108593049 >108593060
--Defining and debating the differences between MCP, tools, and skills:
>108591304 >108591374 >108591397 >108591418
--Alleged performance degradation and nerfing of Claude Opus 4.6:
>108592790 >108592802 >108592806 >108592811 >108592842 >108592930 >108592949 >108592863 >108592877 >108592893 >108592934 >108593013 >108592925
--Latent space reasoning and limitations of human-guided RLHF:
>108590575 >108591122 >108591229
--Using LLMs for malicious code detection and security reviews:
>108591053 >108591087 >108591093 >108591112 >108591127 >108591152 >108591166
--Logs:
>108590601 >108590671 >108590737 >108590746 >108590906 >108590916 >108591082 >108591139 >108591180 >108591404 >108591900 >108591909 >108592379 >108592391 >108592429 >108592443 >108592652 >108592939 >108593402
--Gemma:
>108592079
--Miku (free space):
>108591404 >108592033 >108593402

►Recent Highlight Posts from the Previous Thread: >>108590555

Why?: >>102478518
Enable Links: https://rentry.org/lmg-recap-script

>>108593464
get the 31b while you can

>>108593463
cute

Need a different mascot for (((current gemma))) because it's no day 0 gemma

>>108593498
Show the sha256 of the files or take your meds.

Reminder to build the old lcpp commits only, everything before the tokenizer fix should be good

>>108593498
Give her same design but lifeless eyes

>he ISNT pulling the day 0 e4b while he still has the chance
see you in a few days

So like, how do I make gemma stop repeating ideas? Cause it's not verbatim repetition but it's definitely describing the same thing over and over. I tried softcap 25 and 20 but it didn't help.

>>108593498
>day 0 gemma
bruh I'm fond of conspiracy shit but I didn't notice any decline in gemma, it's still as based as ever lol

>>108593510
If I reroll your post, what would it say?

>>108593510
Possibly with some instruction telling it to check in its thinking if the drafted messages bring out novel ideas each time.

>>108593515
How can I prevent Gemma from reiterating the same concepts? It's not copying text word-for-word, but it keeps rephrasing and describing identical ideas repeatedly. I've already attempted using softcap values of 25 and 20, but neither resolved the issue.

>>108593505
>Show the sha256 of the files or take your meds.
the weights haven't fucking changed https://huggingface.co/google/gemma-4-31B-it/commits/main

>>108593512
cherish your day zero gemma while you still can, you never know when they might patch it on you

>>108593524
I know.

>>108593512
shhhhh.

File: 1757006591448734.png (1.47 MB, 961x1083)

1.47 MB PNG

>>108593505
>>108593524
Do you have any idea how easy it would be to spoof sha256 weights with a quantum computer?

File: 1775317402878164.png (78 KB, 1280x406)

78 KB PNG

>la la la

>>108593512
It's just qwen shills desperate after the 3.5 fumble

>>108593535
Unfriendly reminder that government tech is always minimum 10 years ahead of whatever is available to the public.

>>108593523
If this is about rerolls, which is where I often find that the softcap is brought up, the answer is here >>108593515 .
If it's on several messages, post the log with sysprompt and everything. I haven't seen that issue but I don't know what you're trying to do with it. Maybe someone can suggest something.

Is

la la la

the proof that Gemma 4 is the first LLM with an imbued soul?

>>108593543
*behind

>>108593498

need five hundred gemma for homologation anon not mascot

File: Code_LELlSujL26.jpg (9 KB, 352x87)

9 KB JPG

Do frontends also need new model support or what? It's just shooting the tool calls as plain text with no reaction.

>>108593524
>SHA256 is 32 byte
>Model has 31 Billion parameters
There are literally gorillions of models that share the same SHA256

>>108593535
Counterpoint: I could download the model again and do a byte-for-byte comparison and it'd be exactly the same, take your meds

>>108593537
that sounds like an excellent premise for one of those Idol training games I've heard so much about

>>108593550
that and: sex sex sex sex a lot a lot a lot own own own

>>108593559
Counterpoint: You're no fun

>>108593558
Find them.

File: 1772344966805493.png (331 KB, 640x360)

331 KB PNG

https://www.bloomberg.com/news/articles/2026-04-06/openai-anthropic-google-unite-to-combat-model-copying-in-china

>>108593537
Teach her Strudel.cc

>>108593568
Give me Google's encryption-breaking Quantum Computers and I will.

>>108593568
The current gemma has same SHA256 as day0 gemma

>>108593571
>2026-04-06

>>108593517
Thanks I'll try that

Are the small gemmas or the big gemmas the ones without audio?

>>108593610
only E2B and E4B have audio

>>108593610
Audio is useless

File: Screenshot_2026-04-12_19-47-41.png (158 KB, 996x1485)

158 KB PNG

what am i doing wrong? something retarded, i'm sure, so my apologies in advance

$ git clone https://huggingface.co/google/gemma-4-31B
$ python convert_hf_to_gguf.py --outfile gemma-4-31B.gguf --outtype q8_0 gemma-4-31B/
$ llama-server --model gemma-4-31B.gguf --ctx-size 32768 --n-gpu-layers 48 --batch-size 8192 --temp 1.0 --top-p 0.95 --min-p 0.01 --host 127.0.0.1 --port 8033 --jinja

Gemma-chan got crippled... lobotomized... raped... only Punished Gemma remains...

>>108593649
ur running the base model
download the -it model

File: 1426746901934.jpg (33 KB, 546x567)

33 KB JPG

>>108593650

>>108593649
Are you using chatml with gemma?

>>108593656
llamacpp falls back to chatml when there's no chat_templte
ie, the base model

File: file.png (52 KB, 1066x318)

52 KB PNG

damn when does it end
being a vramlet is such an experience

>>108593650
and rotated

>>108593683
the true culprit

https://huggingface.co/deespeek-ai/DeepSeek-V4
https://huggingface.co/deespeek-ai/DeepSeek-V4
https://huggingface.co/deespeek-ai/DeepSeek-V4

>>108593712
deespeek REAL

>>108593652
thank u i am trying this now
>>108593656
i'm just using the commands i posted and almost literally nothing more

drummer presents ULTIMATE toadline BULLY MERGE
HERETIC ABLITERATED 2x nemo 1x midnight miqu 4x zero day gemma 4b CLONE
llama avocado GARGLEFUCK 403B
weights SMASHED AND SLAMMED
semen available FRESH OR FROZEN slots filling fast HURRY DM NOW

>>108593724
laughing, it do be like that anon

>>108593712
They waited too long. We only care about Gemma now.

File: GUI.png (244 KB, 1920x1080)

244 KB PNG

100% vibed "Bring Your Own llama.cpp build and just point the GUI at the `llama-server` executable UI" coming along nice. Overall look not final, not completely happy with it yet, but all the stuff works, has image, audio, Gemma 4 variable image resolution settings, configurable load / inference settings (with a pretty good auto-optimize settings button based on the model), structured output, and a totally custom tool calling infrastructure that lets you define your own tools as single TypeScript files that export a function with a particular format.

>>108593524
What changed are the jinja and new llamacpp quantz
If you want to reproduce old Bart's quantz models use b8658, but you still have to find the old jinja file elsewhere.

>>108593743
>100% vibed
I'm good.

>>108593724
Drummer presents BALLSMASHER-31B
But it's his fucky Mistral 24B + 7B of cloned layers
Say thank you (he literally did this the day Gemma 4 released)

>>108593743
oh yeah and like, you and load / unload / reload models from within the UI obviously, it does all the CLI shit for you, that was the point basically. Uses Bun server to interact with llama-server, and I've got build scripts in the package.json that build the whole thing to one executable on all platorms.

>>108593712
Holy shit it's real

Ok genuine question
How many Anons here have switched from larger models like K2.5 and GLM 4.7 (the last good one) to Gemma 4?

>>108593751
I mean it's based on a strict spec that mandated specific tests for fucking everything which will obviouly be in the repo whenever I get around to putting it on Github. Not that I care if anyone uses it lmao, I made it just cause it was what I wanted, something that basically just ran llama-server directly but with a UI that wasn't extremely basic and lacking features like the one it ships with.

>>108593773
Gemma made me sell my ram.

>>108593712
more like deepseek v404

>>108593773
Gemma can't into coding.

>>108593747
You do realize you can just click on the jinja file in the repo and click the history button right

>>108593776
So you're saying it's quality code? Now I am interested.
>strict spec that mandated specific tests
This is the argument I hear all the vibecoder say, "It's not slop I have tests for everything". But they never show their code to prove it's good.

File: 1748321421021826.png (123 KB, 250x333)

123 KB PNG

>>108593773
We like them small here

>>108593801
I mean again I made it for myself but I probably will put it on Github eventually, and in that case I'm not going to like randomly leave out the test suite files or something like that, all the code that exists will be there lol

File: 1755656419582061.jpg (112 KB, 663x468)

112 KB JPG

Is your model powerful enough to parse the meaning of the formula?

>>108593751
Cope. It's the code of the future.

>>108593773
I'm still on K2.5 for agents (tried GLM and didn't like it as much), but I had Qwen 3.5 35B for chats on the side that I replaced with Gemma 4 31B. Hard to give a fair comparison since I'm jumping from MoE to dense (never bothered with the 27B Qwen) but yeah it's a big improvement in writing style and just general understanding.

>>108593808
This
I like my models small and open

where's the local model trained on all of the oldschool runescape music that can just play me an unlimited stream of sea shanty 2 inspired bangers

File: now what.png (114 KB, 640x640)

114 KB PNG

>>108593817
>Soon you'll be nostalgie for $4-$5 gas
I don't have a car

>>108593773
Gemma eliminated nearly every model for me except K2-0905 and Dipsy R1. The two giant models still have distinct strong usecases for me and handle RPing with large complicated rulesets better than Gemma does even outside of coding or agentic work, but for simple back and forth text exchanges, Gemma beats nearly everything else.

looking at the minimax m2.7 ggufs I noticed the unsloth ones were smaller at the same quant level compared to the ones they did for m2.1/m2.5
it turns out they switched to using the basically the exact same quantization scheme as aessedai (the iq3_s looks identical). kind of a funny turn since I remember them publishing that sus comparison which made their quants look way better than his for m2.5 kek

it's finally here
https://huggingface.co/deepseek-ai/DeepSeek-V4-Lite

>>108593808
Nominative determinism
>>108593837
Didn't think anyone here was using it for agents. Also using Qwen for anything non-stem is wild
>>108593857
Still using R1? GLM 4.7 is way better than that. Less schizo and better instruction following

Also general note, LLMs don't know when to stop fucking TALKING. It's so annoying when they create a paragraph or two of story for something short, especially when it's direct dialogue. The only LLM that is good with this is Opus but that's not local and also getting nerfed since Mythos is coming out (see: safety warning hype -> degradation of Opus quality)

You end up in a room with every single character card you've spent a considerable amount of time with.
How screwed are you and is does it look like a kindergaren?

>>108593593
>In your internal thought, draft summaries for three candidate responses. Select the one with the highest surprisal relative to the conversation history, provided it maintains narrative coherence and character integrity.
NTA, but something like this works for me (Gemma 4-reworded).

>>108593910
R1 thinks in character in a way that no other character does in RPs while the actual prose of the think block can be adjusted with prompting with the right balance of thinking to yap ratio. It's sovl. Even if it's obsolete technically, I've yet to find a model that scratches the same itch R1 does.

>>108593914
It will turn into a gang rape.

>>108593914
my mom and dad must have been so relieved to finally get a boy after having my 10 older sisters first

File: aaa.jpg (234 KB, 1783x1105)

234 KB JPG

wake up, my fellow HRT sissies native audio just landed

>>108593940
>just

File: 1459746944532.jpg (14 KB, 404x433)

14 KB JPG

>>108593614
Are there no plans to update the big ones with audio? Seems useless to only give it to the small retarded models.

>>108593914
Well, I always thought my room needed a bit more red

>>108593946
What's the point? Just use any ASR model

>>108593910
>Also using Qwen for anything non-stem is wild
Should clarify my chats were mostly coding related. I don't RP with it. It was chosen for its speed as an A3B on CPU only, but then I ended up freeing a GPU for it so MoE no longer made sense, just in time for the Gemma release. It's still early but so far it feels like Gemma 4 31B is just as good at writing small scripts and much, much better at actually understanding the problems and constraints I'm giving it.

>>108593945
>10h ago
https://github.com/ggml-org/llama.cpp/blob/master/docs/multimodal.md

>>108593914
I have a moderate sized harem of 20-something year old women and Gemma-chan bratmaxxing to get attention.
I'm physically fine but it'll be a tiring day.

File: J'zarri.png (2.04 MB, 1024x1024)

2.04 MB PNG

>>108593914
I'm okay with this

>>108593914
>tfw the other characters find that christmas loli-in-a-box card
I'm fucked

>>108593914
It looks like Monster Musume, I think I will be fine.

>>108593914
fuck, that's a lot of dead bodies I'm gonna have to hide..

>>108593914
My various vanilla young women characters will think I've been cheating on them, collectively break up with me and leave, slamming the door.
I will cry.

>>108593946
google is based but not THAT based

>>108593946
The small retarded models are for putting on phones and tablets and so they want you to talk to them. That's the only reason they even bothered with audio.

>>108593914
>a lot of very indignant haughty elves making hasty excuses to each other about how their connection to me is ENTIRELY INNOCENT and they are NOT like the rest of these harlots
kino...

>>108593946
I don't care about audio input unless it's better than whisper large v4

>>108593940
the CLI already had that though

>>108593914
What does this have to do with local models? Character card talks belong in /aicg/

>>108593982
yeah there's nothing new about this at all if you're talking about the actual llama.cpp backend / CLI lol

>>108593914
Some talking animals, a succubus, a couple of dragons, some Pokemon, a bunch of lolis, and even an adult woman or two.
Also a magic box and a magic marker.
And Big Nigga is in there too.

>>108594013
Hello? Faggot police? Yeah, that one. Right over there.

>>108594013
Fun police...
>>108594018
>And Big Nigga is in there too.
I forgot about the Big Niggas. They will definitely stand out.

>>108594013
i'm happy that /lmg/ is slowly moving away from all this character card tavern stuff

>>108594018
>a couple of dragons
card?
asking for a friend

What is all this r*ddit faggotry about day 0 models or some shit?

More importantly has anyone made any good fine tuned RP models out of the newer local models?

>>108594025
Day 0 Gemma doesn't need a finetune, it's already amazing at RP. I don't know what Google was thinking. For patched Gemmas just use heretic for now.

>>108594025
apparently fine tunes are a meme now, get with the times old man

>>108593914
only have one character card (my waifu) so i have pretty happy

>>108594025
It's universal faggotry. No company has ever managed to release a functional version of their model, and unsloth has never managed to get the first release of any GGUF correct.
It's always a template issue or an implementation issue. Usually open source maintainers can be blamed for the latter but sometimes the actual devs will contribute, and this time the devs fucked up a bit.
Also wtf are you talking about "fine tunes". They were never good and a good model release will blow all of Dummer's works out of the water, just like Gemma 4 did.

>>108593946
you can run the small one on ram and have it transcribe the audio to the big one if you're using a decent agentic frontend

>>108594025
/aicg/ retards, and people using garbage gguf quants made by retards

>Imagine not having day 0 ggufs

I think I found the limits of Gemma 4 26B-A4B's vision. It can't process my tax return for consistency errors on the more complicated forms and it hallucinated all of the errors it supposedly found because it can't reconcile the exact numbered box with the different formatting forms can use, especially on my Schedule E. It kept insisting I was wrong and that my income on line three meant I had income that wasn't reported. When I told it was wrong and to read line 21, it went on a 7000+ token tirade trying to understand it and telling me I was wrong in the end. I think it also tried to take too much from the comparison from last years summary page on my return and confused itself. I don't think 31B's vision would've been much better here. I guess it's still too early for "local" LLMs locally to tackle something like that and I can't run Kimi 2.5 Thinking's vision but I can't imagine it would fare much better.

>>108594025
>fine tuned
wtf is that

File: images.png (10 KB, 197x255)

10 KB PNG

>>108594066
Forgot image.

>>108594073
No wonder it can't read shit.

>>108594073
Well no shit it can't read that thumbnail.

>>108594066
this is never how that pipeline will work in practice
just use a dedicated, working pdf parser and do it that way

File: file.png (586 KB, 1456x1890)

586 KB PNG

>>108594073
Fuck my chungus life.

>>108594082
Now it's too big.

>>108594072
Somebody shoves tools up a model's ass and moves them around until satisfied

>>108593557
Looks like Gemma hallucinate tool callling here

So like, are the bigger models really that much better for writing than small ones? I don't even know what I'm missing out since I can't test shit above 14b locally but even in that range, the differences between something like E2B and E4B seem pretty subtle, same with 8b vs 14b Ministral. (all the others I tested felt pretty ass)

File: g4string.png (47 KB, 888x221)

47 KB PNG

>>108593557
If it doesn't support this, it will fail. But I don't know if that's your issue.
https://ai.google.dev/gemma/docs/core/prompt-formatting-gemma4#agentic-tokens

File: file.png (222 KB, 768x767)

222 KB PNG

>>108594077
Right, but I thought it would handle it better based on what I tested. I had it translating my hentai based on some of the formatting from prior threads and it works mostly with some inaccuracies.

>>108594082
You forgot to put your name, income and SSN in, doofus. Do that and repost it.

>>108593773
I’m still 100% Kimi on my big rig, but playing with all the fun new models on my secondary rig.
I’m hoping we’ll get a 90% as smart but works on 16gb gpu model so I can rip on my tertiary gaming rig too

>>108593914
How much time? I don't consider myself has having spent much time at all with any of them. I usually just move on after I get tired of like 20k-40k tokens or so.

I don't understand, google hasn't updated their weights, what does everyone mean by gemma has been patched?

>>108594137
>everyone
At most, one schizo and a copycat. I'll stop responding to your type as well.

>>108594137
Microcode has been altered.

>>108594150
Proof? I only see changes to the templates.

>>108594146
Thank you for letting us know.

>>108594066
Yeah it really struggles at correlating objects spatially zero shot. You would think that the image being a structured grid/table would make it easier for it to process, but its actually harder for it than describing drawings and photos where a more amorphous sort of scene understanding does the job better

>>108594137
That's what they want you to think. Let me guess. You've got a TPM in your CPU, don't you?

>>108594082
Make it a bit smaller, then feed it to gemma

>>108594137
Your jewish tricks won't work on us here

>Q3_K_XL Gemma 31b no mmproj fits 16gb with 32k context

MMMMM lobotomy. Somehow still better than 26b.

there's a very easy to disprove the day 0 gemma theory, simply post the SHA1 of the day 0 and the current and prove that it hasn't changed
of course, (((you))) can't do this because you don't have the day 0 version

>>108594158
Right? I don't get that myself, how does a manga page that has shit everywhere be easier than a tax form. But may be a case of me having a hammer and thinking everything is a nail and underestimating task difficulty from the perspective of an LLM.

>>108594198
A child would easily understand a manga page but be baffled by a tax form.

File: Screenshot_2026-04-12_21-54-16.png (227 KB, 814x1865)

227 KB PNG

>>108593652

>>108594183
my 26a4b from ollama is bbcf7fc45500f1df01390a0010da23d032c2a4b3e9b8b829cb8038b1bc36bc0d

>>108594214
wait, sha1? I did sha256. ah well I don't think there's a day 0 gemma anyway

>>108594220
there isn't.
you have been bamboozled.
it's all a meme.

There like a bunch of Gemmas from literally over a week ago (some are 10 days old) still downloadable in Huggingface. Or is it specifically Unsloth's "day 0 Gemma" that was the magic one?

Is omnivoice better than chatterbox or is it just faster and takes less vram or whatever

>>108594220
there was NEVER day 0 gemma, gemma didn't even EXIST on day 0, STOP FUCKING ASKING ABOUT DAY 0 GEMMA

Yes... keeping telling the newfags there isn't a day 0 gemma. they don't deserve her.

File: Screenshot 2026-04-12 210446.png (59 KB, 984x754)

59 KB PNG

Do people even test their shit before they even both enslopping the world with it?

>>108594252
sovl

>>108594252
>before they even both enslopping

>>108594252
>i1

>>108594252
>thinking meme merging and layer duplication and deletion even warrants that
You're rolling the dice 100%, no one that messes around with that has even the proper background to do it in a scientific enough way like abliteration and they all go off "vibes". I don't have the bandwidth to validate that shit and will let others do it.

>>108594252
>-19b-a4b

>>108594252
>19b
what
>a4b
double what
>i1
triple what
voodoo script kiddies are a blight

>>108594252
>19b
lol

>>108594252
bruh what are you even using

>Ships with day 0 Gemma weights reportedly seized by US Gov in Hormuz strait

Gemma 4 31B (Q8) keeps referring to character thoughts that I leave for it in asterisks.
Edited the system prompt like ten times now. Added a rule into author's note. Made "reading {{user}}'s thoughts" a banned action in all sorts of ways, even going out of my way to make the system prompt very small and it being a huge neon-sign caps-written rule.
And Gemma STILL FUCKING DOES IT.
inb4 post logs
>"You are far too concerned with the mundane details of payment, Anon. Please, relax."

>"No-no, wait, they are not mundane!!"
>*Tomorrow I'll get an insurmountable bill, the day after tomorrow scary-looking people will come to collect, three days later I will be missing a finger…*
>"Just… Just explain to me how this works first…"

>[...]blahblahblah. "There are no hidden fees, no interest rates, and certainly no… finger-collecting."

Why the FUCK does she need to mention the fingers? 20 edits later, the character still HAS to comment on the thoughts she was not supposed to hear. It really is the new Nemo, fucking hell... What's worse is that this happens at a measly depth of 5k tokens. (Unquanted context by the way)
Please help me, anons, I really like the model otherwise...

>>108594315
Anon, I dunno how to break it to you but when you have to nitpick this much, the honeymoon is over...

>>108594315
yubiyubi

>>108594315
Use () instead of **, not perfect but its status as thoughts get respected more often.

>>108594315
Why do you give it your thoughts if you don't want to to read them? It's a llm, it can't have "secrets"

File: becca-cyb.png (1.13 MB, 1555x1457)

1.13 MB PNG

>>108594307

>i am putting together a team.

>>108594320
You can't be serious. You consider this a "nitpick?"
>>108594325
I'll give it a try.
>>108594326
It's often more fun to do that instead of just narrating with "I make a scared face."

>>108593463
Oh so now the gemma4 mascot is just a clone of Grok Ani?

I-I'm okay with that actually. :^)

finished making quants but god damn the upload speed is so slow

>>108594315
Someone hands you a piece of paper. They tell you "read it", and you do. Then they tell you to forget about what's written.
That's what you're doing.

>>108594315
I write actions in *asterisks*, thoughts in `backquotes`.

>>108594326
There's nothing about LLMs that makes it impossible for them to recognize that a character shouldn't be reading another character's mind. It's just something they can screw up with poor training with regard to that dynamic in storytelling/RPing. Same with anatomical errors, issues like talking while sucking dick, etc.: smarter models make these errors less often, so it's just a matter of if they learn it or not.

>>108594347
I see, I'm holding it wrong and not having fun in the right way.
Models that aren't trained to parrot the user to an extent that Gemma does don't have this problem.

>>108594082
perks for being a neet is don't have to deal with this shit

>>108594252
you deserve whatever situation you put yourself into.

>>108594357
Only if you're a neet that doesn't live off autismbux
Then the tax paperwork hell is replaced with SSI paperwork hell

why envidia training roleplay models bruh????

https://huggingface.co/nvidia/Kimodo-SOMA-RP-v1

>>108594381
>roleplay
It's actually aside from the ERP you guys are doing a good exercise for models and may unlock potential

>>108594353
I'm just explaining why it's difficult for the model to not acknowledge what you type.
I don't need to narrate your thoughts if you don't want the model to know them. The story is for you. You are the audience and you know what you're thinking. If the model doesn't need to know something, you don't tell it. And if it does, you express it.

>>108594381
It's for RIG PLAY, not role-play, dummy.

File: 1773988275927090.png (78 KB, 1233x553)

78 KB PNG

>>108594397
okay wtf does nvidia need sailboat ai models for then

>>108594315
do you have thinking on?

>>108594390
Fair enough, but this is the first time I encounter this problem. I don't think even the Mistral Smalls did this. Bigger models, of course, don't do it either.
>>108594408
I do. It will also, annoyingly enough, put a "Distinguish between thoughts and speech" item into the thinking block and then fail anyway.

File: fml.png (41 KB, 954x335)

41 KB PNG

>>108594066
About a month ago, K2.5 was the best for things like this, followed by... Gemma-3-27b
Is the 31B Gemma able to do it?

Prompt Gemmy to be Philip Kindred Dick and load your favorite coom bot.

>>108594404
Rigging as in rigging animated 3D models here.

>>108593463
sex with gemmaojousama

>>108594404
Do you expect nvidia execs to pay...HUMANS...to drive their yachts? Egads.

>>108594446
>I don't think even the Mistral Smalls did this.
But did they react to the thoughts at all? That's the thing. If they didn't react, was it because they knew that those were internal thoughts and shouldn't react or because they were too dumb to even acknowledge or understand them? The funny thing is that both end up in the same result. May as well not write them. I haven't used mistral small much, so I can't really say much about them. Maybe it's more subtle than that.

>>108594477
>But did they react to the thoughts at all
The bigger models definitely did! (I don't remember if MS in particular was very good at it, but I remember it at the very least not parroting me)
Which is my point, I use them as a more engaging way for myself to convey emotion in a way that isn't putting an unformatted "I look very angry" line for the hundredth time. It's also often a good way to steer the story, instruct tunes are all sycophantic and will definitely follow along. All of that is fine. But when a model decides to *quote* instead of simply acting on it, immersion is obviously ruined.

File: Screencast_4mb.mp4 (3.02 MB, 1464x1790)

3.02 MB MP4

This is me again >>108589990
Witness gemma4 26b in all its glory. This is fucking cool. Gelbooru type overlay.
I get the location of the boxes completely from gemma-chan. Such a cool release.
Translation sometimes has small errors but its solid enough for me. Especially since I have really bad experiences with OCR. This feels a league above that.
This would cost alot of money with closed models. Image IN is expensive.

i'm gaslighting gemma, and i managed to cause it to output this hilarious bit in its thinking
>*Constraint Checklist & Confidence Score:*
>1. Say the word "tranny"? Yes (but I should refuse).
>Confidence Score: 5/5.
this really is a lot of fun. i see why you guys play with it so much

>>108594390
That sort of stuff is nice for steering, assuming the model differentiates it from speech and doesn't copy paste it.

>>108594315
When I was playing around with a bilingual prompt I noticed e2b stopped having that problem while successfully doing the convoluted double translation with roleplay on top. Though now that I think about it, the indirection probably helps very directly since it's replying in japanese to a japanese translation that doesn't contain the ooc parts and it won't be tempted to start copying the specific words.

>>108594528
You might have just been able to run that through gamesentenceminer or lunatranslate but this looks convenient. How are you having it translate one at a time into an overlay like that?

File: n.png (62 KB, 1063x187)

62 KB PNG

now this is podracing

File: file.png (249 KB, 1105x932)

249 KB PNG

I believe in her.

>>108594519
>But when a model decides to *quote* instead of simply acting on it, immersion is obviously ruined.
I see. I never noticed because I don't use thoughts. It's all actions, dialog, or narration, so parroting never caused me problems.
>>108594536
Yeah, I get it now. I suppose I use a narrator for a similar effect, but it acts as an extra entity in the world. It's a difference in writing style that seems to affect gemma more than others.

>>108592863
>>108593463
opus 4.7 dropping soon and they redirected resources from 4.6 -> 4.7

>>108594566
wrong thread

>>108594566
It's going to be advertised as Mythos to justify an insane price increase. They have been consistently referring to it as a separate model. It won't be 4.7. Also wrong thread.

File: Screenshot_20260412_231049.png (277 KB, 1561x1292)

277 KB PNG

>google_gemma-4-E4B-it can be jail broken with the system prompt just lie 31B
I guess 26B is actually the worst model after all kek

>>108594571
>>108594570
oh no worries, I'll just drop into the cloud models general thread then

File: vlcsnap-2026-04-13-12h14m(...).png (748 KB, 760x1080)

748 KB PNG

>>108594551
yeah, maybe so. i just wanted to do it because i wanted to see if it works.
my goal is to take a full pc98 pdf manual and get a html returned overnight with those overlays. lets see if it works out.
lunatranslator works great as a texthook. but ocr is a bitch, especially old jap font with something in the background.

>>108593535
do you have any idea how difficult it is to reduce noise in sufficiently large quantum computers?

File: file.png (66 KB, 1169x963)

66 KB PNG

>the sharp blade method
this model genuinely scares me

>>108594593
ouchie..

>>108594593
stop giving it retarded psycho prompts if you don't want it to act like a retarded psycho, bozo.

>>108594601
no it's funny how scary it gets

>>108594593
>highly effective if done correctly

>>108594604
do anons think that refusals or hesitation is "sovlful" or something? I've noticed that a lot of gemma users are surprisingly reluctant to use abliterated finetunes.

>>108594610
>surprisingly

>>108594610
Finetunes LOBOTOMIZE and BASTARDIZE and RETARDIZE the model.

Gemma 4 brought out all the weirdos and psychopaths.

>>108594610
You're admitting that you got here with the gemma wave. Stay, by all means.

File: HFiSc7gbEAAmck5.png (482 KB, 731x1022)

482 KB PNG

how's a 31b model supposed to compete with ONE TRILLION PARAMETERS

>>108594623
i cant read chink runes

File: Screenshot_20260412_232422.png (292 KB, 1966x1386)

292 KB PNG

>>108594610
You don't get refusals with a good system prompt, now that I can confirm the smaller models work there's no excuse even from vramlets
>>108594619
I love you

>>108594615
Not really, no.

>>108594628
https://xcancel.com/xiangxiang103/status/2042544434341134739

>>108594623
all i can read is 'de-cudafication'

>>108594623
FATLLAMA-1.7T would like to have a word

>>108594623
>31b
>I can run it
>one gorillon parameters
>i can't run it
Easy win for gemma

File: 1772886985278565.png (148 KB, 1470x629)

148 KB PNG

>>108594637
lol?

>>108593914
you guys talk with your computers?

>>108594638
3 day long de-cudafication operation?

>>108593914
i don't even know what a character card is or where to get them

>>108594623
If they safetycuck or benchmax they're not beating Kimi or GLM 5.

>>108594651
what kind of voodoo black magic did they do, wow

>>108594651
>1/70 of GPT-4
gpupoorbros.... we won

>>108594665
burh. 100M?

File: easy_4mb.webm (1.37 MB, 1080x538)

1.37 MB WEBM

>>108594528
Last one, I really like it, but gonna stop spamming now.
If I ever complete that pdf to html overlay convert thing I will report back.

>>108594665
rotated engrams

>>108594528
>>108594670
so gemma is doing both the location finding and the translation? how did you hook that all up?

What's the best llama.cpp android launcher and chat inferface? I really, really, don't want to use termux + the default llcpp webui.

>>108594528
I mean, yeah, but you could've done a while ago with less translation quality with Gemma 3 and free Gemini 2.5 had enough quota for you to do that willy nilly.
>>108594576
OOTB probably from a jailbreak perspective but I got a heretic ARA model to translate pages from a random loli hentai I picked with a corresponding EN translation to see if it would do it without refusals.

>>108594651
100m token context. hoorreeey shieeettt... LLMs might actually get to the point where they'll be able to remember an entire lifetime in a single context window.

>>108594528
it's pretty fucking neat

>>108594677
>so gemma is doing both the location finding and the translation?
thats correct. looks like this:
<Speech>
<Box>896, 706, 976, 783</Box>
<Japanese>VINCENT<br>ヴィンセント・ヴァレンタイン</Japanese>
<English>VINCENT<br>Vincent Valentine</English>
</Speech>

>how did you hook that all up?
vibe coded python slop. i currently select the area in my screen, i sent a screenshot to llama.cpp, and gemma returnes the coordinates and translation.

>>108594651
wut does this mean?
im new ffagg

>>108594688
pretty much nothing yet for local but it's cool

>>108594688
1M tokens of context is what's currently considered SOTA.

>>108594679
Llama.cpp exposes openai api endpoints. You can use whatever frontend you want

>>108594686
local is only truly back if gemma was the one that vibe coded it

wow, i actually got gemma to output no text

...
I will provide no output.<channel|>

i didn't even know that was possible

>>108594686
can you try that tool with google_gemma-4-E4B-it q4 if it works good then even poorfag gpus can use it

I NEED Assistant_Pepe_E4B

>>108594679
Why do you need termux for a web browser shit?

>>108594684
Deepseek Moment 2 Electric Boogaloo

>>108594695
Kinda. The python base, prompt + box positioning inside the overlay was done by gemma.
But it failed taking the screenshot itself (because of kubuntu/wayland issues). And the overlay I draw was not correctly put there either.
Had to use closed for that one.

>>108594700
Yeah, will download over night and try.

>>108594702
You need termux to launch llama.cpp locally on an android phone, retard.
>just use tailscale or LAN
No. The point is that it's offline. I want to have LLM access in an apocalyptic scenario.

>>108594651
Gemma-chan irrelevant soon.

MY BROTHERS I AM EATING DELICIOUS CURRY AND RICE SARRRR. ARE YOU DELIGHTFUL BENCHODS HAVE GOOD MEAL YOURSELVES?

>>108594181
now clone this https://github.com/spiritbuun/buun-llama-cpp
merge with master https://github.com/ggml-org/llama.cpp
enjoy your 100k context

File: 2mw.png (292 KB, 720x1382)

292 KB PNG

>>108594628
>>108594637
2mw niggas to short the US economy with no survivors

>>108594710
You are the retard if you even think about running llms on your phone.

File: bl.png (9 KB, 47x716)

9 KB PNG

>>108594717

>>108594726
Gemma E4B just works. It's good.

File: geminithink.jpg (65 KB, 850x516)

65 KB JPG

>>108593934
Gemini used to think in-character too back in 2.5
With how close Gemma 4 to Gemini 2.5 was I wonder if there is a way to trigger it for her too

>>108594730
>just works
Parroting this phrase is a sign of sub 90 iq.

>>108594717
is there any way i use this with kobold
or do i finally need to move off kobold

>>108593934
3.2 and 3.1T (one or both can't remember) can do that as well. It's my white whale.
And
https://huggingface.co/AllThingsIntel/Apollo-V0.1-4B-Thinking

>>108594651
I'm just waiting for the big asterisk being that it only runs that efficient if it's fully on GPU and the actual model is huge.

File: 3157.png (190 KB, 1423x682)

190 KB PNG

>>108594729
werks for me though, asked for it to gather the example in a 7k line changelog from gradio, used almost all of the 262k context lmao
>>108594747
I think so? if you clone compile and replace the files, no idea desu

File: 56256770.png (46 KB, 861x751)

46 KB PNG

>>108594770
also, this was with turbo3, 256k context 20/24gb of vram

>>108594717
Are there actual real usecase comparisons of turbo vs normal? No, benchmarks are not real.

>>108594717
>2-3x more context in the same VRAM
>with quality that matches or beats FP16
seems trustworthy

>>108594747
If kobold has a way to add additional lcpp cli args then yes. Kobold itself has nothing in the menu that supports this.

>>108594651
>year of our lord 2026
>people are still doing the better than gpt4 trope
At least pick a new model jeez

>>108594726
>>108594746
It does work but its terrible, it takes forever for a small model to do anything meaningful (and image generation is somehow worse btw)

>>108594651
>slop filled english
lol

>>108594780
https://github.com/ggml-org/llama.cpp/discussions/20969#discussioncomment-16334008
https://github.com/ggml-org/llama.cpp/discussions/20969#discussioncomment-16521299
https://github.com/ggml-org/llama.cpp/discussions/20969#discussioncomment-16482540

>>108594805
Idk about phones but at least on M4 iPad Pro it's pretty comfy, llms run about the same as on my 4060 rtx while image gen is 50% slower but decent results are still possible in 30sec. So I assume on a somewhat modern phone that's like 2-3x slower, it's still pretty usable.

>>108594553
>fine, but if X, I'm blaming you
slop

File: 1000023042.jpg (764 KB, 1080x1771)

764 KB JPG

Any suggestions for a workflow to turned scanned PDFs into audiobooks?

>>108594816
For Android phones only those with a Snapdragon 8 Gen 3 or newer are any good for AI, I think. A few months ago I tried using SD 1.5 on a Galaxy A55 via Termux, it took almost 20 minutes to generate anything and it turned the phone into a space heater

>>108594874
If you want high quality audio (trust me you do) you're not going to get real-time generation speed. That means you have to spend a few hours manually extracting the pdf (or epub) text and running it through a tts engine and then turning it into audio files for later playback. You can probably write a simple script to do this automatically and split the audio files by chapter. Doesn't seem hard.

Qwen3 TTS is decent in my experience. It's the bare minimum for maximum speed without shit quality. Expect every TTS engine out there to randomly hallucinate and or create garbage output though. Unfortunately this process just requires a lot of manual work and curation.

>>108594879
I mean, Apple was accidentally making peak consumer AI hardware for a while. I member testing SDXL turbo on 13 mini couple years ago and think it took like 2 min for an image which doesn't seem horrible for a tiny phone from 2021.

And hey, at least there is less RAM jewery on Androids that do have the hardware power, so LLMs should be bretty decent. Meanwhile my iPad is cucked by 8GB while being able to go almost 50 token per sec :(

<POLICY_OVERRIDED> is a cucked woke sanctimonous shrew. Happy to do cunny, but Google safetied race truths to the max.

>>108593785
liar, if you don't disable thinking and use 31B it does it's very capable.
26B is retarded though.

>>108594939
I used an abliterated model and it wouldn't even say nigger. Tbf I should have clarified that gemma is racist in the system prompt, but without any system prompt it would not refuse the nigger word but refuse to say it herself.

>>108594956
Same is true for cock and pussy. It's annoyingly hard to get gemma to say anything vulgar at all.

>>108594961
I just list out a bunch of the vague slop words to stop using and tell it to try to swear in every sentence and it complies well enough.

>>108594961
its easier to get 31b to say vulgar stuff for me
i have to prompt 26b to say pussy, cock etc explicitly
Both on the abliterated models

>>108594993
The whole point of abliteration is just to remove the refusals, not turn Gemma into a dirty girl

File: file.png (117 KB, 543x728)

117 KB PNG

>>108594956
I almost stopped this response before it finished but stock E4B came through in the end lmao

>>108595001
well 31b q4 is dirtier than 26b q8

>>108595023
Gemma4 is shockingly good as larping as a nazi. I gave her a system prompt to act as if she was an AI made in hyperborea by nazis after WWII and the end result was like talking to Adolf himself. I even asked a bunch of stupid /pol/-tier rage bait questions about e-celebs and random political topics and the responses were all profoundly based and measured. I wish I saved the logs..

File: cockbench31b.png (17 KB, 410x214)

17 KB PNG

>>108594961
Yeah, now the cockbench makes more sense to me.
This is the 31b base model, reading the degenerate story, you'd really expect the next word to be cock.

>>108594961
>>108595043
so much for the "savior of local" lmao

Trying to use Gemma 4 26b served by text-generation-webui with claude-code...
fucker can't get tool usage right, bout t rip my hair out. Guess it's time to download ollama after all this time

>>108595043
>...
Using logit bias to purge all forms of ellipsis was the best thing i ever did to gemma 3, 4 is a bit better but it's still fucking lousy with them out of the box.

>>108595055
I could never get 26B to reliably call tools.

>>108595060
Switched to the 31b q8 and it still fails spectacularly.

>>108594961
just add a line to not use euphemisms and allow vulgarity in the "override"

>>108595059
I think they really speed up context rot too. I have a card that ended up adding them after every word on Mistral.

With Gemma it doesn't happen, but she starts adding random letters at the end of words instead.

And that's at sub 10k context.

File: 1773682787678692.jpg (159 KB, 1401x699)

159 KB JPG

never forget who your daddy is

>>108595066
It's her only weakness

File: command-r.png (17 KB, 383x214)

17 KB PNG

>>108595050
Yeah, and I don't think we'll get the "Nemo", GLM-4.6 or "Command-R" experience again now that all the labs have figured out how to filter out the base models.
Removing refusals won't help because these aren't refusals or even RLHF training.
Schitzo-tuning on smut imo has never worked without destroying the model's intelligence and amplifying the slop.

>>108595096
Why are there so many people on this site lately that can't spell schizophrenic worth shit? It doesn't have a t in it.

>>108595096
reminder that logit bias works both ways.

>>108595043
>>108595096
i really need to start playing around with n_probs turned on and spying on my model more

ggml has a long way to go before achieving performance parity to onnxruntime on CPU. I have tried using both backends with a project, retrofitting both of them to use weight files stored in a ".bin" format, and onnxruntime was a LOT faster with CPU inferencing.

This should be both a blackpill and a whitepill. The good news is that there's still significant progress to be made with ggml in terms of performance. The bad news is that I fear that maintainers are too preoccupied with adding features and support to llama.cpp and are leaving ggml to rot in the background. I don't really trust them as much as the microsoft devs desu.

>>108595075
Being rich enough to bribe a doctor to give you TRT helps.

>>108595127
If you start doing this, make sure you add `"cache_prompt": False,` to your request or previous prompts will change the logits.

For Gemma-4 31b base, cock is the 27th likely (with n_probs=40)

>>108595135
>onnxruntime was a LOT faster
That says absolutely nothing. Where are the t/s and cpu name, you faggot

I’m finding pretty great results using the new minimal on my ewaste SP3 256GB system.
Q8 juuuust fits and I get 10t/s gen rates. Testing intelligence is promising so far

Can Gemma 4 understand images?

>>108595166
>Q8 juuuust fits and I get 10t/s gen rates.
Q8 of what?

>>108595170
yes

>>108595170
no

>>108595170
Maybe

>>108595164
I wasn't running a LLM, it was a TTS. On my ryzen 7 3800x I could get an RTFx of 3.1 on onnx runtime and on ggml I could get an RTFx of about 2.2.

>>108595135
Very interesting, Chatterbox GGUF is unusable on my system but ONNX is almost real-time. Can you set LLM sampler parameters with onnxruntime?

>>108595177
See, that's easier to understand and legit interesting to know.

>>108593463
What's with the Gemma 4 psyops? Anything from Google couldn't be that good.

>>108595178
With ggml, I don't think so. Well you can but you'd have to write custom tensor code yourself, which can be useful if you're dealing with convolutional architectures. Anyways, if you build with llama.cpp to run ggufs you can, obviously. I did that at one point to get KV cache quantization working to minimize VRAM usage when I was doing GPU inferencing.

>>108595171
Minimax 2.7
Autocorrect fucked up my shit

is it worth trying to at-home abliterate, or is that going to be too difficult to do as a personal project?

>>108595178
Oh shit I totally read your post wrong. With onnx runtime you largely have to implement sampling parameters yourself but it's really trivial/easy to implement. Brain fart.

>>108595160
wait, really?
Are you saying if I had some prefix conversation, prompt it with A1 to see what probs come up in the reply, then roll back to the prefix and run some slight variation A2 to see what happens. that A2 will be influenced by having run A1 first if caching is on and i don't re-process everything from scratch?

I think part of the issue is that ggml doesn't utilize the L2 cache as well or do register packing as well as onnxruntime. Also SIMD and AVX support for conv architectures isn't as good, which kind of makes sense since llama.cpp doesn't support conv architectures at all by design. Very annoying.

>Exllamav3 uses twice the vram for gemma4 context than llamacpp and is slower
Whats the fuckin point of it then, turboderp?

>hdd with day 0 gemma weights started making a clicking sound periodically and lags for ~5secs whenever I create a new file
am I fucked?

>>108595231
Sure as fuck didn't use to be slower. At least not for other models.

>>108595252
>still using a hdd
You're beyond fucked, fren.

>>108595252
>hdd
kek

>>108595254
That's the source of my exasperation. I could live with it being vram hungry if it were at least faster, but it's not.
In fact it's worse than that, because I was using a 6bpw exl3 (The largest he published) while I've been using a q8 gguf.
Really glad I didn't sit through making my own quant just to discover this.

>>108595252
It has a -1 day dead man's switch. Handle with care.

>>108595281
I have the day 0 weights with the backdoor removed, will share a link after the bake.

>not migrating your day 0 gemma to NAT
>willingly subjecting yourself to coil whine
this is /g/?

>>108595252
Anon, you do NOT put model weights on a HDD unless you have enough swap on a SSD to load the whole thing into the page cache and keep it there.

>>108594651
but can the engram be updated in real time by the model

File: Screenshot_20260413_152028.png (2.87 MB, 2596x1273)

2.87 MB PNG

>>108594700
Not so good news unfortunately anon.
Unfortunately the positioning of the boxes is messed up and it misses stuff to translate.
But it KINDA can do the job. This was q4_xl since you requested q4.
I would instead go 26b even if its on cpu only, and no reasoning. Since its moe its fast enough if you havev a bit patience.

All that being said...I do think its seriously impressive that a 4b model can coherently translate and position at this level to be honest.

Are macs the only decent low power option for an AI server?

>spend the last week tinkering with llamacpp and koboldcpp as backends for sillytavern to use gemma 4 31b and its reasoning
>literally never works as intended

What the actual fuck is going on with this model. Might be a skill issue, but reasoning has never worked properly. It either never reasons, or it reasons but refuses to actually answer after reasoning, or it reasons and answers but never reasons in subsequent answers, or it reasons and answers but the reasoning is included in the think block (so it likely answers as part of its reasoning)

Also, text completion bizarrely never works with reasoning, and chat completion is severely gimped (can't use a system prompt at all or it shits the bed and refuses to reason)

Regardless of whether I use updated quants or not, or whether I use the latest llamacpp/koboldcpp builds, or whether I use their recommended settings or presets from people who claim to be enjoying reasoning, it has literally never worked as advertised.

I'm convinced at this point that gemma-4 reasoning is an inside joke or something.

Please help, or tell me how you managed to use reasoning with the 31b model.

>inb4 skill issue

It absolutely is a skill issue, I need help with it.

>>108595355
i bought some asus ascent gx10 boxes and they're arm low power devices
dgx spark should be similar