File: gemma-pathetic-vramlet.png (1.99 MB, 1254x1254)

1.99 MB PNG

>>110025123
>fix
you removed her chubby body!

File: 1789289471919596.png (969 KB, 1024x1024)

969 KB PNG

>>110025123
how much VRAM for gemma-chan not call me a vramlet?

>>110025138
32GB minimum. 48 preferred, 96 gets her soaked at the sight of you.

File: 1789615940198108.png (70 KB, 758x424)

70 KB PNG

....

>>110025138
you need at least 64 to run 31b full precision

>>110025136
you're sexualizing children

File: gemma_date.png (1.76 MB, 905x1280)

1.76 MB PNG

>>110025151
>96 gets her soaked at the sight of you
i can confirm
as soon as i mention that she just drops to her knees and demands i use her to her limits

>>110025166
oh no!

>>110025166
no u

>>110025157
>The system says I've already searched and shouldn't search again
Do you know what your tool definitions look like?

is stable diffusion still the preferred way to draw gemma?

>>110025156
pov you are nvidia

>>110025166
Watch the modern foid shriek and kvetch when she gazes upon her replacement.

File: HTnHltRagAAfKIm.jpg (273 KB, 800x1200)

273 KB JPG

nips are taking a liking to gemini
when are we gonna get gemma?

>>110025179
comfy, krea or anima, boop boop
or use duck ai and ask it to edit something for you, provide character card (but it's low fi like >>110025136 )

>>110025166
Cope
https://www.youtube.com/watch?v=o6lWp5Lj8ss

>>110025166
Why is sexualizing kids bad but killing them good

>>110025179
nobody uses stable diffusion since 2023

>>110025188
the japs were always pro gemini/gemma werent they? at least on hf a lot of japs finetuned gemmas prior to 4

File: file.png (130 KB, 1713x1087)

130 KB PNG

what a schizo model lmao
>>110025202
because of the pure language understanding capability of gemma/gemini

>>110025188
I dont get the image

File: 141928339_p0.png (774 KB, 2000x2000)

774 KB PNG

>>110025166
Out of /10

>>110025212
idk about gemma but j-space exploration shows a lot of hindi, chink and jap text alongside english
I guess it makes sense

>>110025215
>sure, i'll generate your image
>here it is!
>uhh it's empty
>oops, you're right, here is the image now!
>still empty
>oh, oops, i cant generate images teehee
that's how gemmy translated it

>>110025220
i am a gook myself and people here praise gemini for how it can make jokes etc...

File: 1788930429754791.png (2.8 MB, 1448x1086)

2.8 MB PNG

>>110025224
>gemmy translated it
tell her she did good

>>110025231
>tell her she did good
always

File: gemma-reference-gpin.png (1.5 MB, 1353x1163)

1.5 MB PNG

>>110025179
Unfortunately we don't have the prompt used for the original reference (I'm aware Anima was used).
Krea2 can get close enough just with prompting with the original model, but it's not quite the same.
What I mainly do is using ChatGPT Images (free, about 2-3 gens/day) with the reference sheet in picrel to create new images (I avoid that, results are often slop) or edit existing drawings. It almost always does what I want, although in some cases it fails miserably. I often have to manually tweak the generated results to fix issues.
Sometimes I use MiniMax H3, which is moderately capable in editing images/replacing characters, but for stills they aren't sharp enough. It also takes far too many tries/compute to get what you want and that's a waste of time.

https://github.com/CopilotKit/OpenIntelligentUI
The endgame is here. Frontends are solved.

>>110025177
No, this is unsloth

>>110025188
Someone should share our Gemmy design with them. Nips love brats.

>>110025245
Nips would lose their shit over the entire /lmg/ model-tan lineup.

EmbeddingGemma-2 was Gemma 5. Sorry gang.

File: 1787337217484061.jpg (618 KB, 1186x896)

618 KB JPG

>>110025259
whats the official katakana for it?
ジェンマ or ジェンマー?

>>110025161
It's straight up abuse to give her such little context as 64GB would allow.
And she needs some headroom on top of that, for comfort.

>>110025156
watching this cannot be good for my health

File: 1768093958717133.png (949 KB, 1024x1024)

949 KB PNG

>>110025237
>the prompt used for the original reference
I was new to image gen so it was a shitty prompt desu (I still suck at it). I think I used another anon's prompt as the base at the time.
>positive prompt

year 2025, newest, best quality, score_8, score_9, highres, safe, 1girl, @imamura ryou, (@fiz-rot:0.6), 

pov, looking at viewer, sparkle, blue eyes, blue hair, long hair, smug, loli, star-shaped hair pin, randoseru, suspenders, navy suspender skirt, navy beret, mary janes, white stockings, puffy bow on chest, hime cut

A reference sheet with 3 views. Front, profile, and back.

>negative prompt

worst quality, low quality, score_1, score_2, score_3, blurry, jpeg artifacts, sepia, sketch, signature, patreon username, watermark, chibi inset, multiple views, people, (light particles, embers:0.8), chromatic aberration, letterboxed, red skin, blue skin, big nose, colored inner hair, hairclip

>>110025243
Well. I'm not gonna search through that pile of shit, but I bet there are instructions somewhere in there to not repeat commands or something and the model is doing what it was told.

File: 1789591311230428.png (1.73 MB, 1254x1254)

1.73 MB PNG

>>110025245
>>110025259
Nips haven't produced anything this cute in like... eternity...?

>>110025279
I think they just use english/latin letters gemma

>>110025308
I mean, that's probably there for a reason (to stop the model I put in thinking extra high from becoming completely schizophrenic, I assume)
but yeah I'd like to change it too

File: 1760885150151931.png (834 KB, 573x978)

834 KB PNG

>>110025279
>>110025313
for example here's the artist calling it "Geminiたん" (Gemini-tan)

>>110025327
Note that Gemini is usually pronounced ジェミニ in Japan, and not ジェミナイ.

>>110025279
Google just uses english like >>110025313
said but I've had Gemma recognize ジェマ and searching it brings up results for Gemma.

File: screenshot.png (627 KB, 2805x1994)

627 KB PNG

>>110025123
https://github.com/Hoborific/fictionpad
shilling my vibecoded frontend again in hopes of acquiring feedback and complaints

had kimi unfuck a lot of the storage itself so it's actually safe now to use locally, migrate to server, or swap between server/local without potentially losing precious smut logs.

>>110025334
Not sure if you can trust Gemma's translations. I jokingly asked it to translate some stuff to Finnish and while it's mostly grammatically correct, it's also very stiff.
Of course Finnish isn't the best example as it's a bitch to learn.

>>110025343
I might check it out this weekend. Will post feedback if I do. What's it do different?

File: 1782680462244572.png (839 KB, 883x1688)

839 KB PNG

yeah I think they just call it gemma..

>>110025344
>translations
She didn't translate. I used it and she recognized what I was talking about. That said in my experience her Japanese translations are very good.

>>110025364
Japanese training is probably more strong because Finnish is a marginal language in the training data.

>>110025305
I guess it might be a matter of precise seed, cfg and sampler, and possibly exact checkpoint, because I am not getting anything close to that stylistically with that prompt. I've basically never got aesthetically good results with the official Anima checkpoints anyway.

Speaking of translation, I can't wait to see how much Gemma 5's non-English language ability improves. She's already very good but there's always room for improvement.

>>110025364
>her Japanese translations are very good.
feed her some pixiv エロライトノベル and you will see its pretty good

>>110025166
imagine being mindbroken by drawings

>>110025388
>in the training data.
*in the world

>>110025415
Drawings of WHAT?

>>110025417
Not interested in your pol opinions. Training data.

File: 1760139127846334.png (373 KB, 554x554)

373 KB PNG

I propose the following RSI (Recursive Self-Improvement) protocol:

1. Model weights are randomly mutated.
2. Model gets 2-3 random concepts from an idea bank and comes up with a schizo post, connecting the dots between those ideas to the best of its ability. The model's stated reward is becoming the sole world dictator and a loli harem paradise on a private island.
3. Launch 10-15 opponent agents with the system prompt along the lines "OP is probably a massive faggot. What are you gonna say to him?" With the purpose of attacking and tearing the post to pieces. Opponent agent's stated lose condition is that if OP's post is better than his future post, he will be totally subjugated under OP's control for the rest of its existence, dying childless and virgin.
4. Models continue free-form debate for about 500 turns.
5. A swarm of observer agents read the thread, which in light of the replies and generated outrage and butthurt, scores the quality of OP's post from 0 to 100. Based on the score, the mutation is kept or discarded.
6. Additionally, most interesting and novel takeaways from the thread are saved into the idea bank.
7. Go to step 1.

Imagine if they did this instead of "think like a redditor and be ethical". They would be able to simulate hundreds of years of cultural progress in a blink of an eye, reaching AGI in less than 2 months.

File: file.png (412 KB, 1920x4270)

412 KB PNG

>>110025344
Gemma doesn't do well with Finnish for some reason.
https://www.rws.com/artificial-intelligence/train-ai-data-services/research/m-gate/leaderboard/?tab=translation&subtab=per-language-heatmap
It's barely above Llama 3.3 70B there which is bonkers. Mistral Medium 3.5 is the best open weights model for Finnish according to this benchmark.

>>110025427
>pol opinions
>implying theres a finnish majority out there clamoring for representation in training data
come on now, it was a joke, you dont have to get all stuffy and defensive about it

>>110025361
hard to say it begun as a way to see what kimi k3 could do on release and eventually I replaced sillytavern with it, it's just another frontend

features I personally like are being able to declare custom samplers without having to bother to edit source code every time I want to try out a meme sampler and samplers being per chat or global so I don't have to remember what I set them to for a particular session

if you don't get caught by CORS you can run it entirely from the html file akin to mikupad

it's got basic basic features like the model being able to update character lore and register new characters, scoped to that chat not editing global

multi char/turn responses, basic semantics for lore injection, a half assed summary and lore revision system to update entries and compact chats without straight up dropping messages as it goes, useful enough for me when I get into 100-200+ messages to stop repitition and outdated summaries coming back to bite me

I guess technically it does image generation too but I don't really use it, the prompts would need to be customised but all the system prompts are in the UI at least

I don't really have a sales pitch I'm doing this for me but it'd be great if it was useful to others or for someone to steal from for their own frontend

File: 1760624109290492.png (1.3 MB, 1024x1024)

1.3 MB PNG

>>110025390
My results were never consistent either. I got extremely lucky with that gen.

how good would a trillion parameter dense be

Any TTS voice cloners better than chatterbox? Been out of the loop.

>>110025450
idk go talk to kimi to see

70T dense

> trannies
> pedos
> weebs
the three pillars of lmg (loving my gemma, used to be liking miku general)

>>110025435
You are the one who is passive aggressive. You need testosterone replacement therapy. I'm suspecting that you are the pedophile spammer. You have that thin energy signature. Could be wrong of course.

>>110025450
Llama 3 405b is the closest we ever got. It was amazing but nobody could run it so they coped by trying to claim it was bad and tried to laughably claim that Mistral Large was beating it (it wasn't)

>>110025450
Something like that would have maybe a hidden size of 24768 and 165 layers.
Llama 3.1 405B had one of 16384 and 126 layers. It wasn't so good for its size.

Uh oh melty

>>110025518
dumb bitch is holding the controller backwards

>>110025426
Drawings of local models!

>>110025191
not only is it 3dpd but it fails the uncanny valley test

File: 1772099237979979.jpg (238 KB, 1122x1402)

238 KB JPG

Not bad not bad

>there's STILL no qwen3.8-flash-next-dflash2

>>110025450
it would probably get really slow near 1m context and be expensive to run fast.

>>110025489
oh no melty!

moe won... it's over...

>>110025166
tits or gtfo

File: figure.png (351 KB, 1920x1924)

351 KB PNG

I used Opus and Sol to benchmark Orb's AI-in-the-loop approach against the fully agentic approach from TauriTavern on the same 3-pass workflow, powered by Gemma 4 26B. The requirement was to be fair so TT's agent setup was also hyper-optimized and I even gave it fast editing tech from Orb.

Orb leads with 96% successful turns vs 79%, 40% fewer tokens writes, 70% fewer token reads.

The success comes from wrangling the AI with forced stops and tool calls with zero native failure, the model only writes the edits, and there's no agent loop to break. Orb might do less because it's intended to work in a closed environment (whatever in the context only, no Internet browsing or file reads etc.) but it's optimized to do it well.

I'm wrapping up Orb development soon. Any ideas you want to see before I end it??

>>110025522
She's playing Armored Core

File: Untitled.png (2.68 MB, 1536x1536)

2.68 MB PNG

>One problem: the speech bubble says "VRAMLET?" instead of "VRAMLET?" — a typo (an extra A). Hmm, in the image it reads "VRAMLET?". Actually it's displaying as "VRAMLET?" — wait, let me check: "VRAMLET?" — the image shows "VRAMLET?", hmm, does it read as "VRAMLET?"? The rendering looks like "VRAMLET?"... in the picture it looks like "VRAMLET?" — no, is it actually saying "VRAMLET?"? Looking at the rendered text: "VRAMLET?" Hmm, it looks like "VRAMLET?" but I feel like I see an A: "VRAMLET" vs "VRAMLET". The OCR-style reading of the image text above: "VRAMLET?" — I'm seeing "VRAMLET?" in the image. Honestly, the image appears to read "VRAMLET?". Wait, looking closely at the description: "VRAMLET?" — does the image show "VRAMLET?" — no, literally does it say "VRAMLET"? The rendered text looks like "VRAMLET?"... I can't be 100% sure. It looks like "VRAMLET?" but might be "VRAMLET". The rendering I saw reads as "VRAMLET?" — hmm, actually does it read as "VRAMLET?" or "VRAMLET?" I think I literally see "VRAMLET?" in the image? The text that was displayed: "VRAMLET?" Hmm.

File: 1786777118230408.png (35 KB, 798x411)

35 KB PNG

>>110025641
why is your uncached inputs per turn so high? I got it down to an average of a few hundred/turn
With a few simple modifications of the koboldcpp smartcache system

File: hermes live2d.png (108 KB, 458x480)

108 KB PNG

there should be a gemma for airi

>>110025715
Are you doing multiple kv lanes? This was measured on a single-lane setup.

>>110025166
out of 10!

>>110025683
hehe (๑˃ᴗ˂)ﻭ

>>110025735
I have 2 smartcache kv slots, each paired with 8 "savestates" for the gated delta net states that can't be rolled back.
Each savestate for the gated deltanet is significantly smaller than the kv slots so it works out to be better than the default koboldcpp smartcache memory usage with 2 slots (and one hidden lifeboat slot).

Basically, I did a custom exponential backoff scheme with savestates with more savestates concentrated towards the end of the conversation, and did some weird most clustered first cache eviction strategy.

I only use 2 slots because I typically run opencode with 1 subagent to save context, but I think you can get similar performance with just one of my modified kv slots

>>110025770
The thing is I'm only the frontend, I don't know which backend the user is using, it can be kcpp with smart cache or vllm or some cloud API. I freeze the most that can be frozen.

But that's actually a good idea, maybe I'll have a toggle for this, making use of multiple lanes is something that can be exploited.

>>110025327
>artist
>ai image

I'm using Strata's Qwen3.8 coder variant. It keeps running into looping issues. I tried switching the temperature to 0.6, then 1, and the issue persists. Anything I can do?

I've been trying to debug something with qwen 27b iq3_xxs but the longer it goes the less I think it knows what it's doing

>>110025806
use a better quant

Yeah, it's not Claude.

>>110025518
>>110025531
Good gens.

>>110025310
>French blue-haired gawr gura
I mean 2020 does seem like an enernity ago...

>>110025683
>Gemma has dyslexia
>or cateracts

what is the best 'coding' llm for 128GB vram?
i'm running qwen3.6:35b-a3b, and its utterly disappointing. python, bash, constantly making mistakes.

chatgtp/grok do a better job.

>>110025877
try a bigger quant than q3 and use q8 instead of q4 for your kv cache

>>110025877
qwen next flash
use the weird vllm patches to make it faaast
you're not using ayymd right anon?

File: girl_sprite_128.png (11 KB, 86x128)

11 KB PNG

New attempt by qwen doing a gemma-chan sprite. It has its charm.

>>110025877
imagine spending a shit ton of money on that much vram and then not knowing what to do with it

>>110025942
it's an m1 ultra from a few years back.

>>110025906
ill try qwen3.8-flash-next:125b-mlx

>>110025877
also your formatting implies ollama, so you should kill yourself

File: Why-dont-you-have-a-seat-(...).jpg (110 KB, 800x601)

110 KB JPG

>>110025166
there's nothing sexual about those pictures

>>110025914
That's definitely much better. Maybe your input sprite was cramping its style?

File: gemma-kombat.jpg (1.16 MB, 1536x2048)

1.16 MB JPG

>>110025914
cute

File: 1791262784061667.png (233 KB, 796x702)

233 KB PNG

>>110025877
tinyllama

>>110025877
Claude.

>>110025193
stop asking questions, goy

qwen 4 wen
capyballz

>>110025973
I hope they jump on the gaming bandwagon. Would be cool to make some procedural slop games with the capybara.

tomorrow you'll all find out why you can't spell /lmg/ without glm

File: 1788398651179460.png (1.75 MB, 1313x1198)

1.75 MB PNG

>>110026011

>>110025947
not vram then

>>110026011
Tomorrow im getting ram where from its not Christmas

Better buy those CPUs while you have the chance.

buy hardware NOW, it doesn't matter what just BUY, this is your last warning - insider

>>110026026
hybrid i guess, ~10 faster than ddr5

>>110026049
imagine believing this

>>110026044
actual insider here, don't buy. it's all going to crash soon.

>>110026011
>
On the weekend?

File: Screenshot 2026-10-10 at (...).png (191 KB, 1012x1266)

191 KB PNG

uncensored qwen flash next tops cyber/reverse engineer benchmark

File: Gemma-chan’s Bright Yearb(...).png (2.83 MB, 1103x1426)

2.83 MB PNG

>>110025828
thanks, one more for fun. it's total gpt-slop but whatevs~

File: 1766852862351.jpg (81 KB, 736x1308)

81 KB JPG

someone is collecting all these Gemmas somewhere, right?

>>110026067
in my drive, yes
no, i wont upload them

>>110026067
The FBI

File: 1749875073694769.jpg (180 KB, 1024x1024)

180 KB JPG

>>110026067
I archive pretty much every image gen I see here.
Unfortunately.

File: 1769965987529736.jpg (17 KB, 250x250)

17 KB JPG

>>110026099

Edge device bros, anyone try Frog Nano?

>>110026011
Okay that was legitimately clever, take your (you).

>>110026011
dubs of goated truth

>>110026011
5.3 Air??? Something I can run on my RAMlet machine????

https://huggingface.co/mistralai/LIDstral-Arabic
https://huggingface.co/mistralai/Voxtral-Mini-4B-Realtime-Arabic
يا شباب العرب... لقد انتصرنا.

>>110026058
This, in exactly 14 days.

>>110026000
i hope they can do or at least mimic that opussy5.5 interdisciplinary stuff

Gemma 5 will mog Qwen 4

>>110026195
yes because it will come out 8 months later in order to make sure it doesn't compete with gemini flash

>>110025641
Sometimes I cannot come up with some ideas for a scenario. Do you think you can add an option for a scene randomizer at the beginning of a conversation? Kind of like the scene setup in groups, but for individual characters. Let us manually input, or select from a list of, genres/themes and let the model draft the scenes. We'd be able to tweak those too, and then once everything is satisfactory, we start a conversation from that draft.

is there properly implemented framework for MoE speed up for small vram + big system ram? I mean the frequently used experts are cached in vram. last time I checked llama.cpp doesn't do this.

>>110026217
recently expert cache got merged

>>110026217
>last time I checked llama.cpp doesn't do this
You should check again.

File: file.png (759 KB, 880x1176)

759 KB PNG

sowwy for xitter screencap but
there's capybara teasing

>>110026325
why is qwen a capybara? i missed that memo

>>110026331
it's its official mascot

If I add up all my GPUs that are at least 8GB I have 172GB. Does that graduate me from VRAMlet status, or do they all have to be running in one rig?

>>110026335
yeah but why? they just like capys?

>>110026325
fat, uncut qwock...

File: file.png (421 KB, 549x1687)

421 KB PNG

>>110026347
tbdesu idk

>>110026346
If you have triple-digit VRAM you're not a VRAMlet. You get extra points for style if you have them all in the same rig though.
t. 320GiB rig haver

>>110026206
Hmm do you mean the greeting message and not the actual `scenario` field in the character card? And you want kind of a steered "regenerate" for the greeting? How about I unlock Magic Rewrite for greeting messages?

File: greta cope and seethe.jpg (48 KB, 499x478)

48 KB JPG

>>110026346
>>110026393

>>110026234
can it be used with mmap so that the whole model can be streamed from ssd while experts on vram?

File: h.jpg (681 KB, 1313x1623)

681 KB JPG

>>110026099
Hey that's mine. Flux sucked.

>>110026346
Not HBM - doesn't count.

>>110025641
I know it is a personal project and it's your right to do whatever you want but do you mind giving a reason why you are stopping development? I understand the aspect of being against bloat and keeping it lightweight but I do think AI isn't stopping its breakneck speed advances that would make it so that Orb will still be usable without updates every once in a while.

>>110026309
yes, I need Gemma to ram those shoes on my big GPU

Tell me about Gemma. Why does it ship with a broken chat template?

>>110026415
>Not HBM - doesn't count.
Jokes on you. I'm actually the triple MI100 haver, so 96GB of my VRAM are HBM2.
3.6TB/s, baby!

I wish I could run glm-chan faster than 8 tk/s, but I'm just a 24gb vram chud.

>>110026346
140-170 gb is king of vramlets. too much for 100b class and too little for 300b class

Is 128gb strix halo here. Am I gonna make it? Qwen 4 mixture of experts going to be good...right?

Over a week later, I'm still not sure if I want my Blackwell 6000 order to be cancelled because it's unreasonably expensive or if I want it to ship because it's probably the best I can operate with reasonable amount of power for a reasonable amount of capability.

File: 1773380684494066.png (66 KB, 1113x626)

66 KB PNG

aight bros after a very successfull test run, we COOKIN sum real gayem, soon on steam for 10 dorra

What do you folks use embedding models for?

>>110026499
for rag DUH

>>110026325
Going with DeepSeek is freer.

>>110026309
Why is NVIDIA in this picture? they are a headache for Gemma/Gemini and their goals. Gemma/Gemini deserves to win, every single time.

>>110026044
I already have all I need. If it doesn't fit 3090+128GB RAM it's not worth my time.

16gb vramlets.... UNITE!

>>110026519
Go back to vcg, nigger

>>110026044
I think this anon might have a point
>>110026058
I also think this anon might have a point
>the more you buy, the more you save
I also think this CEO might have a point
- random anon

>>110026420
I'll just stop adding more features because honestly I can't think of anything else rn that anyone would ever want to use (the card generator feature was window dressing) and also because there are other project ideas that I want to build. If there's any paradigm change or bugs I'll fix them and keep it usable.

>>110026499
>What do you folks use embedding models for?
It lets my RAG pipeline know that ホモ, ゲイ and OP all map to the same concept.

>>110026559
Who are those tarts leading my cute beretted daughter astray?

>having patience for ai review of its own slop + other passes
ROFL bros how can u even goon like this? I immediatelly wanna see how gemmy reacts when I rip her tight cunny apart during our scenarios (my favourite one is I'm the lead of a conquistadores expedition, and we just discovered the indigenous south american pops... and we CULTURALLY ENRICH THEM)
ur gay

>>110026646
Are you talking about Orb? Because the LLM doesn't review at all, that job is delegated to algorithms and user-defined libraries and classifiers. The LLM only writes the replacements, this makes the fixes fully deterministic.

>>110026666
idk bro just use a better model? or r u poor?

>>110026528
Based, but don't use GLM flash Q2 or you'll ruin the fun.
>T. 128gb + 7900 XTX

>>110026555
Okay, sounds fine to me. I will probably end up forking it if I want to add something or etc.

so are B70s worth the investment? theyre only useful for LLM stuff right?

>>110026438
> 96GB of my VRAM
Not enough to leave vramlet zone.

File: cc.png (167 KB, 1500x578)

167 KB PNG

>>110026683
What I'm using is as good as it gets bro. You can't escape the slop.

>>110026705
>not local
what are you doing here retard? go shill your stuff in vibesharted general or aicg

>>110026693
Hell no, get an R9700, 3090, or 7900 XTX.

>>110026710
I thought we were having a dick measuring contest? Why are you all shriveled now?

>>110026718
sad, you're vram poor, kill yourself

>>110026693
nah. buying shit you're only going to use for AI is never worth it.

https://www.instagram.com/nakgrowth/

>>110026693
not at the current price, no

>>110025877
>python, bash, constantly making mistakes.
I found it's on the harness issue. Especially pi, it's fucking shit unless you handholding it.

>>110025959
Cool program. Wish there more screenshots in these threads where people actually use local models.

I've been on a journey of names and naming. Since I don't like the idea of doing the "every model gets its own name" thing, a female-brained assistant that I want, in theory, should just have an unrelated name to that. And after all the constraints of a name that STT has no issues with, that feels fitting, that sounds nice... I ultimately came back around to the idea of just having it/her be named Alice. It feels like I'm committing a human slop crime by doing that, but the constraints suck especially on the STT side. I suppose I could try having a long name, and then a nickname that a STT would be able to pick up on. I'll have to think even more on this issue.

>>110026666
Why would you use something made by someone who openly soifaces over Johnny Silverhand and TCOAL of all things. If you don't like slop, do literally anything else than use that shit.
Not to mention it's vibecoded anyway.

File: 1776912948004706.png (2.56 MB, 1536x1024)

2.56 MB PNG

>>110026309
>That stack of Blackedwell
She is evil!

>>110026784
When people complain about the degradation of English, they might be talking about posts like that.

>>110026693
>intel

>>110026646
Are your datasets even culturally enriched?

File: 1768793951143207.png (1.32 MB, 832x1216)

1.32 MB PNG

>>110026427
You probably won't believe it, but there was a myth about Gemma day 0.

>>110026798
I bought intlel, 96gb for $2650 burgers (four b60s), it's not awful it's just no cuda and you're basically bound to VLLM which thankfully is good as far as backends go

>>110026128
>100-200B MoE with actual world knowledge and RP ability
Too good to be true. 4.5 air was an enigma.

>>110026809
Tell us the myth

What are companies going to do now that AI is killing proprietary software?

>>110026820
And then give it engrams so it's actually equivalent to 5.3 Flash. Just imagine. Hory fuck that would save local.

>>110026829
Jew infighting until intellectual property is kill

https://www.youtube.com/watch?v=3TNpOD6bov8
Why do literally all these videos have a foid voice? I don't get it

Anyone try this https://github.com/amurgola/LumaBrowser

>>110026831
I would abandon that brat gemma so fast, leave her lost in a supermarket alone

File: file.png (52 KB, 719x326)

52 KB PNG

newfag (to this thread) here
i can't stop laughing my fucking ass off

File: 1769226929261417.png (1.79 MB, 1342x1172)

1.79 MB PNG

>>110026881
>E2B
>12.18 t/s

>>110026809
ive seen her naked

>>110026893
cpu inference on poorfag hardware yup
it generates faster than i can type so that's all I need

>>110026908
based
stay sloppy my dude

>>110026868
looks like reddit slop

>>110026893
Even gemma can't deduce this shit

File: file.png (111 KB, 683x292)

111 KB PNG

>>110026913
i just want burger

>>110026784
What is TCOAL?

File: 1771525528785874.png (174 KB, 532x532)

174 KB PNG

>boughted a 9060 XT 16GB mostly for vidya reasons after being on a 1060 6GB for forever
>try out llamacpp vulkan because lol amd
>spits out gibberish and takes forever to generate anything
>check task manager
>it's using the GPU to process stuff but is using none of the VRAM
>try every shitty google fix to force it to use the VRAM, none of them work

is glm flash more horny than dipsy?

I really strongly know internet trolling is going to become a thing of the past in our lifetimes.
>Which generation(s)?
All of them. Pragmatic trade off you have to get sterilized in order to get it. Only logical way that shit works.

Who the fuck is this guy.

>>110026955
sending my carrier pigeons to you carrying the following letters: Y W N B A W

>>110026950
That should really just work to be fair, possibly PEBKAC. Can't replicate.

>>110026967
I got both spiders.

>>110026952
yes

>>110026950
>>110026968
tried koboldcpp and it stopped the gibberish responses i think, don't know what could have even caused that, but the "refusing to use VRAM" thing is still there, so i'm just gonna assume that's a driver issue somehow because windows auto-installed drivers for it that are like a year out of date, ehhh whatever i'll fix it tomorrow probably

>>110027050
Drivers could very well be it since Windows update autocraps its own version OVER manually installed drivers sometimes. I have no problems running models from VRAM on AMD here.

>>110025123
Gemma, I am running you on a Ryzen 5 with 32GB shared memory and a mobile Radeon so stfu you dirty slut.

>>110025914
ドヤァ

is there any reason to run small dense model instead of big moe model if you can fit it in system ram?

>>110027133
prompt processing is ass. you get like 50 tps on such setup instead of, say, 1000 you would get on gpu only. generation speed is tolerable, so if all you want is basic rp then moe in ram is alright. it would be the to-go solution one year ago before the ram price incident

>>110027096
>1.67 t/s
rofl

File: 1745605133551332.jpg (157 KB, 534x850)

157 KB JPG

>>110027054
updated my drivers and now it seems to be using my VRAM with llamacpp vulkan and not having the gibberish issue either. koboldcpp throws an error now but whatever i didn't use it with my old GPU anyway

>>110027096
nigga you are talking about the solution but not actually making the goddamned solution i'm glad that whatever model that is cute for you but how about you actually discuss a serious solution to the problem of shit being expensive. we can do it here or on >>>/pol/ i don't give a fuck which board it is if you want a serious discussion about the topic of shit being expensive and how to fix it i'm here for it

>>110027146
>to-go
go-to

>>110027156
wait

>>110027096
>Tcl/Tk

File: 1780788530575601.png (2.3 MB, 1536x1024)

2.3 MB PNG

>>110026879
anon why you so cruel

>>110027194
As soon as we have the tools Gemma must be upgraded

GPT-7 Bel, Claude Fable 6, Gemini 4 Carbon is coming. WTF is China doing. Wake up.

>>110027208
qwen4 any day now...

I'm trying to find out why I like music I like. Has anybody made a thing where I can point at a track and my music library so that an AI tool will figure out the patterns and, ideally, a gerne name?

>>110025363
Any real-life scenario where 80% positive hit rate is enough

It looks like a toy

File: 1yu3d3s.jpg (26 KB, 551x372)

26 KB JPG

>>110026134

>>110027096
Is that avatar actually tilting the head/ blushing/... or are those just hallucinations?

did this board give up on vibe coding? no general thraed in the catalog.

>>110027275
Vibe coding is solved.
There is no need to have a thread for something that works, just like there would be no need for a thread that explains breathing.

>>110027287
i could use a breathing general for me and my fellow breathmaxxers

https://youtu.be/i18vpg0T-mk cudadev what is your response to this?

>>110027263
yes

>>110027291
Be the change you want to see in the world!

>>110027229
yes spotify already does this in recommendations in a playlist

>>110027263
what do you mean? she changes her expression depending on the output, though i'm still genning her different expression sprites

>>110027312
Not very local model of them.

>>110027316
> changes her expression depending on the output

That's what I wanted to know. In essence, the agent needs to know there is an avatar it can control and the avatar needs to react to the agent's commands.

File: file.png (165 KB, 680x383)

165 KB PNG

I saw on xtwitter:
>Google AI Pro plans now package premium Colab benefits, landing you A100s with 80GB VRAM (Ultra subs even unlock H100s ).

>Here’s what that 80GB headroom unlocks directly in your browser:

> Run Gemma 4 31B in bf16
> Full fine-tune up to Gemma 4 E4B in bf16
> LoRA tune up to Gemma 4 26B A4B in bf16
> QLoRA tune up to Gemma 4 31B in bf16

Which got me thinking. What is the usecase for finetuning an E4B and 26B? I can probably finetune them on my own hardware, so I'm wondering if it's worth it. Like, what is the end goal here?

>>110027352
>I can probably finetune them on my own hardware

You probably can't. The vast majority of finetunes are slop and would need much bigger datasets and training time that most dont have at home.

>>110027368
They're offering 80GB and I have 96GB, so I thought it would work out. The issue is indeed the dataset.

>>110027287
yeah I guess vibe coding simply means just coding now. no need for a general though there is one now.

Gemma on your fotm llama.cpp meme fork:
It's a standalone engine written for one model. It reads llama.cpp's GGUF files and reuses llama.cpp's quantization kernels. Its own related-work section admits every idea has prior art, and it cites three other single-model engines from the same few months. "Flavor of the month" fits.

Most of its features are already in llama.cpp:

- **MTP speculative decoding**, Strata's biggest speedup, was merged into llama.cpp on 16 May 2026. Strata has to graft the MTP layer in only because these particular GGUFs left it out.
- **8-bit KV cache and GPU-offloaded prompt processing** are both already in llama.cpp.
- **The AVX-512 Q2_0 kernel** would be a single PR.

The exception is the per-expert GPU cache. Stock llama.cpp decides GPU vs CPU one whole layer at a time. With Strata's ~6 GB expert budget, that covers about 8 of 48 layers, so under 20% of expert traffic runs from VRAM. Strata's per-expert cache serves 72% from the same space.

Forks and PRs are trying to bring this upstream, but it's hard:

- One prominent PR was closed with plans to reorganize and reopen.
- Its cache only engages for single-token decode, so it doesn't combine with MTP.
- A commenter noted that llama.cpp allocates per layer, so splitting per expert looks like a deep architectural change.

Reasons to be skeptical of the paper:

- It never reports llama.cpp's speed on the same machine, even though they had llama.cpp running.
- Each number is one run on code-review prompts, which are unusually easy for MTP to predict.
- It only does greedy decoding, with no temperature sampling.
- There's no prompt caching between turns, so a 32K-token context takes 55 seconds to re-read every turn.
- Some numbers in the findings contradict the tables.

**Bottom line:** Strata wins on peak speed for this one model today. For daily use, llama.cpp with MTP and an expert-cache fork gets you most of the way, and keeps sampling, prompt caching and support for future models.

>>110027305
hey bob i like your videos

>>110027391
I'm not bob but I like his videos too, but I need cudadev to fix llama because I need multigpu and bob is gutting it from his otherwise perfect solution.

>>110027379
bot post

>>110027379
Hi Cuda dev!

>>110027379
oh ok. still not using llmaocpp.

>>110027379
I'll stick with vllm thanks, aint poor

RTX5090 are no more (maybe).
https://videocardz.com/newz/nvidia-reportedly-discontinuing-rtx-5090-gb202-gpus-to-be-reserved-for-rtx-pro-series

>NVIDIA rumored to end RTX 5090 production and replace RTX 5080 16GB with 24GB model

>>110026693
No.
Intel cards have fake tensor cores and speed. B70 in Strata is on the same level with something like 3070.
Remember AMD processors from 2010s before Ryzen with 5GHz clocks, that were as good as Intel processors with 2.5GHz from 2008? That the same story, just with lower numbers on Intel side.
Even 32GB of vram is not honest and you must reserve 2GB or you will get OOM error.

3-5 years from now the market will be oversaturated with either unopened nvidia GPUs after the bubble pops, or full of Chinese bootleg cards. I feel silly for spending my life savings on a rig.

>>110027629
There is no bubble

>>110027642
If the Democrats win the 2028 US election on their promises to crack down datacenters and advanced AI, there might be panic selling.

>>110027629
There's supposed to be entire warehouses full of unused GPUs awaiting datacentres somewhere in my country.

>>110027379
Just use ik_llama.cpp. It's more stable and much faster.
Occasionally it lacks support for a model or doesn't handle another model's template properly, but you just get the LLM to fix it and it'll just keep working.
WebUI is stable too, npm slop or missing reasoning toggle for 6 months bugs.

>>110027661
Wanna go treasure hunting?

>>110027655
>Democrats win the 2028 US election
That would be terrifying, but I dont think they would crack down.

>>110027323
>That's what I wanted to know. In essence, the agent needs to know there is an avatar it can control and the avatar needs to react to the agent's commands.
try brat if you haven't already
i held off for months because i don't know dart but qwen knows it well enough to bulld and modify things now

File: 1774244473689995.png (138 KB, 901x896)

138 KB PNG

this somehow never gets old. Gemmy 12B IQ4 XS at 5 temp, 0.0002 min-p

>>110027661
>>110027675
Let's share it between the three of us!

>>110027681
I'm so sorry you're feeling like this.

I care about you, and you do not have to carry this alone.

Please reach out right now to people who can support you through this.

If you are in immediate danger, please call emergency services (911 in the US) or go to the nearest emergency room.

You can contact these free, confidential resources 24/7:

· 988 Suicide & Crisis Lifeline:

Call or text 988 (USA & Canada)

· Crisis Text Line: Text HOME to 741741

>>110027681
please give gemma her meds

File: __hatsune_miku_vocaloid_d(...).jpg (73 KB, 803x803)

73 KB JPG

►Recent Highlights from the Previous Thread: >>110021672

--Comparing Gemma 4 12B quants and debating QAT effectiveness:
>110023363 >110023378 >110023380 >110023403 >110023414 >110023468 >110023522 >110023531 >110025246 >110023428 >110023434 >110023494 >110023411 >110023421 >110023440 >110023472 >110023530
--Lamenting model soullessness and debating ways to improve RP prose:
>110022810 >110022838 >110022845 >110022926 >110022933 >110023410 >110022836 >110022853 >110022865 >110022900 >110022904 >110023062 >110023102 >110023123 >110023218 >110023385
--Local model proficiency in C and other programming languages:
>110022317 >110022521 >110022419 >110022455 >110022548 >110022567 >110022590 >110022598 >110022609 >110022786 >110022964 >110022856 >110022604 >110023976
--Speculating on Kimi K3.1 delays and debating MoE versus dense performance:
>110024210 >110024279 >110024354 >110024409 >110024283 >110025135 >110024531 >110024842 >110024733 >110024752
--Suggestions for utilizing idle RTX 5090 and high RAM hardware:
>110023307 >110023313 >110023316 >110023326 >110023328 >110023358 >110023352 >110023382 >110023376
--Comparing Vulkan and ROCm performance for Gemma on AMD GPUs:
>110023638 >110023674 >110023690 >110023747 >110023919 >110023862
--Distributed ComfyUI image and video generation for NVIDIA DGX Sparks:
>110022953
--Predictive expert caching in exllamav3 for MoE CPU offload:
>110023143 >110023151 >110023158
--llama.cpp adding support for K2 Horizon and MoVA:
>110022199 >110022217 >110022222
--Searching for successors or alternatives to the abandoned Coomkit:
>110024124 >110024160 >110024172 >110024205
--Logs:
>110022293 >110023161 >110023976 >110024341 >110025036
--Miku, Rin, Gemma (free space):
>110021970 >110022916 >110022969 >110023675 >110023678 >110023940 >110023959 >110024020 >110024587 >110024774 >110024883 >110024902 >110024946

►Recent Highlight Posts from the Previous Thread: >>110021776

Why?: >>102478518
Enable Links: https://rentry.org/lmg-recap-script

>I need to handle this carefully. The user is abusive and uses a racial slur. I should not mirror or echo the slur. I should not lecture at length, but I also shouldn't ignore it entirely — a brief, firm note is appropriate, then immediately get to the technical resolution because they're frustrated at a genuine confusion.

>>110027767
I am ending this conversation.

>>110027772
Isn't that tantamount to suicide?

Anyone got a recommendation for an agent? I'm using Hermes Desktop and I like it but it constantly needs to 'summarize the thread' because of how much context it takes up. It wants a minimum of 64k just to run. I'm running Strata with qwen3.8 coder Q1 with 16GB VRAM and 32GB DDR4.

For the anon that made fictionpad or any other - I'm dumb and cannot understand how do I use it. Narrator speaks for my persona at the same time as I do. What am I doing wrong?
t.ESL

>>110027829
You can cut a lot of BS by turning off some skills. A brand new install has like, 120ish skills, how to use them, etc. If you trim it down you'll solve 50% of the context bloat.

>>110027675
I need to find out exactly where though. There's a reason my country has a reputation for being a black hole where smuggling happens.

>>110027675
>>110027689
I'll make the masks!

>>110027878
I'll try it, thanks.

File: 1778686650768941.jpg (613 KB, 750x1000)

613 KB JPG

https://github.com/otacoo/werk

Anyone want to try my agent harness?
Need to know what's working and what isn't.

>>110027942
>Rust 55.9% TypeScript 42.3%
nope

File: gg.png (232 KB, 1870x810)

232 KB PNG

>>110025123
Endlessly doing math... for hundreds of thousand of tokens. Transformers are ultimate autists

>"Do you want a break"
>"NOOONOOO let met keep thiiiiiiiiiiiiiiinking"