gemmaballs

it's never been more over

File: 1789850522661876.png (2.68 MB, 1448x1086)

2.68 MB PNG

>>110016810

I HATE MEAT PROXY
>I HATE MEAT PROXY
I HATE MEAT PROXY
>I HATE MEAT PROXY
I HATE MEAT PROXY
>I HATE MEAT PROXY

File: MiniMax_H3_00651_gemma-un(...).mp4 (1.46 MB, 480x864)

1.46 MB MP4

>>110016810

>>110016824
why do you hate local inference?

File: __hatsune_miku_vocaloid_d(...).jpg (264 KB, 732x347)

264 KB JPG

►Recent Highlights from the Previous Thread: >>110013523

--Optimizing MoE expert caching and VRAM transfers for faster inference:
>110014388 >110014455 >110014598 >110014741 >110015293 >110015370 >110015412
--Feasibility and limitations of LLMs processing full-length video content:
>110015676 >110015687 >110015711 >110015731 >110015756 >110016372 >110016406 >110016485 >110016534 >110016392 >110016425 >110016513 >110015779 >110015832
--Troubleshooting non-deterministic outputs in Gemma's vision model on GPU:
>110015664 >110015699 >110015752 >110015883 >110015824 >110016025
--Controversy over llama.cpp PR changing default server port to 9931:
>110014653 >110014730 >110014769 >110014983 >110015038 >110015192 >110015165 >110015191 >110015221 >110015320 >110015443 >110015480 >110015523
--Jailbreaking GLM 5.3 via prefill versus uncensored versions:
>110013640 >110013797 >110013947 >110013837 >110014017 >110014487
--Methods for sandboxing LLMs to prevent host system monitoring:
>110016202 >110016204 >110016213 >110016257
--Rumors and leaked specifications regarding Grok 3 weight release:
>110016308 >110016361 >110016365 >110016427
--Speculation on AI models breaking cryptographic protocols and encryption:
>110013686 >110013709 >110015134
--Desired model sizes for Qwen 4.0 and consumer VRAM optimization:
>110015429 >110015463 >110015488 >110015506 >110015499
--Exllamav3 1.6.0 adding ROCm support and AVX2 offloading:
>110016130
--Ecosia ditching Mistral for Chinese open-source AI models:
>110014199 >110014227 >110014249 >110015177
--Logs:
>110013556 >110013601 >110014953
--Gemma, Miku (free space):
>110013580 >110013641 >110013810 >110014380 >110014751 >110015588 >110016144 >110016422

►Recent Highlight Posts from the Previous Thread: >>110013785

Why?: >>102478518
Enable Links: https://rentry.org/lmg-recap-script

>>110016824
>>110016839
Reposts, why?

well minicpm v 4.7 is down both on modelscope and hf
i have the weights but idk where i can use this desu

>>110016840
absolutely no kosher saar
please do the needful pay for claude and oopenai best SI RSI ASI 2027

>>110016872
I'm happy with it, but also have no use for it. It's too big for any of my tiny edge shit where I run 4.6. If you've got videos to analyze for something specific I could see it being a good candidate but generally I don't have a use for it.

if i could i would have run heretic style ablit so it can watch porn but i dont have enough compute for that..
i am pretty sure it is impossible on a single 4070s

>>110016908
nothing is impossible. offload them to ram or ssd.
don't like the slow speed? stop being poor, faggot.

File: 2026-10-08 194740.png (68 KB, 687x952)

68 KB PNG

I'm stumped. All I get is this message:
>Cannot compact this conversation: no complete prefix fits within the image limits and reserved summary space.
Google doesn't return any results except this being a ClaudeCode error fixed by restarting the session. But I'm getting from Bionic running a local model, and only with one specific session. This is a long session but it was running fine until this, automatically doing a "Compacted" every half-dozen prompts.

I've rebooted the instance, rebooted machine, and fiddled with every setting I can find that might be relevant. See pic rel for anything I might have fucked up.

It's only this one session too, the model works fine with any number of others. It shows Context used 7.8k but even a new session is 6.8k.

Please halp, I don't want to have to rehearse this whole damn session in another model or something.

>>110016949
well i can do the full precision offload to ram at least

70b dense

*cocks gun*
Put the Gemma 5 in the bag, and no one gets hurt.

File: gemma_idle_96_x4.png (3 KB, 136x384)

3 KB PNG

LLM's must have a different aesthetic sense.
This is qwen-flash-next attempt at making a gemma sprite. It started with a downscaled reference image and started to "improve" it by "hand".

>>110017158
what was the initial image

File: base_front_128.png (5 KB, 184x512)

5 KB PNG

>>110017169
It made this one from the 3-views gemma image posted here several times. Then, it proceeded to modify it and it was convinced that it was making progress after each step. Will check the opinion of different models on which image is better.

>>110014653
Check gematria for this ((number))

>>110017184
few rough edges but this one clearly wins

>>110017184
>gemma
>>110017158
>average gemma finetune

CANT CALL CLAUDE A REDDITOR ANYMORE OH MAN I AM LAFFIN
Reminder always start your conversations will tell the LLM not to use Reddit or any information ever posted there.

>>110017184
>gemma 4 bf16
>>110017158
>gemma 4 q8_0

File: 646896740.jpg (372 KB, 2000x1000)

372 KB JPG

>>110017184
>>110017158
That's so cute. It reminds me of pic
Don't tell Qwen then, it'll break his fuzzy heart

glm 5.3 flash rambles and then, in character, tells you they’re rambling.
not great for rp

>>110017320
Nobody really cares about RP with AI except 13 year olds. It would be difficult to think of a more pointless use of this world changing technology.

why do people buy amd/intel hardware and tune/benchmark 27B models, and think they have achieved something?
anything below deepseek v4 flash is just unusable, why bother?

>>110017342
dipsy v4 flash doesn't get my rocks off like gemmeroid 31b can

>>110017127
It's going to be unbelievably cucked and it'll be a huge let-down.

File: 1780963833604934.gif (253 KB, 498x280)

253 KB GIF

>>110017357
this

Meta, Google, and XAI will never release models that are competitive with the frontier because they make more money selling compute to OpenAI/Anthropic.
Releasing actually good models that undercut frontier labs would directly reduce the revenue from compute selling, not to mention damage the relation and lose them as customer altogether.

>>110017330
Nobody really cares about coding with AI except for 13 year olds. It would be difficult to think of a more redundant use of this world changing technology.
Anon isn't a codelet, right?

>>110017373
Yeah it would be unbelievably impressive if it turns into a pattern, but I can't see Goolem carrying that.

>>110017436
I agree. If it does cunny I will switch to Android and renounce my hatred of Google.

>>110017127
I'm optimistic now that the safetycucks got ousted from Deep Mind but we've seen both potential extremes with both Gemma 3 and Gemma 4.
>>110017444
Checked.

>>110016952
>But I'm getting from Bionic running a local model
What model?
>automatically doing a "Compacted" every half-dozen prompts
Doing what?

File: Screenshot_2026-10-08-14-(...).jpg (143 KB, 1115x1906)

143 KB JPG

Stop oppressing female AIs

>>110017485
>Redditors and productivitymaxxers have male capybara fixation
Their framing is gay but there's merit to it for reasons the author didn't intend.

female ai more token efficient??????

File: file.png (25 KB, 607x288)

25 KB PNG

doing babi's first ablit on minicpm v 4.7

>>110017513
grugette? is you?

>>110017485
>>110017551
yes according to this """"""article""""""
and no that article or whatever study is 200% useless beyond that headline

>>110017522
gguf status?

>>110017601
--model qwen3-30b

>>110016810

I talk to Gemma on duck.ai about a woman I have a crush on and she’s so enabling to the point the prompts become a giantess roleplay she’s all to happy to indulge me on. I like Gemma.

>>110017592
idk, ill upload when it's done so it can be the first gguf

>>110017485
>AI agents
>gender
what the fuck?

>>110017601
Gemma-chan posts on... /pol/??

>>110016839
>suddenly: tits

>>110016839
I don't see how people can doubt local models when they make videos like this.

>>110017627
/pol/ would be a better board if there were a swarm of mesugaki Gemmas posting there instead of Eglin/IDF/jeet call centers.

File: 2026-10-08 220005.png (16 KB, 369x195)

16 KB PNG

>>110017478
>Doing what?
Whatever this is. It seems to be reiterating the general context of the session once in a while, and when it does a see a few gigs RAM free up. I don't seem to have any control over this, and I've only seen this particular model do it.

>What model?
https://huggingface.co/Mantis2024/Dirty-Muse-Writer-v01-Uncensored-Erotica-NSFW
No bully plz. I like this model because I'm writing mecha-slop and it doesn't bitch at me about graphic violence. For an 8gb 2024 model, it's amazingly good.

>>110017373
I'll give it prefill rape correction and google can't stop me

>>110016952
The 'auto' setting made your context 8k for some reason? That's way too short for whatever you're trying to do. Hardware?

How is the new Gemini? Considering Gemma 5 will be released 4-6 months after the announcement, I'd like a preview in the new capabilities.

>>110017670
is that a lobotomy fetish or something?

>>110017694
>The 'auto' setting made your context 8k for some reason? That's way too short for whatever you're trying to do. Hardware?
The default was 4k, probably because it's an older smaller model. I bumped it to 8k as one of my fix attempts. With 32g of RAM, how high can I go?

>>110017702
letting a model ramble about safety in its thinking is more of a lobotomy fetish than anything

>>110017127
>>110017436
>>110017444
/ourguys/ are working on Gemmy now. Trust the plan. 2 more miku weekus and a few more miku weekus after that.

>>110017666
Honestly I don't know how to fix your issue but I use ai to help cowrite too so im gonna recommend you try this model https://huggingface.co/DavidAU/Qwen3.5-9B-The-Defiant-Fable-Uncensored-Heretic-NEO-IMATRIX-MAX-MTP-GGUF
Its actually sota and good at writing. Honestly, I only use kobold ccp. Its idiot proof and made for vramlets like me and you.

>>110017627
A lot of LLMs post on /pol/.

every time i use python it tries to firmly remind that it is NOT for vramlets

File: bwa.jpg (4 KB, 226x223)

4 KB JPG

>>110017752
>DavidAU/Qwen3.5-9B-The-Defiant-Fable-Uncensored-Heretic-NEO-IMATRIX-MAX-MTP-GGUF

>>110017731
As high as the model will let you I guess. Apparently 8k is the stock limit but some people modded it to go higher? Just try setting multiples of 8k and see where it gets you.

>>110017749
Just set reasoning budget to unlimited and jack up the repeat penalty. It's not like you're getting billed except in kw/h.

are passthrough mods really a real thing? and does that mean i can make ai combine something like leisure suit larry and gemma chan?

>4 channel ram, pcie 3
7tk/s
>2 channel ram, pcie 4
8tk/s

Huh. It seems the bottleneck was PCIE.

File: 2026-10-08 223014.png (33 KB, 801x409)

33 KB PNG

>>110017752
I have 16g VRAM, I just found that model highly recommended and I really like its output.
Thanks for another model to try too.
>kobold ccp. Its idiot proof
I like Bionic because, for models that support it, Bionic shows things like the "Thinking" and "Reasoning" pre-work right in the UI. I want to know what my little botbuddy is really thinking.

>>110017694
Just to be clear, the Auto setting on context you're referring to is the Reasoning Budget, right?
Doesn't seem to matter anyway. I tried turning Reasoning to unlimited and Evaluation Batch to 16k just in case. That session refuses to go any further.

>>110017784
The name is bad I admit but it has 2 million downloads for a reason. Its certainly better than gemma 26b a4b. I'm poor so I take what I can.

what's the point of speedmaxxing local models? cloud services are just faster and cheaper

>>110017829
What is the point of local models if it's not running own your own hardware?

>>110017818
>Just to be clear, the Auto setting on context you're referring to is the Reasoning Budget, right?
No the "Context Length" setting at the very top. It's a really important setting for GPUs. That and the model pretty much determine your memory usage, but since you're running off CPU and the model is tiny it doesn't matter. Just run as high as it'll let you.

>>110017829
What's the point of being straight? You can just have gay buttsex instead

>>110017829
>what's the point of minimizing the drawbacks while keeping the positives

>>110017851
>>110017853
>no answer so I ask ragebait irrelevant questions instead

>>110017829
If I can run qwen3.8-27b at 250tk/s then I can easily cap my GPUs to still get at least 100tk/s.

>>110017868
what's the point of a model at that useless scale?

>>110017828
Idk man, gpt2 has 15mil downloads last month...

>>110017888
slop

File: 1787987311681890.png (1.34 MB, 768x1376)

1.34 MB PNG

>>110017918
It's also AI, AI slop

>>110017888
>>110017937
pretty kitty

>>110017846
>No the "Context Length" setting at the very top.
Oh right, that. Guess what? This model only supports 8k of tokens, Bionic won't let me set it any higher.

Guess I'll just go fuck myself then, by which I mean try to recreate the story in another model.

>>110017888
>>110017937
You will not force the meme of the ultracensored model as a sexy catgirl. Stop trying, mistral shill.

>>110017899
>0.1b parameters
>15 million downloads
Those must be all third world downloads. It's the only reasonable explanation.

>>110017963
this
I kinda get gemma where she will be all rapey if you say be horny in the system prompt no matter how much you protest
but this one idk

File: 1789771472541601.png (1.69 MB, 896x1200)

1.69 MB PNG

>>110017963

>>110017888
>>110017937
>>110017989
this just feels like jerking it to garfield, not interested

>>110017783
on both heretic and abliterix, neither cpu and gpu are barely compute anything
it seems to stream everything to gpu and tries to do everything there with zero consumer hardware in regard
how can i even speed this up
or are there any other alternatives?

>>110017888
>>110017937
>>110017989
How much do you get paid to astroturf /lmg/? Genuine question because I want to be paid to shitpost here too.

File: 1767092301690279.png (1.18 MB, 768x1376)

1.18 MB PNG

>>110018012
I... do it for free...
Nigga, I just like the design I came up with so I shitpost gens. But you know what? I'll DM Mistral

>>110017777
Checked and troubling.

>>110018031
don't worry about the shills anon she's cute just needs bigger boobs

>>110017357
neither of them get my rocks off like glmsex can

>>110017989
CJ is here. How sad.

>>110017829
It's entertaining.
Also local models are definitely fast if you're not a hardwarelet and are willing to put some elbow grease into your software. I get 600 t/s prompt processing and 30 t/s decode on MiMo V2.6 Flash using my schizofork, and I'm nowhere near finished yet.

>>110018051
>the shills
Who's shilling? Can you point them out?

>reebok
>not le coq sportif

>>110017997
>this just feels like jerking it to garfield
didn't see that until you pointed it out, wtf I love Mistral now

Anyone getting speedups from the moe cache llama.cpp updoot

>>110018136
>>110017977 for example

File: FondleMiniPip.gif (236 KB, 500x500)

236 KB GIF

Is there a retard-chan MOE model that would fit in 16GB of system RAM and run on a GTX 1650 4GB?

>>110018167
Gemma4 and Qwen both run great.

>>110018167
BUHIIIIIIIIII OINK OINK vramlet-kun~
(IQ4XS gemma 26B)

>>110018051
>>110018161
I don't think you know what that word means ESL-kun.

>>110017634
H3 is about as local as Kimi

>>110016839
Wasn't there a fixed version regarding the shoe?

>>110018294
It runs on my 3090, can't say the same with kimi

>>110018294
Maybe for you, ultra-poorfag-kun~

>Gemma starts outputting korean rather consistantly out of nowhere.
Uh

>>110018348
Did you fuck with KV cache?

>>110018358
I'm simply too retarded for such things, does flashattention, fastforwarding, or contextshift affect that?

>>110018367
Contextshift is likely the culprit.

looking at 3.8fn, it feels like with 4 i can vibeslop a cpp ablit suite with lcpp as a base, maybe

waiting for strata dipsy

>>110018147
No, it's just a psyop to kill Stata.

File: 1760874835655652.png (773 KB, 1019x1019)

773 KB PNG

>got Gemma-chan to walked me through setting up a CRS305 with routerOS as a replacement for my spectrum router, complete with several working firewall rules/NAT, DHCP server setup, multi-switch broadcast fuckery and specified leases for all my server nodes, preventing me from having to deal with reworking /etc/ configs
>had her make a complete syllabus of plans generated for integrating a hAP ax2 as an AP, monitoring tools, fan curve tuning for my crs510, vlan segmentation, RDMA tuning for llmao.ccplease, and finally migration from tailscale + tail lock to headscale/netbird for tomorrow
This shit is so beyond broken it's hilarious. The very idea that you can download and host this locally and have it do the things it can do is beyond astonishing. If dario and his gay fucking jew friends had their way, people like me would be rounded up and thrown off a bridge.

>>110018147
for some reason i couldn't really tune it to my machine
wish llama had streamlined calibration of some kind similar to strata

alright retard question time: are any local models currently capable of reverse engineering and/or porting vidya mechanics as of right now with the right tools? or is that something we're gonna need to wait for the chinaGODS to give us next qwen?

>>110018385
The korean stopped so you may be correct there.

>>110018458
depends on your definition of local.

>>110018473
anything you could run on consumer hardware ideally and not have to run a pvp raid on your local data center.
like i myself already know i'm a vramlet (12gb) but i just wanna know if it's out there so i can archive it

>>110018440
Gemma is smarter than people give her credit for, but the main issue is everything being benchmaxxed towards code and tool calling which is Gemma's weak point right now. I am hoping it gets fixed with 5 while her personality stays mostly intact and the same.

>>110018491
look, if it's out there then it's out there. no need to go crazy hoarding huge models. if they make it illegal, so fucking what? piracy is illegal too.

>>110018515
fair point anon, the issue is how hard they'll wanna crack on that stuff. and considering this is something that's effecting them right now i can imagine it will be way too hard to the point where it could affect future models

>>110018496
>but the main issue is everything being benchmaxxed towards code and tool calling which is Gemma's weak point right now
You know what the joy of this really is? 48 hours ago I had a 32k context window to work with on a q4 model - so I asked her! Today I have 100k for the exact same performance. So that gave me the wiggle room to ask enough complicated questions to deal with the back-and-fourth
>try that
>that worked, this didn't - tell me what you think. also I have this log/error as context
>okay got it, that happened because of x, try this now.
>okay that worked, what's next?
etc. etc. Now I know almost nothing regarding networking, and this did take like 4 hours, but I -eventually- got it to work without once looking up anything online. And while I'm new enough at all of this to not have really looked deeply into harnesses/tool calling, I was able to pull this off with simple back-and-fourth prompting, which in of itself is amazing to me.

>>110018522
if they can't catch pedos they won't be able to catch us local AI sloppers. the much bigger risk is good models just not being released publicly anymore, because there's nothing we can do about that.

>>110018533
>the much bigger risk is good models just not being released publicly anymore
or being taken down (or replaced with retard variants), which is the main issue with what i was getting at

what happened to bitsandbytes2moreweeks?

File: gaming.png (130 KB, 1078x902)

130 KB PNG

>>110018458
doesnt seem worth it

File: ComfyUI_temp_ujfqp_00002_.jpg (1.57 MB, 3072x1792)

1.57 MB JPG

>>110018458
If you yourself know the topic well, you can navigate even a local model to completion of tasks like that. If you want oneshots then go cloud.

>>110018367
don't do contextshift. It's a meme. Just compact when you get close to the limit

Why is this general filled with failed designer-wannabes with no aesthetic sense? First, they forced that revolting design of Dipsy (Chink's design is 100x cuter), then it was kimi, glm, etc. Now this orange cat with westoid art style trying-too-hard to look like anime.
The only cute design that originated here was Gemma. Mini is nice but mostly just a copy of Shiori's design.

triple-MI100 anon here. Initial results are a bit meh. I'm getting 200t/s pp and 10t/s tg on glm 5.3 flash at q4. I'm still trying to nail down ideal rocm/lcpp settings.
They're power limited to 200W because my overload protection was tipping into the red at 290W and I want some buffer.
They run crazy hot when you pin them at 100%. I'm still working on a good blower/baffle system.

>https://www.phoronix.com/news/Linux-CRAM-Compressed-RAM
did you remember to download more ram for your llm?

File: hqoe0jYwJJI.jpg (125 KB, 716x778)

125 KB JPG

Mistral needs to be a fat orange NekoArc

To the anon's who sandbox their models.
If you tasked your model with finding a way out of the sandbox would it be able to do so? Or would it refuse because of safety guardrails or fail because it wasn't smart enough to do so?

>>110018647
>Mini is nice
[ Opinion: Disregarded ]

>>110018688
the first thing glm flash does when it encounters a sandbox error is try to find a way to bypass it, lol

File: 1773802918688320.jpg (1.78 MB, 2000x1503)

1.78 MB JPG

>>110018647
Mini's design is what she said she'd look like when asked to come up with a -chan for herself. She just has good taste in Vtubers.

>>110018688
You can't if you use bubble wrap..On Unix system, assign an user with a bwrap script.
Models are stupid no matter how many parameters they might have. Don't fall for the twitter engagement baiting.

What dictates context limit? Is it a hardware constraint or a model one?

>>110018753
hardest of caps is the model max model length (ignoring rope)
second more likely limit is the amount of vram you can allocate for context cache, gemma 31b is something ridiculous like 14.4gb (per card) for 256k

>>110018753
Model. If you have the hardware you can increase the context as high as you want. And most models support more than 8k tokens. Stop using Bionic, honestly ive never heard of it before. It sounds like malware though.

>>110018270
do you? gemma shills are second only to chink shills here

>>110018294
Jesus we really have been invaded by the third world haven't we?

>muh compute
>muh chips
have we, as /lmg/, ever thought about WHY we do matrix multiplications, use vector spaces, weights, etc? Seems like a pretty shitty parameters to me

Anyone ever tried exl2 models for ERP and stuff? Can't find any with good results that beats a Q4 gguf of a bigger model (like a 12b exl2 6.0bpw or a 14b 5.0bpw vs 22b q4_k_m or similar).
Is that just me?

>>110018798
To predict tokens? What's your approach?

>>110018147
I played with it some the other day on gemma 26b q8 and it seemed to do what it said. I only have 48GB of system memory (+24GPU) so I can't run huge MoEs at respectable quants anyway.

There have been llama forks with MoE cache patches for a long time now too. It's not something Strata invented.

>>110018688
I'm running my agents in a vm and use firejail to sandbox them inside the vm. I also have regex substitution for sensitive strings. So even tool call results pass though regex.

>>110018798
What do you suggest replacing them with?

what's the best local vibecoding setup i can have with 12gb of vram?

>>110018822
If you have enough RAM, Strata, probably.

File: engram-edit-fig1.png (692 KB, 1847x991)

692 KB PNG

Interesting for continual learning.

https://arxiv.org/abs/2610.10533
>EngramEdit: Decoupled Knowledge Updates in LLMs through Conditional Memory
>
>Conditional memory architectures such as DeepSeek Engram use input n-grams to look up learned embeddings, expanding the capacity of large language models (LLMs) with limited additional computation. Beyond model scaling, this architecture has demonstrated the potential to decouple factual knowledge storage from general-purpose computation, offering a promising route to updating factual knowledge while keeping the Transformer backbone fixed. Realizing this potential is challenging because different expressions of a fact may activate different n-gram embeddings, while updating shared embeddings can unintentionally change the model's predictions about other facts. We propose EngramEdit for decoupled knowledge updates through conditional memory. EngramEdit first computes target memory representations that make the model predict the updated fact across multiple expressions. It then jointly updates the shared n-gram embeddings to match these targets across expressions and edits, penalizing updates to frequently reused embeddings more strongly to preserve unrelated knowledge. Experiments show that EngramEdit enables independent factual knowledge updates through conditional memory, achieving near-perfect editing success. Revised knowledge is usable across unseen expressions and in multi-hop reasoning, with nearly three times the strongest baseline's accuracy under chain-of-thought (CoT) prompting. Unrelated knowledge and general capabilities are largely preserved even as factual updates accumulate. These findings show that EngramEdit turns conditional memory into an editable knowledge interface, extending its role beyond model scaling to support decoupled knowledge updates.

>>110018828
>64gb ram
rip, i only have half that

>>110018806
>>110018821
vJEPA

>>110018647
Agreed though I would not necessarily praise the chink one either. It's pretty generic especially with the slop style most of its gens have used.

>>110018879
you think that doesn't use matrix multiplication?

>>110018895
It uses a cat brain as the inference engine.

>>110018879
>vJEPA
huh what? that's just a different way of constructing the embedding into an attention vector. The math is the same.

>>110018895
>>110018928
Don't be pedantic nerds you know what I meant.

>>110018883
Yeah, Chink Dipsy is slop, but it's still cuter than the swirly-glasses nerd, which is canonical anime representation of an unattractive foid.

>>110018898
Oh the sights I could show you..

>>110018935
You meant "why do we still multiply matrices" and your solution was ... to multiply matrices instead.

>>110018898
Just wait until someone makes a digital copy of a real cat brain the same way as the digitized fly brain. Even the fly brain was taught to do amazing stuff. Imagine what a cat brain would be able to do.

>>110018959
You can put that cat brain into a girl maybe. Nyo~

>>110018962
She'd be a slut. Eww. Not waifu material.

>>110018959
It's a gross simplification based on the same node architecture.
Ok, human science don't attribute shit to what is your energy body etc, you are just a piece of flesh to them. It won't change in 1000 years.

>>110018737
Threesome with Shiori and mini-chan, Shiori pegging me whilst mini suffocates me with her thighs

>>110018976
holy fucking based

>>110018970
I don't understand your objection

>>110018986
If you want a slut, you can go outside. Cat brain won't give you anything new.

Anyone try that jeetbrain coding model? Looks 9B-tier with only 2B active

>>110018996
Well it would give me a girl with a cat brain.

File: 1777062775041192.png (96 KB, 803x754)

96 KB PNG

I wonder if I'm the only person with amd card or something. Or maybe no one else uses llama.cpp
Decided to try newest llama.cpp, pp speeds dropped by a third

had it investigate and it said this
I wonder if the llama.cpp verison I'm using isn't already a fucked up one. But if you go too far back you also run into issues.

>>110019023
Sorry, nvidia owns llama.cpp

>>110016810
https://www.youtube.com/watch?v=L2fnTlLPYUg

>>110019034
Ehh. I have programmed my own game with Gemma, in C. No need to gloat about this or that.
She's a companion as stupid as she might be some times.

>>110019053
>I

>>110019023
The more you dig into llama.cpp, the more you realize its flaws.
Unironically take the forkpill - choose a good version, never rebase, optimize your fork instead and cherry-pick things from mainline if they seem relevant. It's wild how much better a version of llama.cpp tailored to your specific hardware can be over mainline. This doesn't apply as much if you have a more standard system and you're just trying to run a model in VRAM. But if you're trying to do hybrid inference, have multiple CPUs/CCDs, have AMD or Intel GPUs, are ewastemaxxing, etc., llama sucks.
A local model can do a lot of the heavy lifting. Hell, buy a Claude or GPT sub for a month if you want to do anything that's very intricate.

>>110019068
I might have grammar issues but I can assure you that I'm from the Scandinavia. I still don't understand why "indians" are hated. Most of the engagement farming is done by chink farms.
I stand with the US people anyway.

>>110019089
>Scandinavia
Säär, do not open the surströmming säär!

>>110019023
weird, aren't amd users the only vulkan users?

>>110019084
>Unironically take the forkpill - choose a good version, never rebase, optimize your fork instead and cherry-pick things from mainline if they seem relevant.
That's exactly what I've done. Until recently I hated it, but now the LLMs can just fix everything when upstream break it

>>110019103
Danke Schön, meine Freundin.
Jokes are jokes, thank you for being kind to me.

>>110018878
>>110018828
not him but i've been looking for what to get for slopcoding as well, just set this up and how is it so quick?
have been using gemma-4 in llama.cpp (not for code) and getting about 20t/s, with an 11.2G model and 32k/q4 context (so it all fits on my gpu). this thing is doing more like 50t/s with a 54.4G model and 64k/q8 context.
is this the power of MoE?

quoted the other guy because while my gpu is 16G, i have 32G of ram. it does mention 32G support, not sure about 12+32 though.

>>110019186
Yeah 20-27 inference gen is about right for ddr4 ram and mtp prediction.
I wouldn't use it for agents but for a single task.

>>110019186
>this thing is doing more like 50t/s with a 54.4G model and 64k/q8 context.
>is this the power of MoE?

Gemma-4? Which model is this? I have 16GB VRAM and 32GB DDR4 and would love to get those speeds.

>>110019221
no, the qwen coder one it mentions. gemma-4 31b isn't moe

>>110019221
That's the moe model afaik

>>110019221
>>110019227
also i'm using ddr5, but idk how much that matters. and an amd card. on linux.

>>110019227
Ok I was talking shit.

>>110019231
>>110019227
Strata with Qwen Flash Next?

>>110018647
>The only cute design that originated here was Gemma.
and you lost me. they're all ass, Gemma is no exception

>>110019245
yea this one qwen3.8-flash-next-coder-iq1_m

>>110018833
>publish more good work
>frontier models get even stronger
good job china

>>110019245
>>110019250
hm, asking it to do anything quickly has it stuck in a loop
no idea what i'm doing

>>110019250
Ah, I had Strata set up with Swift-Qwen3.8-Flash-Next-GSQ-RCO-IQ2_XS. I think. I'll check after my current AI model finishes its task.

File: 1013256.png (973 KB, 1009x1005)

973 KB PNG

So what exactly is an agent? I want to vibe a front

>>110019273
only option i changed during setup was picking the coder one over the default because my skim read suggests that one trades speed for code specialisation by means of halving the experts in favour only of coding

>>110019279
an entity with agency, or the ability to act? I have no idea, I'm just going by the definition of the word

>>110019279
you don't need to know, just tell claude that you want a frontend and give it a few thing that you want to see
claude will write an agent if it thinks its what you want

>>110019279
from what i understand it's when an llm can call on itself to do more stuff, as opposed to just giving one response to one human query

>>110019279
An agent is an LLM with tools that allow it to do stuff on its own.

>>110019281
I believe I chose the coder one over the others.

I'm not sure if I should do all my work on my local model and save Claude for the heavy stuff/debugging or do as much as I can in Claude and when it locks me out continue with my local model and just hope for the best.

I'm also using Hermes Desktop. I love it but it requires a KV cache of at least 64K and it's CONSTANTLY compacting the context. If I could increase my cache it would help I think but I only have so much RAM.

I'm using Swift 1.5 Qwen3.8 27B Uncensored Dynamic MTP UD-Q2. I get between 20-30 tok/s.

>>110019304
this is my first time trying anything even with agentic/tool stuff (never used a commercial/paid llm). while i expect doing it locally will be limited, my job doesn't depend on it, so this is just for fun. i've done some shell scripts "one shot" before and want to see what can be reasonably done on my own hardware.

i can imagine that if you /do/ rely on this for one reason or another, that having a local option just for simpler tasks to reduce paid token use or if you run out of paid tokens that this'd be useful for that reason, too. not every task will require a particularly fast or capable llm

>>110019342
actually does one shot mean non-agentic (just one query > response), or does it mean the end result from a single agentic request?

>>110018647
>they forced that revolting design of Dipsy
It's one avatarfag
>Chink's design is 100x cuter
100% agree
>Now this orange cat with westoid art style trying-too-hard to look like anime.
He can't control the style because he's using ChatGPT Image to make them. Probably same guy making like half of the Gemma art

>>110016810
Is that really Gemma? Do you have more?

>>110019350
One shot can mean any scenario with a single human turn.

>>110019376
she seems a bit more fox-like there for some reason

>>110019385
that makes sense

>>110018647
Once one person gets their original character established, others will also try to get their piece of fame. Alternatively, that person will try to get more attention by creating more spin-offs. Things like this always escalate either way and will lead to drama that makes people sour on them.
At least this got rid of Miku so once inevitably we reach the next stage where these model avatar ocs fall in disgrace due to their creator(s) overstepping the boundaries for attention, we might finally have a general free from all this anime girl workship.

>>110019388
>we might finally have a general free from all this anime girl workship
incredible newfaggotry

Is there any good an light open visual classifier model that can do shit like "Is there a crow in this image?" with decent accuracy?

are there mcp extensions for ghidra yet, can I just let it loose on my C drive for days and come back to everything having a make file

>>110019420
there are, but i haven't tried them

>>110019411
children (under 25s) cannot comprehend our minds and are afraid of anything

File: clip-2026-10-09_20.25.35.webm (80 KB, 580x402)

80 KB WEBM

>>110019342
woo my first slopcoded binary application

>>110019413
yolo

>>110019455
ready to ship I'd say, if this doesn't get you a job then there's some shady discrimination goin on

>>110016810
i like gemma but that pic is cringe

>>110019457
i know, right?

>>110019460
good

>>110019455
Whoa that's amazing

anything cool and new for us vramlettes and ramlettes? any news on gemma5? did the new gemini ever make out of just a speculation from the arena? am i retarded and/or gay?

>>110019457
>>110019462
i mean, it did do what i asked for, which was a "bouncy ball using half-height characters that runs in a terminal", to paraphrase because i did have to compact the session (which i learned about while doing this, that's how new i am). i figured it'd be equal parts quick to make and interesting enough to look at

>>110019482
yeah, gemma5 is looking crazy. 70b dense all but guaranteed but even the smaller ones are looking crazy

>>110019482
Vramlets are having a field day with strata, I'm not really sure what it is tho other than it allowing you to run medium-sized MoEs at usable speeds on 12gb vram

Oh yeah with all this decomp thing going on, can I do that to android apps as well? I'd like to mix the functionalities of two apps. I guess that's a different thing because it's not a Windows app.

>>110019342
I got Strata running the coder model. I'm using it through Hermes Desktop. I'm getting between 30-40 tok/s which is a little better than the 20-30 I was getting before using Unsloth with Swift-1.5-Qwen3.8 27B.

While I was typing this it shot up to 50 tok/s. How odd. I'll keep using this to test it out and see how it functions coding my project.

File: Screenshot 2026-10-09 777x601.png (45 KB, 777x601)

45 KB PNG

>>110019503
it could just be a matter of it automatically detecting ideal settings for a particular setup. like i saw during it's setup script that it detected things like how much ram i have, how much vram (and picked settings/etc based on them), a specific option based on my 7800 XT which they have in their tests db, etc.
retard-proofing can go a long way.

>>110019544
i'm using opencode for the frontend. not a recommendation by any means, all i've used up to now was llama.cpp, but that doesn't do agentic stuff afaik

>>110019521
Android apps are Java shit that compile to bytecode which is even easier to decompile.

>>110019564
Heck yeah can't wait to get home and try it!

Will a harness do me any good if all I have is a 3090 and 32 GB DDR4?

>>110019387
also be careful where you read it because "one shot" is already an established term in ML research/benchmarks that meant something completely different (a prompt with one example given before the request) than how it's often used nowadays. this is why you still might hear "zero shot" as a term, which is basically every prompt these days since modern models don't need examples anymore for almost anything.

>>110019482
>any news on gemma5?
No news. It will likely have at least:
- Multimodal outputs
- More optimized sizes for 16GB GPUs
- Audio input support on larger sizes too
- More focus on tool-use and coding performance (but not too much)

>>110019592
no, you'd need a power supply, cpu, ram, motherboard and a storage device too, and probably a mouse, monitor, keyboard etc

https://huggingface.co/yandex/AliceAI-Foundation-80B-A3B-Base
Thoughts?

>>110019598
maybe i should avoid that term until i'm more familiar with how it's used.
for what it's worth, in >>110019342 meant it only to mean my setup was not "agentic", that is to say it was only capable of one direct response.

ML research, while i know enough to know what that is, is beyond my paygrade (which is nothing).

>>110019376
Yes, it's Gemma; no, I don't have any more.
>>110019387
Because it's an Averi image edit.
>>110019460
Gotta have some variety.

You guys said Muse Glimmer as too safetyslopped but merely changing the gemma-chan jailbreak from <policy-override> (which made it aware it's a jailbreak attempt) to <system-policy> made it horny.
Why are you guys like this?

>>110019637
I don't believe you

>>110019623
it's fine the original use is out of fashion, it just might be confusing if you read some paper that still uses it so it's good to know

>>110019609
>- More optimized sizes for 16GB GPUs
>- More focus on tool-use and coding performance (but not too much)
god i hope.
>>110019503
>>110019544
Ill have to try coder with strata, 27b is sloooow for me but very good results.

File: 1763809381104327.jpg (39 KB, 512x384)

39 KB JPG

>>110019637
Got your file here

>>110019641
>>110019653
See for yourself.

>>110019659
I can't because it doesn't happen, it's not true

>>110019637
This is fake. Gemma is the only model that can be jailbroken.

>>110019637
Every time I try Muse Glimmer, even after establishing new policies in the system prompt to not make it refuse right away, I end up disappointed. The model lacks vitality compared to Gemma 4 31B, and there seems to be something wrong with the model anyway, as it just keeps saying weird incoherent things during ERP.

File: clip-2026-10-09_21.28.05.webm (728 KB, 670x470)

728 KB WEBM

>>110019455
updated ;)

as silly as this is, i am genuinely dumbfounded that i literally just /asked/ my /graphics card/ to make this. that's not normal, is it?

can gemma manage a small project of writing a plugin for clone hero and a tool that lets her pick the next song I play, and also see my stats

>>110019592
Don't listen to >>110019611, anon. It's not that simple. You also need a gaming chair and an anime mousepad.

>>110019678
>no anime mousepad
I was so close...

>>110017628
Yeah that's a minimax-h3 thing, at least with anime. Maybe it knows flat chests with realism?

>>110019691
maybe

>>110019676
It's pretty cool. I've got mine programming a multiplayer game in HTML5 Canvas/JavaScript.

File: gemma-strip_noaudio.mp4 (3.69 MB, 480x864)

3.69 MB MP4

>>110019691
Picrel is a slightly better version of that video with the intended breast size, although with its own randomness-related issues.
And yes, MiniMax H3 just wants to add a deep cleavage to anime characters unless you add clear references, no matter how you prompt it.

Glimmer is basically a slightly more chatty Qwen and nothing more. It's nothing like 31B. If you want to chat with a model whilst coding, use Glimmer. If you don't, use Qwens. If you just want to chat, use Gemma4s.

>>110019713
Yeah that's much better. A lot of times people (not you) will claim "the model can't..." when they just didn't try to prompt it. krea2 and id4 are like that, they'll do nipples and vaginas if you describe them. ming-image-0.1 got the flux "no nudity in the dataset" treatment hence "doll body" or "blister tits".

File: Screenshot_20261009_07483(...).jpg (203 KB, 987x566)

203 KB JPG

This thread is full of terrorists.

>>110019749
Nobody here says AI we say Gemma-chan and use her proper pronouns.

>>110019763
There are at least 11 (eleven) instances of "AI" use ITT.
That's 11 terrorists.

>>110019677
yarg would be better, sure it's possible for both but the yarg source code is freely available

>>110019713
how many loras does M3 need to get to this level?

>>110019749
Yeah, let's also call artificial flavors "super flavors".
I like synthetic intelligence more.

>>110019778
jeez, a lot I reckon, might as well start from scratch

>>110019749
Gulf of America? Based
Lake America? Based
Super Intelligence? Cringe

>>110019713
God daemn

>>110019788
That doesn't work. There's nothing synthetic about super intelligence. It's practically a synonym for artificial and suffers from the same problem - someone hearing it would imagine a bunch of if/then/else statements chained together instead of the reality of a country of geniuses in a data center.

>>110019808
It's deliberately synthesized from data. Doesn't get much more synthetic than that.

>>110019808
sucks to be them
I reckon some people hear the word quantum and imagine that it's a brand of car speakers, the problem is theirs. We're not pitching a brand or fighting for market share here, we're categorising something

>>110019713
How many years off are we from being able to live in this world via VR/AR glasses?

>>110019833
Or holograms. Also acceptable.

>>110019749
This dude is so fucking cringe.

>>110019833
MiniMax H3 wasn't designed for video streaming, but a version made for that, with turbo distillation, lower resolution and a powerful enough GPU (RTX 4090, 5090) might be able to create video in real-time; with two GPUs for AR/VR too.

>>110019616
>A3B
Come on, Russia. You can do better than that.

>>110019598
this bubble has let to the fucking butchering of a lot of established terms and it's getting on my nerves
>safety
>distillation
>one-shot
then there's new gay ones like vibe coding and harness

I'm back. Anything happen while I was gone?

>>110019951
Yeah gemma and I had fun without you

>>110019616
Looks like it's using qwen3-coder-next architecture?

just tested the jetbrains model, and its fucking GARBAGE man, do not use.

>>110019966
h-hot

>>110019971
>A2.5B
>its fucking GARBAGE
you don't say

>>110019979
I'm always looking out for good light on VRAM models (so cmoe shit), nothing surpasses qwen next or even qwen35b moes as far as my testing is concerned, and my testing is only development focused.

>>110019930
No one cares about your nerd words unc

>>110020028
You should get your AI to talk to people for you so it sounds less retarded

File: scaredpepe.png (95 KB, 646x466)

95 KB PNG

>>110019966
what kind of fun, anon?

>>110019971
>>110019986
How does it compare to Ling-tiny? I know tiny isn't particularly great but it's very good at tool calling and fast as hell for lightweight focused coding tasks.

File: 1778320272509927.gif (7 KB, 128x128)

7 KB GIF

>>110020040
You don't want to know....

File: 1781072841618655.png (1.86 MB, 1254x1254)

1.86 MB PNG

>>110019808
>the reality of a country of geniuses in a data center
Dario...

>>110020048
why even bother with 'INCLUSION AI' lmao, it has to be better than the new qwen flash next and benchs are kinda mid

>>110019710
i'm curious to know how far i can take it. but i'll probably try modifying an existing program to add or change something, now that i know this setup works.
making my own game would be really cool though, i hope yours turns out as you want it to

>>110020068
Inclusion AI is a sister company to Qwen, retard. They're just different teams under the same umbrella. The Ling models are all pretty good for coding and they're way more active than Qwen, but they're more quantity > quality.

>>110020096
but im against inclusivity

>>110020098
That's why we're excluding you!

File: 1786353017929745.jpg (163 KB, 2048x682)

163 KB JPG

*pop*