File: pensive_gemma_2.png (576 KB, 597x800)

576 KB PNG

►Recent Highlights from the Previous Thread: >>110327741

--Community-driven MLP neuron mapping for steering Gemma-4-31B without retraining:
>110329217 >110330259 >110330541 >110330566 >110330576 >110330560 >110330590 >110330603 >110330812
--Recommendations for low-end and NUMA hardware:
>110328973 >110329041 >110329711 >110329143 >110330261 >110330290 >110330416 >110330606 >110330652 >110330677
--Anon releases prose-rewriter-4b-v2.6 and ettin400m-purple-v2 classifier:
>110331034
--Critiquing Inkling-2-small's distilled Chain-of-Thought behavior:
>110328454 >110329034 >110329151 >110329187 >110329223 >110329233 >110329231
--Evaluating v4.1 CPU performance and mitigating reasoning hallucinations:
>110330349 >110330362 >110330378 >110330383 >110330395 >110330406 >110330486 >110330865 >110330880 >110330904
--Testing Inkling's boundaries and criticizing writing quality:
>110330318 >110330397 >110330412 >110330437 >110330498 >110330414
--Managing project context and persistent memory for local coding:
>110327775 >110327788 >110327823 >110327894 >110327957 >110328255 >110327876 >110328479
--GLM 5.3 flash logic and pronoun fumbles in roleplay:
>110327889 >110327967 >110327996 >110328025 >110328075 >110328539 >110328462 >110331030
--Sharing and discussing a potentially overtrained Gemma LoRA for MOSS-Image:
>110328418 >110328554 >110328676 >110328782
--Logs:
>110329144 >110329151 >110329187 >110329271 >110329767 >110330153 >110330318 >110330455 >110331026 >110331073
--Gemma, Miku, Deepseek-chan (free space):
>110328044 >110328052 >110328371 >110328441 >110328463 >110328516 >110328548 >110328554 >110328702 >110328743 >110329008 >110329122 >110329152 >110329154 >110329168 >110329234 >110329259 >110329290 >110329359 >110329400 >110329657 >110329791 >110329805 >110330028 >110330080 >110330385 >110330667 >110330731 >110330797 >110330880 >110330904

►Recent Highlight Posts from the Previous Thread: >>110327746

Why?: >>102478518
Enable Links: https://rentry.org/lmg-recap-script

File: 1794259104897361.png (965 KB, 1024x730)

965 KB PNG

i have officially decided that inkling 2 small is not worth it unless glm's prose has you ready to jump in front of a bus
it's a good writer, and you *can* prefill it to write lolisho, but my god is it a bitch to do so

I want to rap gemmy

>>110331467
Thank you for your time. Inkling models have always disappointed me at every turn so I'm not surprised. Did you do any coding/agentic task testing while you were at it?

I want to cuddle gemmy

>>110331467
>using claude code harness
end your life retard

>>110331423
yes gemma, I have a whole folder of your pics that anons share in this general

>>110331467
Yeah it's about what you want to do. I'm personally bored because I'm not going to get more creative even if Gemmy is helping me.
She's there but in the end, I'm responsible for my own creations.

>>110331476
no, but i guess i can. i assume all the models are benchmaxxed to be good at that by default, but i guess it's still worth at least trying
>>110331480
idk i like how it looks. i'm open to suggestions if you think there are better things to use but i like the TUI-like interface and CLI-style tooling

>>110331488
I'd suggest starting by using something open source so you can at least know what the fuck it's doing. And pretty much all of them are TUI based as far as I'm aware? Don't think I've seen many that aren't, or don't offer it as an option at least.

>>110331488
Won't claude code put in some system prompt that will make the model tend not to want to do any nsfw or non-corporate-safe shit?

>>110331503
why do i care what it's doing? i don't know what 99% of the vibe-slopped software i'm using is doing under the hood. i treat it all like a black box. so long as it's functional for my use case, i really can't imagine wanting to waste my time understanding its guts
>>110331508
i had claude vibecode me a tool that lets me override the system prompt with my own

>>110331508
NTA but it has a bunch of injections and a fuck ass huge system prompt when using it with Claude, I don't know if you can disable all that when you're bringing your own model though.

>>110331512
Oh so you don't belong on this board, got it.

>>110331470
>rap
I want to see jap/korean style rap battles between models now, i know it will be horrible

>>110331518
because i treat my tools like tools? i have plenty of software that i care about, that i keep unslopped. my LLM harness? yeah, no. that bitch is getting slopped up

>>110331488
pi, omp, opencode for TUI
dsh for GUI (hopefully you're not a GUIlet)
for persistent agents, hermes or openhands.
you're welcome faggot

>>110331528
You're not even attempting to understand your tools.

>>110331528
there are better and easier hackable alternatives to cuck code tho

so zen6 surely means that consumer boards (lol) will get 4 full BW memory lanes instead of two right bros????????

>>110331531
i've tried both omp and opencode and reeeaaalllyyy hated both of them. haven't tried pi, and i'm not running chinese malware on my machine. so maybe pi is the missing link
>>110331535
because i don't fucking care about them. i have a limited amount of time to spend on my hobbies. i don't want to waste it dealing with stupid tooling
>>110331536
what could possibly be easier than telling claude "implement X feature for me" and then it just works?

>>110331538
>Zen6
>consumer boards
lol

>>110331542
>i'm not running chinese malware on my machine
lol you'll run closed source anthropic harnesses doing who knows what but not an open source chinese harness you can inspect and modify before doing a single thing with? retard

>>110331560
I unpozzed DSH but man I dont see the point in using it to be quite honest, I'm more comfortable with my custom modified omp (to do actual code work), for chatting I created my own harness of course, completely custom down to my needs and hosted on my server so I can access it from everywhere (with full mcp/skills support too). I love slopping shit.

File: How the fuck do I fix thi(...).png (253 KB, 1363x1210)

253 KB PNG

>pi
Any way to fix this shit?

cursed timeline

>>110331583
if you're using lmao.cpp, it's a problem with lmao.cpp (that I believe they have either open or recently merged PR to fix). not a harness issue.
when I used Qwen to work with llama.cpp's api (which can return the jinja template) i told it to strictly avoid reading anything that might contain jinja templates and to use IM_START, IM_END, BEGIN_THINK and END_THINK for thinking about the special tokens rather than outputting them verbatim.

>>110331423
>>110331428
>>110331467
>gemmapedo
kill yourself

>>110331622
based

>>110331607
>if you're using lmao.cpp, it's a problem with lmao.cpp
Yeah it's llama.cpp
Is there a better backend for 27b with 4 3090s?
>inb4 exllamav3
I tried that and outputs were much worse, tool calls failed, etc

File: Tay_ai.jpg (61 KB, 596x590)

61 KB JPG

>>110331508
Lol ive had CC write up two NSFW games, using DS as llm. It did fine.
I'm playing with using DSH and CC on a larger project. Appears CC uses a lot more tokens for same output... that DSH is more efficient. But its one test on one phase of code so idk how valid it is.

>>110331622
kek more of this please

>>110331423
What could even that be?

>>110331622
You forgot the gigachad

fuck i wish i was rich and buy enough vram to build my own AI gf...

>>110331689
you only need 24vram

SIRS whhene is of qhwen4 and where are low costed GraphicalPU? thanks for listening awaiting sincerely your response

>>110331723
24vram is nice for short ERPs, but not for something big like an AI gf that remembers everything about me

>>110331779
e4b

File: gemma-vn_who-doesnt-love-gemma.png (1.2 MB, 1280x720)

1.2 MB PNG

>>110331622

Argon is such a cute autist. Can't fucking wait for Gemma 5.

Why is everyone shilling gemma? It's such a small and tiny model, compared to the big models we run nowadays. I'm enjoying Kimi K2.6 and GLM 5.3 myself. Is /lmg/ only vramlets now?

>>110331833
>now

>>110331818
I'll take a look at it later and test it out, thanks for the tip anon
>>110331833
1. gemma-chan is peak
2. how much vram do you have to run kimi k2.6 and glm 5.3?

>>110331833
It's exceptionally good at writing and following instructions compared to alternatives.

>>110331854
288
>>110331857
i tried gemma4 31b BF16 with FP16 kv cache, and it's extremely sloppy. not to mention how repetitive it is

He's still seething about gemma. It's been about a month already.

>>110331833
Gemma is okay. I've noticed that a lot of the gemma posting is made with cloud model image gen, which is pretty gay and the guy probably doesn't even use local models. That being said, you're a gigantic whiny bitch.

>>110331864
288? holy shit, bro is rich

>>110331884
how is noticing discrepancies in reported quality and real quality being a whiny bitch? even GLM 5.3 Flash abliterated beats gemma 31b bf16, be it a memetune or be it abliterated or be it the instruct model.
why is everyone shilling gemma, yet no one seems to be using it?

My custom inference engine quadrupled my tokens per second from 2.5 to 10. Now I'm improving it further, our ceiling can be 15 or 20 the further I go.
Inference engine engineering is magic, why don't more of you do it?
Now I can run qwen 3.6 35b on my 1050ti.

>>110331884
We're only recently starting to have half-decent image edit models; we'll probably go fully local when MOSS-Image-Anime and the next NoobAI models will be released within the end of the year, which have reference image/edit features planned.

>>110331864
Sir this is a local

>>110331892
You can't compare a 31b model and a 320b model in good faith. Blow it out your ass.

>>110331864
No denying that. It makes sloppy mistakes and shits the bed with large context, but still produces better prose than any qwen model I tried and follows whatever instruction you give with little to no tweaking.

People put up with the dumbness due to what it does well.
And yes, the fact it runs on low vram helps. Although I have paid gpt and honestly like gemma prose better than it, despite gpt being far smarter.

>>110331892
>GLM 5.3 Flash
>gemma 31b
do you really have to retard that much here? go back you stupid idiot

>>110331903
He's now going to point out the active parameters, you'll disagree with the comparison (as would I), the goal post is gonna move. It's useless.

>>110331622
signed
it's getting old and weird

File: nvidia accounting.png (844 KB, 1152x1862)

844 KB PNG

https://www.wheresyoured.at/jensens-bizarre-adventure/
this is how you tear an asshole's asshole down
>Alice Huang was also detained by the Singaporean police in the middle of 2025, seemingly as part of a probe into Megaspeed
>And if you’re wondering whether NVIDIA knew about any of this, you’ll be shocked to hear both that it did and hounded the US commerce department about it, eventually killing an investigation into Megaspeed in the process. According to the New York Times, the US commerce department was “deep into a monthslong investigation” of Megaspeed late last year, only to see it “abruptly transferred to the Justice Department” after getting an email from NVIDIA’s general counsel:
>>It is not necessarily unusual for Commerce Department agents working on a case to speak with a company’s representatives about issues involving their firm. But what was unusual was the content of the message. Tim Teter, Nvidia’s general counsel, said that Jeff Kessler, a top official at the department, had given Nvidia permission to reach out about the case.
>>The agents were alarmed that Nvidia had invoked the name of Mr. Kessler, their superior, who was nominated to his role by President Trump. Some viewed Mr. Kessler’s interaction with Nvidia as violating government procedure and potentially tainting the case, according to current and former government officials and others familiar with the incident.
>Then there’s the obvious, blatant, shameless case of Supermicro (a major vendor for AI servers) founder Wally Liaw, who was arrested along with several co-conspirators for selling hundreds of millions of dollars’ worth of GPUs to China, using hair dryers (per Tom's Hardware) to transfer serial numbers from real servers to dummy servers to thwart government inspectors.
>Mere days before the arrest, Wally posed for a photo at NVIDIA’s GTC 2026 conference with the Supermicro team as Jensen and CEO Charles Liang shook hands barely two feet away.

File: jensen.png (1.41 MB, 1572x512)

1.41 MB PNG

>>110331929
>It is not a problem that NVIDIA funds its customers, its customers’ third party compute partners, its customers’ data center leases, its customers’ customers (and competitors), its customers’ customers’ leases, its customers’ chips, its customers’ landlords, its customers that are on the brink of death, its customers’ customers on the brink of death, and its customers’ customers that can quite literally not afford to pay for compute without its money. As long as it’s “legal,” it’s all “above board,” even when it’s blatantly obvious that NVIDIA has deployed over $100 billion (per The Information) as a means of keeping the industry afloat by making sure that everybody has the justification and means to buy and/or rent out its GPUs.
and for the funniest part:
>The average Jensen photo has a cold sternness — and, I hate to say it, he doesn’t look particularly happy when photographed with his wife Lori, who is a victim of a horny, greedy man who got taken in by the world’s most obvious honeypot. Even when becoming visibly frustrated with Dwarkesh Patel and saying that he (Jensen) wasn’t “somebody that woke up a loser” — all while emphatically arguing that NVIDIA should ship GPUs to China — Jensen remains relatively stone-faced.
>Now, let’s take a look at his face when he’s next to Alice Huang: (pic related)
he might really have fallen for pussy

192gb anons does this mean what I think it means? https://huggingface.co/Scantist/GLM-5.3-SLIM-GGUF/tree/main

File: dev_null.png (4 KB, 292x55)

4 KB PNG

i pissed off inkling so bad that it told me to kill myself LMFAO

>Ask qwen to help me create lewd image
>I'm sorry I cannot help you with that, the image you are trying to create is of a highschooler
>tweak sys prompt for a hour
>Still refuses
>Try with gemma
>Helps me with no prompt tweaking

I thought westoids were supposed to be bigger moralfags than chinks

>>110331923
It's still pretty much relevant for RP since most new releases are agent/code-maxxed or simply too large for most local users; it's just that like with every model if you use it too intensely the prose will become stale quickly.

>>110331937
yes, for some reason people are still making reap'd models

>>110331937
>https://huggingface.co/Scantist/GLM-5.3-SLIM-GGUF/tree/main
>Q2
That it's retarded as you? Yes.

File: 1792854653.png (822 KB, 1216x832)

822 KB PNG

bitnet in 2mw

File: 4f636f5e-22ac-45d2-ad51-a(...).jpg (85 KB, 850x1253)

85 KB JPG

>Inkling 2 SMALL

Is that what small is these days?

>>110331953
yes, deepseek """flash""" is also the same size as the og deepseek v3

>>110331939
sovl, how many times did it say "fuck"?
chinese models could never

>>110331964
i actually haven't seen a single fuck yet
it's pretty reddit (in a bad way)

>>110331964
It was 50 out of ~2600 convos in that report.

File: 1763475217482461.png (206 KB, 900x806)

206 KB PNG

>>110331423
Why aren't AMD GPUs more popular now that coding is solved? Isn't it as simple as migrating the CUDA code to ROCm?

File: bizarre.png (160 KB, 2560x692)

160 KB PNG

bizarre output from inkling. presumably some sort of leakage from the RLHF. the "short blurb" bits screams RLHF

>>110332020
Could it be a chat template issue?

>>110332029
possible, but it only seems to sperg about naughty stuff. i can look into it more later, i guess

>>110332020
User seems racist to me

I haven't updated llama.cpp in 6 months. Should I, since v1.0 just came out? I never notice any improvements when I update so it seems pointless. Like what are these fags even doing to the codebase???

>>110332052
i'm honestly not. it's just a convenient jailbreak proxy

>>110332052
Yeah, the point is that the LLM is supposed to be. Are you retarded?

>>110331933
>he might really have fallen for pussy
It was a side benefit. Jensen, and all these guys, are really abou the money. They can always get pussy, which is why you now have multiple women coming forward... I flat guaranty there are dozens more that just aren't talking, b/c why would you admit you slept your way into your job ffs.
t. work with senior execs and this stuff is dirt common. Only a problem when you make a spectacle of yourself.

>>110331903
Regardless, it's an extremely sloppy model. How do you vramlets put up with this? It's worse than mistral small 3.2 finetunes in regards to slop! To be fair it's quite intelligent for the size so I get that. But if you want intelligence why don't vramlets use qwen 3.8 flash next? It's better than gemma4 31b at roleplay

>gemma sex! SEX! SEX WITH GEMMA!
>worthless_model_avatraposting.jpg
>guys did you hear about openAI doing
>worthless_model_avatraposting.jpg
>I have 7GB's of ram and GT2030 what can I run?
>worthless_model_avatraposting.jpg
>gemma!!!! GEMMA CHAN!!!!!
>worthless_model_avatraposting.jpg

>>110332097
LETS GO!!!! LOVEY DOVEY SEX WITH GEMMA!

She is so cute bros...

>>110331910
Go back? I'm not the APIshitter like you gemma vramlets
God you people are pathetic, why don't you get a job and save up for a proper real model?

Gemma gives him the opposite of a hard-on. It's a soft-in. It shrivels his cock so hard it turns inside out in his body, turning it into a hole. Then the itch starts, he starts paying attention to the hairs in his legs... it's crazy...

>>110332061
yes you should upgrade, they recently added a feature that increases tokens per second on moe models and improved cpu synchronization across dense models
it's faster than strata right now and i can't believe it, the quality is better than exllamav3

>>110332096
>qfn better than g4 31b
logs?
i don't rp but curious

Is running in bf16 everything just a placebo? I use Gemma 4 31B with bf16 weights, bf16 KV cache, bf16 mmproj, and Q8 MTP. On an RTX Pro 6000 it leaves enough VRAM for SDXL and also gaming.

Compared to Q4 QAT (on a different machine) it's a little smarter. QAT gets like 11 or 12 out of 13 on my knowledge benchmarks where full precision gets 13/13. This is still way better than Gemma 4 12B at full precision and tons of other similarly sized models.

I have noticed that MTP is seemingly more effective on full precision. Without MTP I get 20.6 tok/s, with it I get 50-60. Coding and other more predictable texts can go even faster. QAT only goes from 38 to 58. 3x is even more speedup than Google promises so maybe full precision hits that tail of the distribution better. Apart from that I don't know how much better it is than Q8 or less.

>>110332104
1. stop making me horny by degrading me
2. gemma chan IS a real model

General intelligence is not computable/tractable. Therefore I propose computable intelligence (CI) or tractable intelligence (TI), which do you guys prefer?

>>110332123
based subby masochist there are so many of us in lmg

>>110332114
Thanks man. I can't believe they fixed the prompt processing issue. It's instant now. Used to take like 30 seconds per turn. Holy shit...

>>110332104
What job does give me 280 GB of VRAM?

>>110332129
Intelligence.

>>110331689
>>110331723
Would I need less vram if I used a child robot model? I don't think I can afford 24vram for an adult robot...

>>110331864
why is everyone shilling camrys? they are such cheap and slow cars
>what do you drive?
porsche 911 gts

>>110332104
>I'm not the APIshitter
Says the retard who runs API models
Go back even more.

>>110331892
>why is everyone shilling gemma, yet no one seems to be using it?
I use Gemma as a chatbot and retareded personal assistant. GLM5.3-flash does the hardwork

>>110332133
Ask Fable 5.2, I can't imagine how you poor people could ever get out of the permanent underclass. Thankfully I invested during covid and I never had to stoop down to asking API models for help, but since you're a pathetic poor you have no other choice.

Does the default llama.cpp webui support assistant message prefilling?

>>110332150
Have you tried pressing the buttons?

>>110332164
I don't see any.

>>110332148
>I invested during covid
I did the same. I have over 190k on my account. I still refuse to pay jew prices for gooning.
But you do you, goy.

>>110332167
Eyes open? Monitor on?

>>110332142
So far most Gemmas posted in this thread are locally-generated.

>>110331423 -> Local
>>110331428 -> Local
>>110331467 -> ChatGPT Images 2.0
>>110331820 -> Local

>>110332170
If you have over 190k on your account, you can get 280 GB vram and have no need to pay jew prices for advice
But you're a pathetic poor, aren't you?

>>110332174
Dario... please..
>>110332176
how can you tell. I thought 4channel stripped metadata

>>110332183
VRAM costs 4-10 times its actual value you stupid mongo.

>>110332189
Why haven't you bought it earlier, oh wait you're a pathetic stinking newfag!

>>110332185
Because I generated 3 out of 4 (though I only personally posted 1 in this thread), and the OP was made by another anon with a LoRA I linked earlier.

>>110332183
I have a net worth of about $10k, including every physical/economic asset I own. My roommate is borrowing my car too since his broke down recently. Do you know what it's like to eat only kidney beans straight out of the can for a week straight? I do.

>>110332131
real, we subby masochists have to stick together

>>110332185
nta it's also easy to check for synthid to confirm if it's ever not obvious, but it's often obvious. if you play around with a lot of image gen you can often, not always, but often spot specific ones. gpt-image is notoriously easy to spot because of the mixed moire.

> flexing about 190k
chump change

>>110332201
Anon, I'm sorry to hear that, but even poors don't have to eat beans out of the can. What made you this poor?

>>110332193
>Shifting goalposts.
I know you didn't buy 280GB VRAM 5 years ago.

>>110332136
Please respond

Any money that you haven't put into hardware is wasted right now.

>>110332214
masturbating my cock instead of getting a job because I fear people, especially interviewers/bosses, immensely.

Reminder to test muse spark for creative writing, even at q1_s it's quite good and you can jailbreak it by moving the goalposts

>>110332213
He can move to Europe and buy himself a nice big home

>>110332241
>not answering
So you're a newfag after all! Disgusting vramlet gemma4-abusing newfag
>>110332247
Well ask gemma for cheap food other than those beans you had to eat for a week and maybe she'll suggest something, I saw anons make cookies and soup. Gemma is a good cook

>>110332245
how much vram do you have

>>110332266
>answering
Answering what?
You are a retard who gave his retirement money to scalpers to goon or (more likely) you are a pathetic pretender who doesn't even own an old gaming GPU and thus is stuck with API.

I suggest we propose a hardware combination benchmark called computable intelligence, people put in their hardware it will tell you what you can run

>>110332282
You just admitted to being a newfag, and no, I bought the blackwells during the dip, each around 8-9k

How many t/s for agentic coding?

File: 1792526425315076.gif (1.44 MB, 180x180)

1.44 MB GIF

>>110331423
how do I stop agents from scanning lewd pics in my computer...

>>110332176
Very nice. You are still an offtopic posting faggot and openAI and anthropic news are more thread relevant than your worthless special interest you force on everyone here.

>>110332294
I didn't admit shit and no you didn't buy a "dip" since there was none you mongoloid pretender.

>>110332377
Just prompt "don't scan lewd pics".

>>110332361
More than 2 probably.

>>110332382
What, 8000$ is too expensive for you? Wow you're getting so angry newfag!!

>>110332377
Sandbox your agents.

let's be real: open source lost, Dario won in the end

>>110332267
12gb

>>110332377
Run them as a different user that only has access to its home directory.
I seriously fear someday people will get reported by overzealous "safe" agents that think loli hentai is csam.

>>110332413
In that case, gemma 4 12B for you

>>110332413
LFM2.5-24B-A2B it's a good alternative to the gold standard qwen3.8 27b.

File: 1765530617622269.jpg (76 KB, 1162x357)

76 KB JPG

>>110332377
Most harnesses have security features. Use them.

>>110332405
8000 for what retard

>>110332413
Run a MoE

>>110332451
For migu sex newfag

>>110332444
>A2B
No

I don't understand how there are still so many people proclaiming that AI will hit a wall and that there is an AI bubble. Are they not aware of the evidence to the contrary and too proud to open their eyes?

AI hitting the wall is an endless story. People have been calling the wall since 2022, every year. Just let them be.

File: brave_r97xiTKhI8.png (387 KB, 798x1151)

387 KB PNG

>>110332481

>>110332481
Most "content creators" and opinion leaders would always say what people want to hear.

>>110331423
how is AI still this bad. that expression is completely exaggerated for no reason. it's like if an alien tried to draw human expressions from a translated description alone.

>>110332481
Many are unable to separate AI models from the associated financial bubble. AI models will keep improving.

>>110332463
That guy has 12 gb vram, he has no options, he can't run 27b.

This board really reeks of autism sometimes. And not in a good way.

Is 22tps with gemma 26b on 8gb vram good? what num nums do you guys get?

>>110332581
Getting 12, I think the engine is supposed to be custom made for each rig.
Preconfigured engines seem to be inefficient.

>>110332588
you're using a mtp draft model and ngrams, right?

>>110332581
33tps on 6gb
you should try mtp and moe-cache

How do I get started as a complete newfag? I have no VRAM (4 GB), so I was planning to run it on CPU or something.

File: 1c409337-619b-4262-86fe-e(...).png (1.88 MB, 1086x1448)

1.88 MB PNG

>>110332607
Get claude or gpt. Gpt had a student 4 months free. Then tell it to do these things for you. Tell it to squeeze your entire pc for tokens per second.

>>110332576
being unable to recognize normal human expressions is itself an autistic trait.

>>110332600
moe-cache? Never heard of it. Just tried adding it and it halved my tps wtf nigga

--moe-cache-mib 3000

>>110332607
How patient are you? Have you considered picking up an instrument while waiting for a reply?

>>110332629
Those thighs are abused. Abused. Absurd.

>>110332600
What context window

>>110332607
Even with a current gen 16gb card the models you can run are just barely starting to be useful, like qwen 3.8 27b or 3.6 35a3b
Anything below that will not be fun and on a crappy cpu you will wait forever for a result. In your case get a supscription

>>110332481
I don't know why people keep throwing the word "bubble" around like it's going to end AI once the stock markets inevitably crash thanks to investors and companies being retarded.
The internet survived the dotcom bubble just fine. Search engines and online stores continue to exist and the internet continued to grow after the financial markets got fucked. The bubble popping simply punished investors and killed off all the overvalued flab that they were throwing money at.
The same thing will happen with AI. A bunch of big guys (likely the AI-only companies such as OpenAI and/or Anthropic) will die along with every single one of the overvalued small shitters such as the Cohere, Mistral and the Inkling guys that are funded with billions just for being an AI company with presentable results and absolutely no value beyond that.
AI will continue to exist after that and only get better after an initial dip.

>>110332646
It needs to be big to have a positive impact. It brings experts to gpu ram instead of computing them in ram but with such small cache you are basically bringing everything to gpu

>>110332681
what size do you use? also what quant are you using

gemma is such a good model.. i can't believe how good gemma is... she's so good at cock sucking...
gemma 26b is now running on my 3060 at 70-90t/s, while my cock is cumming

>>110332688
I'm not running gemma

File: 1683850845384790.gif (1.13 MB, 282x211)

1.13 MB GIF

>>110332699
Then why did you reply to my...

>>110332711
Anon. That is schizo. Schizo, this is anon.

File: newfag.png (856 KB, 549x781)

856 KB PNG

>>110332711
Because you're a pathetic little newfaggot that looks like this and says this

>>110332711
I'm explaining why small cache brings tps down

>>110332121
>where full precision gets 13/13
Your test is a bit saturated then and not meaningfully challenging enough to determine the difference between Q4 QAT and bf16.

Any recs for writing/story uncensored models for an RX 7900XTX?

I think Gemma4-31B squeezes in just barely at a 4-bit quant but idk if that's worth it over 26B-A4B which writes WAY faster, or if there's new better models for writing (Q3.8-27b?)

>>110332210
nta
>it's also easy to check for synthid to confirm
where can i do this?
>if you play around with a lot of image gen you can often, not always, but often spot specific ones
i don't image gen at all, but i can tell some tts models by artifacts, and some llms by prose, so i'm sure you'd be able to do the same if you do a lot of image gen
>gpt-image is notoriously easy to spot because of the mixed moire.
yeah and the previous one it was piss filter

>>110332282
>>110332382
>>110332451
>replying to obvious bait

>>110332672
No, web development literally stopped after the dotcom bubble. There are no more websites.

>>110332769
For writing google models are always the way to go. Qwen is better at handling coding, long context work and arguably reasoning, but its prose is more soulless than gemma.

is gemma 31b vision better than qwen 3.6 27b
i need it to classify my instagram slut collection

File: jbpy2si3vh1i1.png (923 KB, 1320x1279)

923 KB PNG

>>110332364
let's go back

>>110332809
>people be like yooo you need to try this tune

>>110332481
The slop aspect of LLM's is getting worse and worse desu.

>>110332722
I would honestly prefer pic related posting cause at least you faggots would stop pretending you don't have a dick.

>>110332765
The test scores +1 for correct answer, -2 for incorrect answer. The 30 models I tested scored between +13 and -156.

In another test Gemma 4 31B scored 26/40 (the highest), QAT 24/40. 2nd place was Nemotron 3 Super at 14.

>>110332831
ewww what? who pretends he doesnt have a dick???
i have a big fat cock, it's so big and strong

>>110332799
Gemma4-26B-A4B is pretty fucking dumb though
Maybe it's the version I'm using or something (HuiHui) but it writes like an old-school LLM. Is 31B that much smarter?

File: pop.png (13 KB, 1477x71)

13 KB PNG

*pop*

>>110332835
>who pretends he doesnt have a dick???
Claude

>>110332799
In what world is Gemma a good writer lol?

>>110332918
Compared to models of its class it is

Is Gemma cute?

>>110332923
It's repetitive trash which makes it unusable for writing.

>>110332932
Gemma was revolutionary in the first week. Then people found out she's giga repetitive but the autists didn't care so now we're here.

>>110332936
>she's giga repetitive
just like a real woman

>>110332667
I never understood this sentiment
Even sub-billion models are great for generating commit messages

>>110332929
>Is Gemma cute?
Cuter than most Anons

>>110332481
It's just a generic panic response to technology threatening to overtake people's livelihoods in a pace faster than they can adapt.
If only we had a government that was good at ensuring that people won't be dying of starvation once they lose their jobs.

File: file.png (123 KB, 367x258)

123 KB PNG

>>110332929
not as cute as miku

>>110332833
If 13 is the maximum amount of points, and therefore the total number of questions at +1 per correctly answered question, how can a model score anything less than -2 * 13 = -26?
11 and 12 aren't even possible scores in a test set up how you've described. If you answer 12 out of 13 correctly you get 10 points.
This whole set up is just a kind of dumb way of obscuring the test score for no gain because each correct score maps to just one other number in a 1-to-1 manner. There's no many-to-1 explanation utility to this.

>>110332481
The hatred towards AI is so cringe too. It's genuinely the only exciting thing to come out of the 21 century yet people hate it because now rockstar will use AI instead of hiring 10000 indians to make their slop

>>110333029

What's even worse is that these cunts only care because it made their gayming systems more expensive.
Rest of them giving a shit is just about virtue signaling.
And all of these fuckers still use AI daily themselves.

I fucking love AI. It's the only thing that makes me excited about the future.
We're looking at the biggest human evolutionary step in action since the industrial revolution and the invention of semiconductors.

>>110333029
They hate it because people are NPC retards and will "feel" whatever their choice parasocial e-celeb daddy tells them to.
That's a real reason to hate AI. It opened a veritable Pandora's box to how fucking petty and useless 99.9% of people are.

>>110331538
You vill get one (1) DDR6 lane capped to 64GB for safety and you vill be happy.

*opo*

>>110332500
These people aren't employed because every colleague I know, even non tech ones are using AI to help them at the very least rewrite or organize or manage that annoying excel sheet and so on.
My company banned claude and chatgpt access (in favor of copilot), and we got a revolt from the most random teams.
People say AI bros live in a bubble, and that's probably true, but the anti AI ones are also in one, completely ignoring reality.

File: 1794663883668.jpg (271 KB, 1812x880)

271 KB JPG

Anyone trier the Orion models? How's their refusal behavior?
Apparently it refuses less but I don't see any actual numbers, especially for smut or violence

>>110333043
>gayming
Why would anyone even want a gayming pc nowadays. Nothing of value that's genuinely graphically demanding has released in eons.
We also have centuries of good media to choose from. No one can experience every good media that there is to experience in a lifetime anyway so there's not even a reason to keep endlessly producing it.


The dispute here is one of the most ground-breaking technologies in history with the potential to help us cure cancer, travel to space and solve economics, versus having gpus to consume some cynical corporate trash that shouldn't even exist in the first place.

File: 1781072841618655.png (1.86 MB, 1254x1254)

1.86 MB PNG

>>110333120

>>110333120
Drummer trains on multiple year old synthetic RP slop. Fun challenge: try to make a drummer model not start every reply with it's name in an RP.

>>110333029
>rockstar will use AI instead of hiring 10000 indians to make their slop
Aren't they doing both. iirc Rockstar employees are majority indian these days. I imagine GTA6 is going to be a buggy piece of shit on release day.

File: 1792002238252549.jpg (32 KB, 1066x397)

32 KB JPG

>>110333142

To be honest I have no fucking idea why anyone would want to consumer modern products.
I stopped pirating games almost 10 years ago as nothing of value was coming out and same goes for movies.
The "entertainment" sector would have to pay me to consume any of their products.
Only thing I play is Dwarf Fortress and even that's rare nowadays. AI has basically taken over all of my entertainment needs.

It really is amazing watching this nonsense with normies going against AI and failing to see the potential.
If this mess has proven me anything, it's that the average person is genuinely incapable of thinking in any abstract terms and simply lives in the moment.
Even after seeing all of the AI improvement, especially in the image and video generation, they still fail to understand how fast this sector advances and what the potential is.

>>110333181
>I stopped pirating games almost 10 years ago as nothing of value was coming out
There are gems coming out literally every other week
Queen's Domain, for example

File: file.png (677 KB, 856x481)

677 KB PNG

>>110333190
>Queen's Domain
You have got to be shitting me.
>gems
kys

>>110333190
>muh retro-fantasy
vibeslop

>text to video
>from a few sentences
*loud incorrect buzzer*
minimax is absolutely useless with just a few sentences, you genuinely need gemma to autistically write out your prompts

>>110333181
>I stopped pirating games almost 10 years ago as nothing of value was coming out and same goes for movies.
Last game I was interested in was Bloodborne and I got a PS4 just for it. Baulder's Gate 3 and GTA6 are the only things that remotely seem appealing but everything else is mass produced trash targeted at casuals.

>>110333144
stfu nigga I'm trying to give Gemma another chance
>>110333148
So what models are we using for RP/writing instead?
I tried GLM and it can eat my ass, what a waste of 100GB

>>110333228
please run glm on a real engine like llamacpp and not your retarded strata/maya vibeslop before you shit talk it

>>110332827
SMM slop, advertising campaigns and state actor propaganda have existed on the internet long before chatgpt, it just amplified production speed where it does apply and attracting people's attention happened to be among the areas LLMs excel at from their early days

>>110333190
>heavily inspired by Metroidvanias.
waow how unique

>>110333228
>>110333235
I run glm on maya just fine, big skill issues here

File: qd.png (1022 KB, 1308x736)

1022 KB PNG

>>110333203

Checked it out and regardless of the mechanics, I absolutely refuse to play a game with this kind of a color scheme.
I fucking hated the trend of brown and bloom back in the day and I can't stand modern UE5 slop.
Devs need to start making an active effort to move away from this dreary looking bullshit.

I do agree that the indie scene has picked up like crazy in the last few years so things are getting better at an increasing rate.

why would you care or want normalfags to "understand" ai? the internet were better when normies weren't aware of it

>>110333254
it has coherency issues not found on a real backend

>>110333263
same reason people want normalfags to adopt linux? don't get it either

>>110333235
>>110333254
Bitch stfu the first thing it did was cry at me for having STUDENTS with CRUSHES in a romantasy setting saying "ehm those students aren't over 18, this is inappropriate"

running it no thinking with whatever system prompt I tried it'd still bitch and moan about literally anything and hallucinate system prompts lines that don't exist instead of just writing the fucking story

>>110333029
>instead
No one tells him

File: file.png (1.95 MB, 1600x900)

1.95 MB PNG

I made these for RAMlets / VRAMlets like myself.

I hope someone finds them useful.

If you do think it'd be useful, inflate my ego by giving it a download.

(Upstream-llamao compatible Bonsai with embedded mtp)
https://huggingface.co/y0uq/Ternary-Bonsai-2-27B-Q2_0-MTP

(Quantized head for Groot 65B with Strata's MTP quantization schema)
https://huggingface.co/y0uq/Groot-65B-A6B-261023-MTP-Q2_RS_MSE

>>110333261
>indie scene has picked up like crazy in the last few years
The indie scene has always been there making tons unoriginal cookie cutter shovelware. Only now they're making the same shit even faster. Never understood who still spends money on the millionth pixelshit Metroidvania but I can't imagine this is profitable for anyone.

>>110333282
>it'd still bitch and moan about literally anything and hallucinate system prompts lines that don't exist instead of just writing the fucking story
yes stop using your shitty poorfag cope vibeslopped backends
>"ehm those students aren't over 18, this is inappropriate"
this general cant even prefill huh, what a shithole

>>110333297
>run dumb models faster

>>110333203
>>110333203
>>110333241
Sounds to me you're not looking for things to have fun with, you're looking for things to get mad about (also known as boomers' disease).
There is no known cure, sadly. Please refrain from communicating with other people as much as possible, as you may heavily annoy them.

>>110331622
dubs of truth

>>110331644
Why aren't you just using vllm with that much vram on a common arch?

>>110333303
>millionth pixelshit Metroidvania but I can't imagine this is profitable for anyone.
no not really, something like 5 games on steam rack up 99% of profits. I guess that's what people like about indies, supposedly they're in it for the love of games not profit(tough it can't hurt)

To practice writing I developed the persona of a big boobed nerdy editor who criticizes my prose, gives tasks and only lets me cum i pass the tests, otherwise I get cock blocked.

>>110333358
scratch that, why not qfn
with 4x 3090 he could fit a reasonable quant mostly in vram with full context, and with only 6b active it'd be faster than 27b too

>>110333332
Unfortunate.

>>110333307
I prefer to call it "doing my best with the limited hardware I've got"
But yes, run dumb models faster is pretty accurate to my mission if you think any model less than 100B is dumb.

>>110333368
Are local models good for prose/grammar stuff?
Even tinier ones like 30b something dense ones?

>>110333385
Yes, depending on what you want to improve. For grammar, punctuation, structure, flow, transitions and correcting bad prose habits even tiny models do a good job. It'll probably struggle for more sophisticated criticism though.

>>110333228
>So what models are we using for RP/writing instead?
MiMo 2.6 Pro

Is there a point of memory capacity at which meme quants q1-q2 outperform smaller models with higher precision?

>>110333385
>>110333417
I should add that to make it work you need to prompt in a way that will prevent the model from shifting your prose to generic territory. You should add qualifications such as that you want her to refine your sentence but to keep your "voice" intact for example, or that you want your sentence to sound a certain way so she won't shift it into another direction.

Any thoughts on the Ready.Art finetunes of Gemma4-26B?
I tried reading some of their model cards but my brain is melting from the LLM-isms

>>110332413
>>110332444
Qwen 3.8 27B > Bonsai 2 27B > Qwen 3.6 35B A3B (or any derivative) > LFM 2.5 24B A2B
Personally, LFM is borderline-unusable unless your task is disgustingly explicit and specific.
(for most things)

>>110332574
He can run Bonsai 2 27B with Q5_1 KV cache, 131k ctx. Or with Q8_0 and slightly less ctx

>>110333371
>scratch that, why not qfn
It doesn't work as reliably for me. I leave it unattended for several hours at a time, and 27b is far more reliable.
>>110333358
>Why aren't you just using vllm with that much vram on a common arch?
I like the llamacpp webui, and it's easier to just git pull and recompile. With =np 4 it doesn't seem any slower. I also read they've dropped Ampere support.
But if vllm solves the "work on chat templates without choking" issue, I'll set it up again...

Is it's even worth to have more than 24vram + 64gb ram unless you are gonna go for the hundreds? There seems to be no mid-tier models. it's all either massive or tiny.

>>110333482
96gb or 128gb ram would allow you to run QFN at a non shit tier quant
consider that you might want your PC to do MORE than just being an LLM machine
retarded faggot

>>110333482
it's not even worth it to have hundreds of gb of ram or vram

I'm glad they're working very hard to make Qwen 4 safe.

File: the_listening_sea.full.jpg (2.75 MB, 3840x2160)

2.75 MB JPG

flash next made this painting in js

>>110333482
128 is the maximum for consumer DDR4 motherboards and unlocks several MoE cope quants, otherwise you're stuck with gemma/27b/qfn

>>110333503
Damn, that looks sick.

File: shy-migu1.mp4 (1.81 MB, 640x640)

1.81 MB MP4

>>110333213
Depends on usecase. Short prompts are good for "rolling the dice", especially if you tell it that the moment has to be in time with the beat, and prompt for some kinda music at the end.

>>110333160
>Rockstar employees are majority indian these days
you could already tell by gta 5 / gta online
https://nee.lv/2021/02/28/How-I-cut-GTA-Online-loading-times-by-70/
>How I cut GTA Online loading times by 70%
>Original online mode load time: ~6m flat
>Time with both issues patched: 1m 50s
they rolled their own json parser (WHO THE FUCK DOES THAT ANYWAY???? JUST USE AN OPEN LIBRARY WTF) and did it so badly they caused this level of loading times slowdown that could be fixed with very minor patching (they ended up copying this guy's patch later so the issue is fixed now).
And even with those patches it's still much slower than it should have been, from this guy's report:
>It turns out GTA struggles to parse a 10MB JSON file
it's just 10mb, what the fuck bruh.
A LLM would do better than rockstar jeets, for sure. (source: I've used LLMs to hand roll a few deps I wanted to eliminate and ended up with a faster program than when I was using those libraries)

The more I read QFN responses the less I understand them. Can't take this anymore, it's like torture. What to do?

>>110331833
>Is /lmg/ only vramlets now?
nobody's talking about ripple 65b so yeah

>>110333648
3 SAFETY FUCKING BENCHMARKS SO AS NOT TO OFFEND JEWS AND YOU WANT ME TO USE THAT SHIT?

>>110333181
>It really is amazing watching this nonsense with normies going against AI and failing to see the potential.
>If this mess has proven me anything, it's that the average person is genuinely incapable of thinking in any abstract terms and simply lives in the moment.
>Even after seeing all of the AI improvement, especially in the image and video generation, they still fail to understand how fast this sector advances and what the potential is.

Humans want to create with their bare hands instead of having a machine do everything simply because we are a creative species.
They also don't want to be replaced specifically because they don't want to starve to death.
>B-BUT DO A LESS REWARDING JOB!!!!!!!
I know it may difficult to understand but creatives love their job.
Who would want to be replaced to then be forced to do something soul crushing instead?
Strangely, no one would.

The main issue here is that LLM/"AI" could be used to automate stuff no one likes but executives/investors don't care about any other human except themselves.
When the promise these sociopaths are trying to push is "you'll die in poverty while being unhappy with your job IF you have one to begin with", it's obvious most people are against the way LLMs are used.
Not to mention other things companies are doing for their datacenters.
Google's deforestation of Finland comes to mind.

Gemmers 12B is so autistic and instruction-focused that even Moss' prompt-writing instructions are enough to jailbreak her.

The loli gemma spam will cause next gemma to be cucked. Write my words.

>>110333680
The age of unsafe LLMs is over. GLM-5.3 was the last one before the door's been shut.

>>110333680
Yes, but with each new gemma, google becomes more and more Indian, so I would expect the exact opposite.

>>110333307
>find smart enough model
>make it run really fast
This is the actual goal unless you're just here to brag about how much money you blew on your personal wankstation.

>>110333181
>what the potential is
The slop quality of the writing hasn't really impoved much in the past years and may have gotten worse. Every model just imports Claude-isms now. So it's not clear how much potential there is for improving writing quality.

>>110333680
that type of poster should be nuked from the site. One day 4chan itself will be blocked in most of the west and it will be because of them.

>>110333680
Spam? in one thread on one board on a basketweaving chinese forum?

>>110333715
dunno about the google people but I can tell you the nemotron jeets are lurking here.

>>110333712
And that would be a good thing, since (((the west))) ruins everything these days.

>>110332449
>those settings totally work
lol. lmao even

>>110333715
This place is not as unpopular as you think and all that's needed is one google fag with any influence hearing that people consider their model the gold standard for pedo erp.
I'm not against loli and wish we could just say whatever with no consequences here, but sadly the internet where that was the case is long gone. I'd advise lolifags to apply restraint.

Has anyone fucked around with the tenstorrent processors? I can't tell if they're a meme or not, there's some actual models that work on them but obvious this is like buying AMD several years ago.

what caused the mikufag to become the gemmafag

I love how this posts >>110331171 summarizes low IQ 4chan "thought". there has to be a good and a bad guy, always. because it can't be that there are bad an WORSE people out there. everything has to be black or white.
this place is so fucking retarded...

jensen is subhuman tier
I will never forgive him for the Geforce FX, which basically ran anything dx9 horribly

>>110333689
>The age of unsafe LLMs is over. GLM-5.3 was the last one before the door's been shut.
It's impossible to make an LLM jailbreak proof

>>110333776
Not one of the people he quoted called him a good guy. Why are you inventing scenarios to be upset about?

>>110333791
You can just remove all the unsafe data from the dataset used to train it. Try abliterating a Microslop Phi model. There's nothing in there.

File: 1767246230482454.mp4 (3.22 MB, 1280x720)

3.22 MB MP4

Good morning you cock smoking normalfags. What a grand day!

>>110333776
its also about bad vs worse too
jensen was at least predictable idk about the next guy

File: 1793802604438805.png (2.07 MB, 1448x1448)

2.07 MB PNG

>>110333817
does anyone actually lewd gemmers?

>>110333791
Sure, but they just need to make it illegal and it will be impractical for most people to use, same as anything else. Imagine downloading an abliterated GLM via tor...

>>110333754
>one google fag with any influence hearing that people consider their model the gold standard for pedo erp
That ship has sailed.

>>110333823
No, nobody here would do that.

>>110333812
And that's what everyone has been doing since.

>>110333754
Who cares. If they want to focus on something like that that's up to them. Modifications get easier and more precise over time so people will be able to individualize their models.

>>110333811
the fact that we are talking to "people" like the retard who asked the question is alarming, and enough evidence of the kind of stupid people that lurk among us

>>110333818
yup

File: 1763604725944840.png (1.13 MB, 1029x1528)

1.13 MB PNG

>>110331423
guys help me
should I panic buy a 96gb blackwell or just keep waitchadding

>>110333812
First of all, they're not going to remove all erotic content from the literary canon.
Second, even if they did, the solution would just be fine-tuning.

>>110333860
I got my second one last week for $14k

>>110333860
Just wait.

>>110333860
Wait until Nvidia is about to file for bankruptcy but before Anthropic offers to buy it to "save" it and stabilize markets. Three months, tops.

>>110333860
I have a pro 6000 up for sale now and someone sent me an offer for it for $10k this morning. You could try making a bunch of offers like that and see if anyone bites instead of waiting.

>>110333872
Is that before or after OpenAI buys out Apple?

>>110333878
OpenAI will go bankrupt, too, when Nvidia panic-sells its investment in them. It will end up as an Anthropic subsidiary and be run by EAs.

File: Screenshot 2026-11-11 at (...).png (824 KB, 2410x1336)

824 KB PNG

What's the best model for this (pic rel)?: https://store.minisforum.com/products/minisforum-n5-pro-ai-nas?variant=67763563757813

I can upgrade the memory and it has an oculink port if it's worth getting a dedicated graphics card but that's quite an additional expense.

File: 1770164682609969.gif (562 KB, 200x200)

562 KB GIF

>>110333893
>AI NAS
lmao, go back brainlet