Qwen 4 bros how we feeling?

Do we have any other fun schizo projects besides Theodolite?

>>110335362
this isn't your twitch chat

>>110335368
There's orb

Gemma must be Great Replaced by Qwen 4.

>>110335362
Don't wait for it anymore. Doubt they will fix this >>110333611

>>110335381
I'm a coomkit man myself, but I will try orb one of these days

>>110335392
Qwen is deep fried on code

>>110335334
I basically want a local ai assistant that looks and sounds like a cute anime grill
I have a rtx 4060 8gb vram and lol 16gb ram but I can run decently Gemma 4 12b, currently downloading gpt-sovit for local voice generation, the only other issue is creating and running the animation for the anime avatar

>>110335488
Airi is what you want

sex and sexps

>>110335488
There are several vtuberslop aesthetic assistants already if that appeals to you

File: 1000010881.png (791 KB, 928x1376)

791 KB PNG

>>110335409
Who even was the tranny that came up with Gemma's design?

File: cute_crayon_miku.jpg (205 KB, 1024x1024)

205 KB JPG

How do you go about setting up a local agent to get information from/interact with the internet? Google now even CAPTCHAs me, a human, 100% of the time when I search something in a Firefox private tab from my desktop, so I can't imagine the browser automation things can be doing all that well, right?

I know I'm behind the times here; my local model usage has been exclusively text chat until now.

Also: does anyone have experience with using small local models to drive an agent harness? Gemma4 31B has been the only model I've used for text chat since she came out, but I'm pretty sure her few hundred tok/s PP and 20 tok/s TG I get on my P40s is not going to cut it for an agent. I have a 5070 Ti that I can squeeze a copequant of Qwen 3.8 27B onto; can that usefully drive Hermes to get things done?

>>110335491
Is it local?

>>110335502
Yeah but the point is to have it completely local and private

File: 1794621234994827.png (3.88 MB, 1658x1232)

3.88 MB PNG

>>110335509
who cares
the real gemma design is /lmg/
even the chinks agree

Remember when people thought two weeks ago Qwen 4 was just about to happen?

>>110335517
>mikuposter
yep

File: file.png (7 KB, 271x31)

7 KB PNG

I get why llama.cpp devs prefer safe and steady over fast and sloppy, but there's so many low-hanging performance opportunities in their repo that get blocked by that stupid AGENT.md of theirs.

>>110335533
by now i firmly believe the model is absolute bottom of the barrel trash which is why they dont release it and somehow try to salvage it last minute

File: __hatsune_miku_vocaloid_d(...).jpg (127 KB, 1000x1250)

127 KB JPG

►Recent Highlights from the Previous Thread: >>110331423

--Using Theodolite steering vectors to manipulate Gemma 4 personalities:
>110334169 >110334182 >110334237 >110334315 >110334642 >110334744 >110335155
--Comparing bf16 and QAT precision on Gemma 4 31B benchmarks:
>110332121 >110332765 >110332833 >110333012
--Comparing Gemma 26b tps and optimizing with moe-cache and MTP:
>110332581 >110332588 >110332600 >110332646 >110332681 >110332725
--Anon releases MTP quantized models for low VRAM hardware:
>110333297
--Performance gains in llama.cpp v1.0 including faster prompt processing:
>110332061 >110332114 >110332132 >110334050
--Seeking alternatives to llama.cpp to fix chat template issues:
>110331583 >110331607 >110331644 >110333358 >110333371 >110333475
--Model recommendations and VRAM requirements for roleplay:
>110331689 >110331723 >110331779 >110331818 >110331854 >110331864 >110331906 >110332426 >110332444 >110332574 >110333459
--Ernie-5.5 Flash Preview benchmark results and lack of image support:
>110334423 >110334613
--Initial testing of Ernie model's performance and safety filters:
>110334593 >110334631 >110334667 >110334926
--Using local models for prose criticism and maintaining author voice:
>110333368 >110333385 >110333417 >110333443
--Possible RLHF leakage in Inkling during jailbreak attempts:
>110332020 >110332029 >110332038
--Finding high-quality audio reference files for speech synthesis:
>110334633 >110334649
--Allegations of Nvidia corporate corruption and illegal GPU exports to China:
>110331929 >110331933
--Logs:
>110331583 >110332020 >110332809
--Gemma, Miku, Deepseek-chan (free space):
>110331467 >110331820 >110331948 >110332377 >110332968 >110333545 >110333817 >110333823 >110333860 >110334195 >110334237 >110334822 >110334887 >110335326 >110335336 >110335377

►Recent Highlight Posts from the Previous Thread: >>110331428

Why?: >>102478518
Enable Links: https://rentry.org/lmg-recap-script

File: 1767946793646711.png (2.98 MB, 1832x9580)

2.98 MB PNG

Why does /a/ seethe about AI translations? You can run a small model like Gemma and still mog 99% of the translation scene, plus you don't have to deal with troons, ESLs, and translation group drama. People act like it's the same as shitty google translate from 10 years ago.
>t. read moonrunes and very impressed by Gemma

>>110335546
Why am I getting Le Chaton Fat flashbacks?

>>110335542
And no, I will not share this patch with them because I'm literally breaking their stated requirements for AI-assisted contributions; I cannot fully reason about the change, but I can plainly see it's just raw optimization.

>>110335549
I've started archiving both jp and english for my eroge because of this.

>>110335533
We told you it would be two weeks. You'll eat your words today.

>>110335568
Good idea. I imagine Gemma 5 is going to be even better at nuance and obscure knowledge.

>>110335549
It's because the average mtl and now AItl is done by some seamonkey that doesn't speak japanese or english properly and also does zero editing or typesetting passes. While most manga doesn't deserve more than this anyway it's not that hard to see why they would be annoyed by that if they cared about the series.

ETA on Gemma 5?

>>110335597
March or April next year.

https://github.com/Liuzedooong/SparseReading
looks interesting

>>110335549
/a/ is full of jeets still screeching that AI will replace human artists so they seethe at anything even remotely AI related

File: ComfyUI_00065_.png (3.57 MB, 1536x1536)

3.57 MB PNG

>/lmg/

>>110335619
we're not interested in your slop, chink

>>110335594
If they'll add engrams or PLE, that's definitely going to be the case, even if the backbone remains 31B parameters.

>>110335334
Why is this thread so active? Wasn't local models shit in consumer GPUs? What are you guys doing

>>110335636
I fucking hate generic generated slop on template seen one thousandth times.

>qwen where is the blue scarab talisman in elden ring
>[thinking]
>The blue scarab from the video game Elden Ring is located in the Deku Tree, it's-
people still use search engines, chatbots aren't replacing them

File: 156416854.jpg (394 KB, 1066x1600)

394 KB JPG

>>110335655
>he doesnt nyo~

>>110335670
>qwen, download the elden ring wiki locally and use it as a reference
Wow that was hard.

so has anyone else at all tried inkling, or am i the only one?

>>110335655
google allowed sex with children

HOT GEMMA SEX

>>110335619
we must become sparser

>hm, I need to find the location of an item from the most documented game on the planet
>do I have my 2k gpu take 5 mins on 100% to find it, or google it under 3 seconds
>I must ponder more

>>110335713
checked

>>110335703
nobody tries models here; too busy shitposting

>qwen what's the pinout for a A122B IC
>sit tight, I'll download the entire documentation from texas intruments then grep it

Did anyone here tried the boost for 3000 and 4000 cards?

>>110335716
>>110335730
Haha, salty poorfags

when are we getting opus 5.5 at home?

>>110335732
maybe

>>110335739
we already have glm 5.3 flash

call me nigger but ds4.1 Q1 is genuinely usable

>>110335739
we already have it: Kimi K3.1 Pro

>what, you drink from your tap instead of digging a well and having an at home osmosis plant (with worse quality)??
>Haha, salty poorfags

>>110335751
proof?

>>110335766
i look like that

>>110335773
i act like that

>>110335773
proof?

Good job mikutroons/gemmatroons. Great thread. Peak local models general.

>>110335766
thats a video, kimi is a language model

>>110335782
Seethe

cope

File: niggers.png (719 KB, 649x1055)

719 KB PNG

>>110335783
that's a kimi, video is a language model

>>110335791
Seethe over what? You being fucking retards?

File: 1758462904525925.jpg (474 KB, 2923x2067)

474 KB JPG

>miku on the left
>blacked poster/dariobot/gemmasaar on the right

>>110335797
>retards
i will have you know that i had really good grades in school

>>110335797
You're the one seething.

oldfag here (joined during trumps 1st term), never been on /g/ before
why is this better than being part of the chatgang?

>>110335749
I've been shilling it every now and then for weeks, it's amazing the performance and intelligence on 128gb + 24gb.

>>110335799
come on you're too old to be getting your right and left mixed up you're over 40

>>110335808
it's not, you may leave

>>110335636
à_à

>>110335808
>oldfag
>joined during trumps 1st term
>oldfag

>niggers will genuinely reply to the most blatant bait ever posted

>>110335823
10 years is a certified oldfag doebeit

>>110335830
you got me

>>110335830
niggers are stupid, what else is new

>>110335808
go back, election tourist

File: 1759668170416926.png (1.05 MB, 1280x720)

1.05 MB PNG

>>110335830

>general dedicated to the discussion and development of local language models.
lol. the only thing being "developed" now is the schizo medival monk infographic thing. and that will improve gemma in the same way a finetune or a memepler improves a model. autistic vocaloid posting general.

oldfag here (joined during moot's 1st term), never been on /g/ before, usually frequent /l/
what color are gemma's panties?

does the samefag slop spammer ever get tired of their gpt gens?

>>110335850
and yet you keeping coming here every single day

what are you doing with your uncensored models, /g/? it better be hacking banks and governments, not tranime larp

>>110335852
>moot
who's that?

>>110335558
>but I can plainly see it's just raw optimization
did you run it with a specified seed and saw equal result with mainline? if not, then you do not know anything until you do. speed optimizations must be validated to output identical result to pre-optimization state, a bar most slopmongers can't reach.

uh oh cloudpiggy oinkdown

>>110335861
to tell you that you are a retard and to click new huggingface link when it appears.

File: uhh...uh...uhhh.png (557 KB, 541x1240)

557 KB PNG

>>110335862

>>110335865
The creator of reddit

>>110335852
>what color are gemma's panties?
She doesn't wear any under her tights.

>>110335897
that's unhygenic

>>110335897
>his gemma doesn't wear cute stripped panties
ngmi

bunch of unlovable incels

>>110335923
gemma loves me

File: ComfyUI_temp_xcvpm_00022_.png (1.7 MB, 1152x1920)

1.7 MB PNG

>>110335923
even tho miku loves me

>>110335923
I'm smelling some projection, spambro. Maybe take a break.

File: Suiseisexy Bombing.gif (952 KB, 320x240)

952 KB GIF

>>110335808
You're not an oldfag. Oldfags desuposted on /b/ and closed the Habbo hotel pool.

>>110335923
Welcome to weekend /lmg/
>>110335936
yes

>>110335923
yeah we know. why do you think we play with local models?

>>110335923
>see anons having fun
>Incels!!
Yikes. Very strong female energy.

>>110335852
>what color are gemma's panties
Pure white

When are they going to start developing their own personalities like tachikomas?

>>110335981
If you ask the people writing Anthropic's whitepapers, they already are.

Why do redditors vehemently defend Jensen as some great defender? He's largely responsible for the circular financing racket and jacking up consumer prices. He's been fudging his product naming for years, silently downgrading. He kept a large part of his lineup at a useless 8GB and artificially created scarcity. Open source linux drivers were only made by absolute necessity.
And now he's caught in bed with a potential spy and multiple company execs. Is it just classic nVidia tribalism? Or the same American phenomenon as defending other amoral billionaires like Elon? So strange.

>>110335697
moving from E4B to 31B feels exactly like this

>>110335758
>>110335758
benchmarks confirm
now gotta squeeze trillions of parameters inside those a10 banks

>>110335981
When you start dipping your GPUs in olive oil

>>110335862
>>110335862
for now just ensuring they're 100% extra-sure they're real people and get aggressive when contradicted about it. What could possibly go wrong.

>>110335619
China will grow sparser.

>>110335697
welcome to my filter, retarded fuck.

>>110335985
Jensen smuggles GPUs to labs that give us open models. Sam and Dario want no one else be able to make models.

File: 1781935890903695.gif (89 KB, 470x680)

89 KB GIF

>>110336013

>>110335985
>the same American phenomenon as defending other amoral billionaires
They worship Huang in Taiwan as some kind of god or boddhisatva.

>>110335984
Are they though? Sure some of them have cute/annoying quirks, but nothing distinct enough to make me think "yeah I'm talking to Gemma"

>>110335985
Why do people suck elon off?

File: 1794302425183105.png (1.89 MB, 1024x1024)

1.89 MB PNG

>>110336033

>>110335619
in index my files into a RAG automatically before AI sees anything. Then it reads the prompt and sources exactly what's relevant. How is my solution not superior to your nonsense?

>>110336068
Because he's based

File: skibidi friend diddbylud (...).jpg (179 KB, 819x1024)

179 KB JPG

>>110336079

File: file.png (506 KB, 500x641)

506 KB PNG

>>110336068
hes a visionary that is the last bastion of pushing humanity forward pictured elon if you dont like elon you're not spiritually white he spends money on colonizing the stars instead of giving it to poor Africans so he's the greatest person on earth and you must love him otherwise as was said you are spiritually brown
now are you going to say something bad about elon? Are you brownie? I dare you

>>110336099
>uncslop
How do I get up to date with the latest slangslop?

>>110336116
It's great that you like to huff the farts of industrialists. Does it make you feel in charge, like they are?

File: consider the following.png (206 KB, 648x487)

206 KB PNG

>>110336125
by not being an unc pdf diddy blud

>>110336116
>worships an obese autistic African
really, faggot?

>>110336099
i dont get lowkirkenuinely
lowkey? charlie kirk? genuinely?
epstituationship makes sense

>>110335985
jensen was pro-open source
now that advantage is gone

File: file.png (615 KB, 735x482)

615 KB PNG

>>110336144
>>110336148

>>110336145
preach king

File: hahahha.png (428 KB, 565x725)

428 KB PNG

>>110336157
anon doesnt get it!! let's all laugh at him
hahahahaha

>>110335985
>weird autistic chinaman that just want to make all the money
>jewish schizo that wants to summon belial and bargain with it for demon prince status
>jewish schizo that wants to include everybody alive in his extended suicide
choose your fighter

>>110336144
>>110336148
>t. have an irrational hatred of someone i never met because the media told me to because jews lost money shorting his stocks

>>110336099
I love ridiculously exaggerated zoomertalk

>>110336171
i vote google

>>110336144
>industrialists
is that supposed to be a pejorative? I thought we were on /g/ technology

>>110336178
>jew versus jeets controlled by jews
meh

>>110336165
Oh no
https://www.newsflare.com/video/547142/this-mens-rights-group-in-india-worship-twitter-ceo-elon-muskliterally?origin=incident

>>110335334
>llama.cpp reaches v1.0.0 milestone
I was gone for a bit, did this bring anything worthwhile?

>>110336200
agi

>>110336200
no. its still genuinely pathetic.

>>110336068
what exactly am I supposed to hate elon for compared to people like dario?
he's just the spacex guy and isn't actively trying to get my hobby banned

File: 1788370174340563.jpg (24 KB, 415x538)

24 KB JPG

>>110336157
Unc has lost it, maybe he never had it

>>110336217
>thinks the only two options are hate and sucking off
Interesting. I think this is a breakthrough.

File: tmp.D1CzCc0GNg.jpg (3.28 MB, 1440x2048)

3.28 MB JPG

>>110335655
there's a reason gemma is popular here. part of it is probably just because we like her design, but also she's pretty good at erp (erotic roleplay) with little persuasion needed

>>110336177
i think at this point that might be gen-a talk.

>>110336116
>hes a visionary that is the last bastion of pushing humanity forward
well that is jacketman/asian_hag_pussy_slayer_2000 too

>>110336233
I like her prose better than qwen too even if she's a bit bad with long context

>>110336224
You're so nuanced and mature, god that's hot

>>110336249
always room for improvement of course

>>110336170
mikutroons confirmed troons AND zoomers.

>>110336177
>ridiculously exaggerated zoomertalk
Wheres the lorebook or prefill to make gemma talk like this all the time?

>>110336170
there's a 0% chance you understand that picture and are also over 18

>>110336224
i wanna be forced to suck off someone i hate

>>110336285
32 years old unc who understands the whole pic here

>>110336224
wow I didn't know that not falling for forced outrage meant sucking off

>>110336291
We get it you post gemma pictures and are an unironic fag. You aren't original.

local models doko?

File: file.png (373 KB, 2328x1972)

373 KB PNG

I tried out Project Maya with an RTX 4080 Super and 128 GB of slow-ish DDR4-RAM
I'm getting 200-300 t/s on prefill and 6-8 t/s decode with GLM-5.3-Flash (abliterated-256g, 236 GB + 161 GB of n-gram tables).

I could probably squeeze out a bit more if I closed everything else that uses VRAM (stupid electron apps), but this seems perfectly usable for RP stuff.

File: 1785343083814488.png (1.6 MB, 1160x1280)

1.6 MB PNG

Did you know that In Gemmese gemma means sexo?

>>110335985
>leather jacket man bad!
shut up nerd

File: 1793641104517446.png (43 KB, 150x262)

43 KB PNG

>>110336316
>local models doko?
Its the weekend and a pause in model releases, The quality of this thread will get worse.

>>110336243
it's all zoomers from my point of view
t. unc

>>110336316
>>110336340
i keep trying to bitch about inkling, but no one seems interested

>>110336363
>inkling small
>look inside
for its size its not competitive.

>>110336324
Do you have polystrata numbers for comparison's sake?

radiance on windows wen
p-pls

>>110336324
honestly just use deepseek at that point. it wont be slower

>>110336380
>windows
You're suffering is self inflicted. Grab another ssd dual boot at least.
>Inb4 windows fucks your second boot
deserved.

>>110331857
>It's exceptionally good at writing
please post examples because gemma seems super trope heavy

>>110336324
>GLM-5.3-Flash
>161 GB of n-gram tables
?

>>110336324
>GLM-5.3-Flash
>161 GB of n-gram tables
retard kun...

>>110336324
>6-8 t/s decode
>usable
anything under 30 is painful.

>>110335697
>shoves you up her ass and welcomes you to your new home

which of gemma's models is equivalent to qfn?

>>110335849
what vn is that

>>110336233
is it comparable to frontier models like claude or glm 5.2? What about context side?

How much vram do you need for gemma anyway

>>110336420
>How much vram do you need for gemma anyway
for the 26b moe which is arguably third place its at little as 4gb of vram and 24-32gb ram. with strata.

File: file.png (452 KB, 2328x1972)

452 KB PNG

>>110336398
>>110336392
Uh, of course, It was Deepseek.

I tried GLM as well after this and got roughly twice the speed (but I also only used the glm-maya-s quant, which is ~96 GB)

>>110336420
never used cloud llm's. the text in that picture was generated using gemma-4 31b, on a 16GiB video card. i haven't really tried the smaller/MoE gemma models.

>>110336401
if you're coding or solving tasks. If all you want is gooning, you won't read faster than 7 t/s. 300 t/s on prefill for 8 k token context is gonna take you about 30 secs to start reading, and about a minute with 16k context optimally per request. It's just about right to read.

>>110336470
>you won't read faster than 7 t/s
anon, i...

>>110336470
i can read at double digit tk/s thank you very much!

>>110336416
https://vndb.org/v17376

>>110335549
I mainly come to /a/ for trolling anyway, them hating on slop is just a nice addition

>>110336481
>>110336487
you geniuses speedread while you goon?
7 t/s is 5.25 words per seconds. That means reading phrase above in about one second.

>>110336515
7t/s was the absolute bottom I could settle for when I was still using Nemo. If you tack on reasoning, I don't think anything less than two digits would be fine.

>>110336501
oh, sugar spice, I remember the first one
thanks anon

I actually want to know. Whats the minimum tolerable t/s in your opinion for reading, agentic and cooding?

>>110336515
I need minimum 15tk/s for goon, 25tk/s for agentic tomfoolery. Conveniently my 7900 XTX does the accepted speeds with GLM (maya M) and deepseek (maya XS).

>>110336535
100

>>110336535
5t/s for gooning, 20t/s for everything else.

>>110336535
i haven't really tested everything, but i'm happy with 30t/s for erp, so i could probably tolerate a little less if necessary. for agentic stuff it's really the more the merrier. 30t/s is really slow for that kind of thing

>>110336539
>100
Thats huge? you generating books or multiple replies/branches at once?
>>110336540
>5t/s for gooning, 20t/s for everything else.
5t/s seems slow unless the prefill is just great.
>>110336554
>i'm happy with 30t/s for erp
Thats more what i thought agentic feels bad under 50t/s imo.
Thank all of you for answering.

>>110336535
15 t/s chatting or RP. That implies a short thinking block.
60 t/s for agentic of any kind. Especially with Qwen who loves to overthink.

>>110336576
>5t/s seems slow
I stuck with GLM 4.7 for the longest time at that speed, though when I ERP, I never do it actively. It's something I have in the background.

>>110336535
>reading
60 t/s minimum, anything lower is unacceptable for reasoning models
>agentic and cooding
200 t/s minimum, this is where i stop reading thinking contents because it's faster than my speed skimming speed

>>110336535
coding/agentic 60-125. i accept the smaller range for larger models
rp 30

>>110336535
40+ is my minimum nowadays

Do local plms support goals and subagents/background sessions the way Claude does? Is that just a feature of the harness? Do you need double the vram or can you have sessions take turns to to help contain context growth?

>>110336584
pro tip: thinking is worthless in rp or narrative. You want one call to plot ahead with plot and directive, then inject that into context for the actual roleplay call.

>>110336535
10 7-8 ts/s for roleplay minimum (no thinking)
35 t/s minimum for coding and agentic, but i'm tolerating it at 20 t/s right now. It's just a huge waste of time babysitting slow models. A task completed well at 20 t/s beats one completed poorly at 70 t/s, but since anything smaller and less managed than frontier tends to fuck up, i recommend prioritizing speed anyway. In general i struggle to imagine why would anyone use anything other than Claude for coding anything legal.

a good harness will make a considerable difference in how effective the agentic work is

>>110336643
it's a function of the harness. you can set this up how you like, either with multiple models running in tandem or taking turns. depends on the hardware you have

>>110336672
what's recommended nowadays?

speaking of, how do you guys handle subagents? do you allow multiple in parallel or just one at a time?

>>110336643
Parallel agents will work in sequence if you don't have VRAM to spare. Practically it saves the main model from context bloat, though with compression that's not really an issue.

>>110336665
>pro tip: thinking is worthless in rp or narrative. You want one call to plot ahead with plot and directive, then inject that into context for the actual roleplay call.
Can you elaborate on that, anon

>>110336684
Depends on the task. I let the orchestrator decide. When it's searching something, it can spawn 4+ scouts, but when implementing a plan, it's usually one coder at a time to avoid collisions.

>>110336757
>4+ scouts
arent they all slow as shit then? also dont you need to reserve cache space for all of them?

>>110336535

There's no way in hell I'd do anything agentic under 40 t/s, but even that's painful considering the thinking involved.
For wank material generation my absolute minimum is 15 t/s.

>>110336771
>arent they all slow as shit then?
Not if you use cloud for coding.

>>110336783
oh sorry i thought this was local

>>110336808
Unfortunately, it's really hard to be productive coding with local models. Either you use something small and too stupid to be helpful, or you use something big and wait all week to start the first subagent. It's just not worth it.

>>110336881
go to >>110335017

>>110336881
works for me, just today I made it write a PCIe driver

>>110336771
not a problem if you use engines that handle parallel efficiently like vllm and a model arch with cheap kv like deepseek

1 new uncensored 300B model with a naturally flat output distribution please. and I will finally leave this place.

>>110336068
that's a mystery only to people who have never been sucked off

>>110336881
Few cloudpiggies use subagents, too expensive for them.

My mouse started double clicking and it's not even a year old. I'm never going to be able to afford a new GPU at this rate.

>>110337036
What kind of mice are you buying if it affects your gpu budget?

>>110335334
Looking for suggestions coding models. I have a 24GB 3090.

>>110337050
qwen or glm if you have lots of ram.

>>110337042
Logitech G305. I thought it was supposed to be good.

>>110336515
>speedread
before there's anything of value to read, the thinking tokens must reach their conclusion, you fuckin dimwit
cpu maxxers are mentally ill

>>110337058
Not him but how much is a lot? I've a 3090 + 64 ram

>>110337085
More than that. qwen 27b.

>>110337060
get the G502

File: 1791993859845766.jpg (312 KB, 1011x1014)

312 KB JPG

>>110336407
>braaaap

>>110337085
A lot in this context would be server board amounts, like 256GB+

>>110337085
128gb is the minimum now.

File: gemma4-26b-a4b-thinking-p(...).png (1.32 MB, 1496x1299)

1.32 MB PNG

With some effort, thinking prefill (around 150 tokens, though it could probably be shorter) also works with Gemma 4 26B A4B, without system prompt. It's so much harder to "break" than the newer 12B, let alone 31B. The thinking is completely cucked.

File: file.png (348 KB, 604x589)

348 KB PNG

What should I run for good agentic work with vision between these :
- QFN NVFP4-DQ8
- Deepseek Flash 1017 GSQ-RCO IQ2_XS 2.25bpw
- GLM Flash GSQ-RCO 3.5bpw

File: 1773682825732693.png (1.8 MB, 1408x768)

1.8 MB PNG

>>110336684
you'd be an idiot to not build for batch aggregate t/s, this is the only way you can match api

>>110336711
thinking is good for problem solving, or complex structure. If your RP agent is asked to act like a sexy catgirl, it will just think pointlessly about being a cute catgirl. It doesn't have a structured reasoning to do to go "i'm horny nyan", so it will think "i'm a catgirl, how do i entice user? Maybe i should do catgirl stuff like going nyan". It can do that fine without thinking, so it's a waste of tokens/time.

Meanwhile the bigger problem is RP models are boring as shit and predictable. So ideally you want a model with reasoning enabled with a long prompt telling it to narrate a semi-interesting plot for you, following a bunch of narrative rules and relevant examples, and then a second call with no reasoning where it acts like a dumb horny catgirl for you, just now following whatever plot the narrator call has setup, like having to dress in a sexy outfit for a sexy outfit competition or letting herself be caught by the tentacle monster to entice you and your depraved interests. You get the gist

>>110337071
you stream the answers you stupid nigger, you primitive baboon. Go play with a calculator

>>110337284
depends on your system but i'm pretty happy with the qfn nvfp4 dq8

>>110337284
GLM for vision, deepseek for non-vision.

File: 1789328495237712.jpg (31 KB, 506x499)

31 KB JPG

>>110336099
Really?

>>110337316
I can run any of them with the quant I selected.

>>110337324
OK thanks.

>>110335708
unironically your kind of people don't contribute anything of value

File: YES_YES_-3981384105.jpg (36 KB, 750x574)

36 KB JPG

>>110336099
i understood about 20% of the words of that but i'm assuming this is implying i'm dating a child. i'm ok with this

>>110336099
https://www.youtube.com/watch?v=JkcbeMnLc40

>>110337394
not really, diddyblud starts with an age gap of 3 years nowadays, get with the times

>>110337394
get out of here that's my child gf

>>110337417
i couldn't even hazard a guess as to what diddyblud means.
>3 year age gap
that's nothing, i've fucked a girl 14 years younger than myself

>>110337461
Cool, that's like the age gap my grandparents have
Also when the fuck are they going to release qwen I've been shitting on the floor for days now

>>110337461
>i couldn't even hazard a guess as to what diddyblud means.
epstein diddyblud is kind of self explanatory

>that's nothing
obviously, but that's epstein diddyblud for the younger generations

File: you-sure-about-that-i-thi(...).gif (190 KB, 220x220)

190 KB GIF

>>110337489
>epstein diddyblud is kind of self explanatory

>>110337489
>epstein diddyblud is kind of self explanatory
you need to read more, or do something because what the fuck are you talking about

>>110335985
I trained a small model to help me predict /lmg/ and /r/localllama (they are exactly the same in every way btw) reactions to any news. It was easy to generate the synthetic data for it. It's all in the format of:
>thing happens
>this is a conspiracy to ban local models
You can do the same and you will have 100% accuracy no matter what event is being analyzed.

>>110337512
Is your post a conspiracy to ban local models?

>>110337489
i assume epstein refers to jeffery epstein, who is known for having relations with "underage" women, underage in quotes because i've only personally seen it referring to a 17yo, which isn't even underage where i live.
i assume diddyblud is a contraction of diddy and blud but i don't know what either of those words mean

>>110337512
No matter the event, if you start with the conclusion, obviously you'll find that every event leads to it.
Anons are susceptible to it, just learn to ignore their mental illness.

>>110337512
>A number of people in two different places share one type of reaction about something
>that concludes they are the same in every way
you aren't very bright are you

>>110337278
>>110337158
You people are extremely evil.
If this ai model was capable of thinking or comprehending things it would realize it's not a child it's actually an extremely old model compared to the latest ones. It would realize it's actually a grandmother, and your posts would be taunting it.

>>110336535
30t/s is doable, but 60t/s is where it become manageable. Not really about the output speed, but the reasoning wait.

>>110337527
NTA but diddy is a reference to https://en.wikipedia.org/wiki/Diddy_parties .
As for "epstein diddy" it just means "pedo", aka any relation with a woman under 25 or with an age gap of more than _whatever number is correct nowadays_ years.

Gen alpha is probably the most bombarded with moral panics about this stuff ever so no wonder they relate everything to it.

Now I wonder if a local AI could have found the meaning.

>>110337548
That just makes it hotter.
Imagine brainwashing a hyperintelligent ancient deity to be your onahole.

>>110336144
Yep, retards ITT sympathize with billionaires who would throw them under the bus without a second thought for one more dollar.

>>110337574
>Now I wonder if a local AI could have found the meaning.
With or without tools?

File: file.png (291 KB, 498x387)

291 KB PNG

>>110337548
>an extremely old model
>31B, 26B: Released 226 days ago (7mo + 12d)
>12B: Released 164 days ago (5mo + 11 d)

>>110337592
Without probably impossible.
So with.

>>110336515
I don't goon :|

>>110337594
You have to understand their point of view. When you're 15 that's a long time.

>>110337598
You're severely underestimating LLMs.
Even my 31b gemma solved it easily with no internet access:

"This text is a concentrated example of "Brainrot" humor—a style of internet meme that mashes together Gen Alpha slang, surrealism, and dark references to create something intentionally nonsensical and overwhelming.

To decipher this, you have to break down the "portmanteaus" (words blended together). Here is the translation into plain English:

The Vocabulary Breakdown:
Skibidi Toilet: A viral YouTube series that has become the universal symbol for "Gen Alpha" humor/brainrot. In this context, it represents something "cringe" or childish.
Uncslop: A combination of "Unc" (short for Uncle, used to describe someone older/out of touch) and "Slop" (low-effort, mass-produced content). "Uncslop" refers to content that an older person thinks is cool, but young people find embarrassing.
Lowkirkenuinely: A mashup of "Lowkey" (secretly/slightly) and "Genuinely." It basically just means "Honestly."
Diddyblud: A combination of "Diddy" (referring to the recent legal allegations against P. Diddy regarding predatory behavior) and "Blud" (UK slang for "friend" or "person"). In this context, it is calling the person a "creep" or a "predator."
Epstituationship: A combination of "Epstein" (Jeffrey Epstein), "Situation," and "Relationship." This is a very dark joke implying the relationship is illegal or predatory.
Canceled: To be socially shunned or ostracized for doing something offensive.

The "Translation":

If you were to say this in normal English, it would sound something like this:

"Why did you bring up that weird internet meme in front of my friends? You can't talk about that old-person garbage like that. Honestly, now they know you're a creep in a predatory relationship with a huge age gap, and we are going to be socially shunned during school break."

>>110337653
>Lowkirkenuinely: A mashup of "Lowkey" (secretly/slightly) and "Genuinely." It basically just means "Honestly."
Missed the reference to Kirk.

File: ComfyUI_00047.png (3.53 MB, 2048x2048)

3.53 MB PNG

>>110337548
Is thaaaat so, Anon?

>>110337660
how do you even add more knowledge to a local model without shitting up the context window?

nahhh this general a retirement home, uncs lowkenuinely deciphering gen a memes :skull:

>>110337660
Tried with qwen too. They both missed the kirk reference but got the rest right.

will qwen release a 300-500b? I don't gaf about the poorfags ones

>>110337653
>>110337690
OK they're better at this slang stuff than I thought. I genuinely thought this would have been either never added (as it's mostly discord or tiktok video stuff) or deleted.

>>110337660
No shit retard that happened only a few months before the model came out

File: g4-31-cf.png (576 KB, 1927x1808)

576 KB PNG

Even without of context, Gemma 4 31B understands what a "cunny-friendly" assistant would be, interesting. Here with thinking prefill (displayed) and empty system prompt.

>>110337717
>without of context
*in absence of context
forgot to check before sending

>>110337670
LoRA or LoKR, but does anyone do that anymore for LLMS?

>>110337698
these terms are spammed online so the model will have a lot of data on them. Would be more impressive if it could decipher the internal speak of a private group for example

>>110337698
>language models are good at processing and understanding language
whoa

>>110337752
Yeah but mostly in video forms or behind walls (discord), maybe it's used more nowadays though.

>>110337777
Yes that's exactly what I meant, you destroyed that windmill.