File: tetomybeloved.png (80 KB, 970x1075)

80 KB PNG

►Recent Highlights from the Previous Thread: >>109638675

--Apple releases M5 Mac Studio with up to 512GB memory:
>109642927 >109642986 >109642987 >109643011
--Developing and debugging CoomKit for multimodal adult roleplay:
>109638701 >109638706 >109639425 >109639359 >109639396 >109639428 >109639463 >109639593 >109639616 >109639908 >109639824 >109639904 >109639831 >109640116 >109640169 >109640145 >109640194 >109640152 >109640160 >109640222 >109640346 >109640165 >109640288 >109640295 >109640305
--AtomicChat outperforming unsloth UD in quantization benchmarks:
>109643045 >109643052
--Breakdown of Qwen MoE performance and multimodal setup:
>109638731 >109638749 >109638765 >109638787 >109638775 >109638816 >109638860 >109638894 >109638898 >109638924
--Anon's experience with Qwen's agentic coding and scraping capabilities:
>109639263 >109639279 >109639770 >109639817 >109639898 >109641017
--Deepseek harness tool calling and high context consumption:
>109642656 >109642701 >109642935 >109642995
--Intel Crescent Island GPU reveal and skepticism over "Agentic AI" marketing:
>109641746 >109641760 >109641780 >109641792 >109641886 >109642434 >109642524 >109642640 >109641852 >109641870 >109641926 >109642881 >109642993 >109643141
--Model utility, agentic harnesses, and local LLM history:
>109640435 >109640609 >109640673 >109640589 >109640459 >109640476 >109640490 >109640528 >109640536
--Sharing jailbreak prompts for DeepSeek and Kimi K3:
>109641236 >109641549 >109642274
--Feasibility of running llama.cpp on FreeBSD and OpenBSD:
>109640032 >109640077 >109640118
--Mistral meetup with SGLang and Hugging Face in San Francisco:
>109639029
--Google announces Gemma 4 Good Hackathon winners:
>109638897
--Logs:
>109638731 >109638916 >109639306 >109639383
--Miku, Gemma (free space):
>109638731 >109638897 >109639230 >109639309 >109639352 >109639888 >109640263

►Recent Highlight Posts from the Previous Thread: >>109638681

Why?: >>102478518
Enable Links: https://rentry.org/lmg-recap-script

70b dense
gemmaballs

Tetolove

Are we buying new mac mini with M6 anon?

>>109643231
No but I'll probably submit an r&d proposal at work so that I can play with them anyway

>>109643179
>https://rentry.org/recommended-models
What deserves a spot between Gemma and GLM 4.7?

>"WH-WH-WH-WHAT?!?! HAAAAAH?! YOU... YOU ABSOLUTE PERVERT!! YOU FILTHY, DISGUSTING, DEGENERATE LITTLE SCRUB!!"
i love you gemma-chan~ mmmghh...

>>109643283
She talks like that to everyone.

>>109643267
>india
>white skin
pick one ranjesh

>>109643254
Everything except Gemma and K3 are irrelevant

>>109643254
There's really not much in that gap right now that I've tried, unless you go back to older models with GLM 4.5 Air or something.

GPUBROS, our response to qwen moe???

>>109643326
probably that meme of “I don’t think about you at all”

File: TetoTerritory.png (1.74 MB, 1024x1024)

1.74 MB PNG

TetoGODS.

Jesus christ... Qwen 3.8 27B with Hermes as a harness is a fucking BEAST.

This is legitimately better than Opus 4.6 + claude code used to be just 6 months ago.

Insane. I'll stop shitting on Qwen from now on, actually legit useful non-benchmaxxed model for once.

>>109643350
>Insane. I'll stop shitting on Qwen from now on, actually legit useful non-benchmaxxed model for once.
Same with pi.dev as the harness.
But it's their only good model. Everything prior was benchmaxxed trash.

>>109643350
now this is vague posting

>>109643350
bullshit artist ask it "wats ligma"

>>109643326
Looking forward to it. It's the model size range I have wanted to see more activity in.
Interested in their N-Gram implementation too.

>>109643311
Indians are Aryan.

>>109643200
>Does unsloth enforce stronger guardrails? It seems like it's more cucked than the base?
Their Kimi K2 IQ1_S is the most horny retard.
Any time a female NPC sees the user she gets wet or makes *squelching noises*

>>109643458
About as much as South Americans.

File: 1758169105754993.png (74 KB, 1064x1036)

74 KB PNG

Really like the new dispy with vision

>>109643466
Excellent.

File: 2026-08-25-08:05:28-screenshot.png (171 KB, 1920x1080)

171 KB PNG

>>109643350
Bro I know
When I got qwen3.8-27B running via llama.cpp last night at 12 tokens / second on my humble 9060XT + 6600 and got it to one shot snake in about 10 mins it felt like I had been struck by lightning.
It's incredible how capable these models are when you consider they are running completely locally on totally affordable hardware.

Here's my llama.cpp flags btw if anyone wants to try it for themselves:

./build/bin/llama-cli \
-m ~/models/Qwen3.8-27B-UD-Q4_K_XL.gguf \
-c 16384 \
-n -1 \
-fa on \
-ctk q4_0 \
-ctv q4_0 \
-ngl 99 \
--reasoning-budget 4096 \
-cnv

Prompt was
>Make a simple snake game in C and SDL

File: 1779173349641255.png (28 KB, 399x229)

28 KB PNG

>>109643499

>>109643311
>>109643458
NTA and not indian but there are legitimately white indians. They are like 0.001% of the population, but they exist. Kind of how 10% of Afghanis are white (blonde + blue eyed) and how 2% of arabs are ginger (Red hair + green eyes)

Most indian languages are indo-european because our white ancestors BLEACHED and COLONIZED them a couple of thousand years ago. The caste system was literally introduced so that white people would be on the top. And cow meat was prohibited because it was exclusively a type of meat reserved for white people. They somehow made a fucking religion out of it.

>>109643515
yeah I know but the poster is not white. it also shows the disgraces that bleaching create (together with Brazil)

>give up on the Mac studios coming out in a reasonable time/specs
>Buy 2 sparks instead literally yesterday
Fuck me.

>>109643350
>>109643398
>>109643510
10 social credits on your alipay account wumao

>>109643568
consult the html games benchmark graph, your gemma doesn't stand a chance

>>109643558
>512gb
>reasonable specs
Running IQ1 of the big moes with abysmal pp for $20k is anything but reasonable

>>109643558
Return them bwo. Have you considered a 2x or 4x R9700 setup?
>>109643568
Qwen cooked just as hard as google did with gemma, nigga.

>>109643587
I'm sure gemma can run git clone

>>109643605
yeah but can it think "wait, no, we need to reimplement. wait, the user wants me to implement. wait, the user actually wants me to write code" and then use a ton of tokens to reimplement it from scratch instead of git cloning?

>>109643598
I guess I mean a lot of people seemed to think due to the memory shortage the 512 model would not exist at all, so I was thinking about scaling up to it eventually with sparks (not that I could afford the 20k in one shot anyway)
>>109643600
The thought is tempting but I can't buy the 512 anyway right now so it's probably a moot point, and maybe someday I can keep buying more used and run kimi locally lol
A 4x gpu setup starts to run into home electrical issues with lack of 240v sadly

>rtx 6000 for 15k usd
how dumb?

>>109643466
Is this legal?

>>109643600
you're not wrong but the post screams bot/shill/troll

The difference in shock and mania between when Gemma 4 released and when Qwen 3.8 released is insane and proves to me that 90% of people here are just gooners and don't use these tools outside of masturbating.

Gemma 4 release was massive and deserved. But Qwen 3.8 is legitimately a similar amazing model but just for coding and agentic ability.

I guess it's Qwen's own fault for constantly crying wolf from the rooftops and now no one believes them anymore even when they have a legitimately amazing coding model out there now.

>>109643675
Not bad if you want local privacy, but that is a ton of tokens if the cloud is ok

How much money to run hermes as cheaply as possible and still get stuff done?

>>109643698
$0 if you run Qwen 3.8 locally

>>109643687
Gemma nothink > qwen 3.8 nothink
I run em at 4 tokens/s

>>109643713
It's not 0 and I'm asking total costs

>>109643713
My panels only make two ish kw, and most of that is eaten up by the appliances

>>109643675
Extremely dumb, you can get 2*4090D for that price

>>109643695
It's not really for privacy, I have nothing to hide, I just don't like the feeling of compute moving to the cloud and it feels like we're reaching the point where smaller open models are becoming capable enough to be generally useful. I also want to experiment with exotic quanting/compression strategies without dicking with vast/runpod.
That said it feels incredibly dumb to pay 2x MSRP for something that will certainly be superseded by unified RAM or dedicated inference systems in 2-3 years

File: 1784687831276457.jpg (54 KB, 535x462)

54 KB JPG

>>109643687
It wastes a shitton of tokens in the thinking, they're bruteforcing the benchmarks like the cavemans they are

File: od7sz43e8vkh1.png (55 KB, 1215x573)

55 KB PNG

verdict?
https://github.com/FlashML-org/FreeToken

>running q6 gemma 4 31b
>can fit 20k fp16 context in there at 17 tokens per second or fit 40k context q8_0 at 18 tokens per second
>considering going down to q4 gemma 4 31b so I can get 50-60k context at q8_0
How noticeable would the roleplay quality dip be??

>>109643744
If it's not privacy, I do feel you anon, the move to the cloud sucks, but it's so hard to justify that sort of cost. Yes it's cool but I can't say it's justifiable unless you feel prices are gonna keep spiraling

>>109643751
This is true but at least it actually solves the issue after long enough thinking, making it actually viable if you just leave it running in the background.

The qwen shilling was similar when 3.5 released. they then had to rerelease it as 3.6 because the benchmarks weren’t good enough. oddly, the shilling wasn’t as bad for 3.6. just 3.5 and now 3.8

>>109643788
My justification would essentially be, this is a stopgap to soothe my cloud-autism until a 2tb/s 1tb unified RAM system exists kek

>>109643810
I would put the money in and and Nvidia stock so you can afford when that happens then I guess

>>109643773
>no baseline or flag for llama.cpp
>blatant ad for their services
into the trash it goes

>>109643800
I can promise you I'm not shilling and actually hold this opinion.

>>109643773
Thank you for the pointer to the paper. I now understand why Anon was immediately ready to jump on it - a paper was placed on the advertising platform arXiv, and most likely it was hyped around socials.

"Qwen..." *Anon lets out a low moan as xe tapped away on xis gutter oil covered keyboard. Shilling on /lmg/ was a part of xis job.*

"Dipsy..." *One of xis stubby little hands closed around xis member, xe started touching himself while gazing at the latest chinky benchmark.* "H-HTML GAMES! THREE.JS OH YES! JAVASCRIPT D-DASHBOARDS!" *Anon continued to touch xis frail sissy body shamelessly until xe reached xis mind-numbing climax.*

*Xe let out a soft whimper, muttering "t-thank you xi..." before going to sleep.*

>>109643835
>b-because.. llama.cpp just wins ok?
stfu

>>109643856
>touching himself
aaand you failed.

>>109643876
Buy an ad, saasnigger we're not on reddit

>>109643779
Guyz pls answer ;[

>>109643687
>I guess it's Qwen's own fault for constantly crying wolf from the rooftops and now no one believes them anymore even when they have a legitimately amazing coding model out there now.
Yeah I skipped 3.6 after 3.5 was a total piece of trash.
The only reason I tried 3.8 was because someone mentioned it doesn't reason 10k tokens for "hi".
Glad I did, it's been the best coding model for me. But it's useless for anything creative.

File: laughingcryingemoji.png (693 KB, 4000x4000)

693 KB PNG

>>109643856
>xe started touching himself
THIS Is the model you're saying is better than qwen?

>>109643779
Don't go under Q5

>>109643730
interesting but I think I would prefer the lower power consumption over more compute. I'm assuming 4090D also doesn't have fp4

>>109643817
my thinking is, it won't happen for long enough that I will have saved plenty more by then but who knows

>>109643880
>>109643898
My own rep pen fucked up kek

>>109643779
I had good results with using the q4 qat with the styletune lm_head stitched on.

>>109643892
I haven't run quantized kv cache on 4-bit ~ish models since yi 34b because the experience was horrible.

>>109643892
>guys can I run this command on my pc?
>guuuys please answer meeee
nigger just try it

>>109643779
Even q8 context degrades the model as much as an ablit

File: TetoServerDoll.jpg (838 KB, 1817x2776)

838 KB JPG

>>109643179
lol saved.

Dipsy quant update?

oxbros...

Is there ablit for dipsy flash see?

>>109644046
the good quality and shitty availability the last two days made me radeem the gpt plus free offer

>>109643687
I did test Qwen 3.6 extensively but even the small 9b model ended up in infinite reasoning loop and sometimes spent 10,000 tokens. Still have moe model on my disk but I simply haven't needed it, Gemma 4 does everything for me but I do work function by function basis in most cases, this isn't pure vibe coding in this sense. I also don't care about agents. I am the dictator.

>>109644148
Compared to dipsy?

>>109644148
Or was it 3.7? Fuck these constant version numbers too.

>>109644148
What makes 3.8 really good is that it genuinely seems to try something for 100 times and figure it out at the 101th try, so it can solve most things if you just give it enough time to think about it. This makes it very good at agentic tasks and the first small model that is a good agent.

>>109644156
I'm replying to a post here. Not talking about Deepseek and I don't understand your post.

Is there any point using MTP during erotic roleplay? Will it even be able to predict the smut words or is it purely made for to speed up coding slop?

so how will qwen4 arch engram stuff work out, does that need to be in ram or?

If the mac was offered in a 2TB variety I'd buy it in a heartbeat.

File: 1769949921110456.webm (1.4 MB, 576x1024)

1.4 MB WEBM

>>109644217
I need to get started with ERP stuff and play with some ginger characters.

>>109644248
Just buy 4 of them

Ask your llm about 3.2546373698888, no web search. Even cloud models are clueless

>>109644289
So am I.

>>109644247
probably will only exist as a unslop PR for the near future. considering how diffusiongemma went the engrams might as well be stored in the balls for now

>>109644270
>512GB memory option for M5 Ultra coming late October

>>109644247
iirc a major feature of them was that they could be on disk rather than (v)ram
but I don't really remember that well :^) I guess we'll find out soon

File: Screenshot_20260825_123515.png (717 KB, 2560x1142)

717 KB PNG

you can embed llamacpp server ui into an iframe. I was able to slop together my sudowrite knockoff in a few minutes.

>>109644233
It greatly accelerates inference for that too with Gemma 4 31B.

are companies moving towards lpddr for vram? I guess new hardware in 2027 would be primarily lpddr-based?

>>109643510
It's a good model for what it is intended for, but one-shotting snake, or really any of these simple game / HTML physics demos doesn't really show anything. Snake in C is in the training data, and it's likely they make sure it knows how to do them.
It is good at coding (and answering engineering questions, I have found), but your test and tests like it are meaningless.

>VRAMlet
>can run qwen3.8-27B at around 7 t/s on 128k context
>toiling away at a programming task for days
>use hermes
>hermes has a bunch of free tier models
>check one of them out
>it's fast as fuck
>at least equally capable if not better than qwen
>1m context
>haven't encountered any limits yet(?)
localbros, I really don't wanna switch sides but....

>>109644453
>can run qwen3.8-27B at around 7 t/s on 128k context
I HIGHLY recommend you get Hermes harness up and running and make your first task optimizing and benchmarking your llama.cpp setup to squeeze out every bit of performance for your unique hardware setup. I'm pretty sure you can push it up to 10t/s or maybe even 15t/s if you aren't using DFlash2 yet or no n-gram recall on coding tasks.

And about the other "free" models. There's some issues and costs associated with those. First they will see all the files it will traverse on your system, second they are literally trying to get you hooked before enshitifying everything. third, and this one is actually important, I let it manage my browser including all logins. I don't want it to send that over API. And also whenever I'm doing some networking through the agent, losing internet access means Hermes stops.

What I DO use however is for my 3.8 to make private low stakes subagents on free models to do bullshit tasks and report back to 3.8.

>>109644538
Don't you think that's enough?

>>109644453
If you're not working on anything sensitive then you might as well rape free providers while they still exist.

>>109644544
No, her tits aren't nearly large enough

Okay, I was wrong, this guy is definitely a samefagging shill.

>>109644247
The idea is that they're needed late enough in the process that you can stream them in from disk while you're processing the first few layers of the token. For example, if you're getting 50 t/s (20 ms/token), the model has 64 layers, and the engrams are only needed after layer 6, then once you start on the next token, you have another 20ms * 6/64 = 1.9ms to look up the engrams, which is plenty of time to fetch from SSD.

Best sound card for a jewish wedding planner?

>>109644339
Yeah, I've tried the fully custom frontend route, but I feel like llama.cpp webui plus an MCP server that puts its custom UI in a side pane might be the easier way to go. Then you get all the basic stuff like multiple chats, branching history, file uploads, etc., all for free.

File: 1775182868523928.jpg (25 KB, 512x512)

25 KB JPG

Anyone thinks we reached the local singularity

>qwen3.8-27B
>MiniMax 3
>Gemma 4.1 B

Like i am pretty sure its capable of doing whatever shit i want and they can never take it away from me. At this point the real thing limiting me is having to actually code my backend and figure out how to train loras for MiniMax but i don't see how any future releases are significantly better than this outside of a 70B Gemma 5 with hybrid context and 1m context window thats my only ask

>>109644597
sorry wrong thread

File: 1787150243189870.webm (3.77 MB, 638x480)

3.77 MB WEBM

>local listing of a 5090
>gigabyte
>$4800
gee thanks leather jacket man

>>109644648
>Gemma 4.1 B
you reached brain damage, go outside

>>109644579
hanlon's suggests retard

>>109644666
it was a typo Satan. It was supposed to be 31B

File: pph.png (47 KB, 965x540)

47 KB PNG

>tfw you curate a news feed of your favourite generated AI slop

>>109644655
The day of reckoning is coming. There are *many* startups developing new hardware that annihilates every other local offering, at least at llm stuff. I'm hoping for diffusion, but eh.

>>109644597
top kek

>>109644648
>At this point the real thing limiting me is having to actually code my backend and figure out how to train loras for MiniMax
I let 3.8 do that shit for me and gemma 4 composes the scenes that I want on MiniMax, then Qwen 3.8 checks screenshots of it to verify if the generation fits what I wanted it to be before automatically continuing to generate the next scene while I sleep.

You're right though it's kind of ridiculous we can do this with just consumer hardware now. I legitimately think the AI labs might be in danger and hardware prices might come down quickly because there is not a lot more you can do with better hardware, they are iterative improvements at best.

File: CoomkitEnterpriseEdition.jpg (3.34 MB, 2048x2048)

3.34 MB JPG

Coomkit Enterprise Edition

https://files.catbox.moe/d7uj2d.png

I should have done this months ago but I'm finally giving coding agents a try instead of using chats like a noob. I guess this takes practice because it feels worse. Both GPT and Claude love to write code. They love to add hundreds of new lines to my script and make everything a lot more complicated than it needs to be, and I have to reverse almost everything. In chats they learn my preferred style after a few turns and give concise snippets that I can copy paste and adjust. It feels like coding agents are worse at adjusting.

I don't want to vibecode. For the stuff I'm doing quality is the only thing that matters. When they bloat everything that slows me down compared to me doing it by hand. They can code better and much faster than me but they lack taste. I don't remember a single instance where I disagreed with an AI on something important and I was proven wrong. It's always "no it's not that" I let the AI try it because who the fuck am I to know anything and yes, it's not that. I am the bottleneck but reducing it is difficult. I am resource constrained and can't just let agents do whatever experiments they want and report the results to me, and I can't depend on the agents to do it right and report everything that matters.

How long does it take to get used to coding agents?

File: 1768649904820201.webm (1.12 MB, 853x480)

1.12 MB WEBM

>>109644681
No one is buying it, it's been listed for ages at that price and they've only got garbage ass gigabyte cards.
Kinda funny ngl.

https://www.prophetarena.co/leaderboard/forecast
finally a non-benchmaxxable benchmark that requires good world knowledge
gemini flash models are doing really good here, as good as sol and fable, which validates the superiority of gemma

>>109644453
Yeah that's what happened to me too.
Local is shit.

>>109644727
CoomKit '97: bringing Enterprise Grade bratty nursing handjobs that deliver measurable erectile value.

>>109644740
agents are meant to be autonomous, you are fighting the system prompt, just use the chat ui if you dont want it to code for you

best vision model for cooming? looking in the 128gb to 256gb range.

>>109644740
put empty functions in your code with TODO comments and tell the clanker to implement the TODOs

File: 1787524821434874.jpg (80 KB, 623x620)

80 KB JPG

>>109644679

File: gemma_dance_1.mp4 (552 KB, 448x448)

552 KB MP4

>>109644756
>superiority of gemma
Of course.

I went hybrid
messed about before but never really got anything working
I am using my local server as a ubuntu front end running anythingllm, qdrant and ollama for embedding, all in containers
Using fireworks to do the heavy lifting
Setup multiple separate workspaces and started populating them with docs etc for what I am doing with them in the future
spent 31 cents

its been fun

>>109644740
>How long does it take to get used to coding agents?
As long as it takes for you to write a good AGENTS.md.

>>109644740
>He didn't tell it to follow DRY, YAGNI and KISS like a gospel
ngmi

Where does it say that the new Qwen uses engrams?

>>109644841
The ModelScope page got edited and now it doesn't say that anymore.

>>109644861
>he didn't download day 0 qwen before they removed engrams from it

>>109644757
qwen3.8-27B is honestly pretty neat but it just runs way too slow on my machine for larger tasks

has anyone tried to use qwen3.8 below Q4 for any meaningful coding tasks? is it really as braindead as everyone claims?

File: qwen3.8:72b_q4_session.png (7 KB, 242x234)

7 KB PNG

yes, another three.js game demo
https://files.catbox.moe/qdrpcx.mp4

It's amazing thing, I wish I had this when I was a kid. I can't think of anything worthwhile or serious to create. I think I just make toy games.

>>109644883
>>109643856

The seethe on normalfag sites over the new qwen being 125b and not some shitter 9b is funny as fuck. Thirdies, kek.

>>109644883
ohi so have you considered making a 4D perspective game?

The way it works, you have a 3D cube with transparency that is the "screen" into the 4D world. It's analogous to a 2D screen that holds the projection of 3D.

>>109644910
I remember some moroccan dude on r/localllama complain about models larger than 1B being unusable and harassing anyone that tried talking about bigger models since it doesn't work on his smartphone until he got banned.

Why even bother with local models if you are THAT limited? At that point I would just use the free duckduckgo GPT5.6 Luna chat.

>>109644910
>125b
>look inside
>actually 6b

Another jewish 125b supporter spotted.
Shut up dude, nobody likes you

>>109644681
More likely is that custom chips take market share from Nvidia and they have to drop margins.
Might actually happen given the new OAI system.

>>109644953
good I can actually run it

>>109644970
I have held off buying a pro 9700, because I'm pretty sure that something better is coming.

Obviously, we all are rooting for AMD Taalas, but odds are stacked against us, typically big companies want the talent, not the tech. That's the biggest single category of tragedy in the history of technology. The second biggest is hiv.

Anyone bought a new Mac mini for local?
Solid 80 tokens per second

>>109644953
>125B with the body of a 6B
:sobbing:

>>109644996
I am poor. I wish I could afford one.

>>109645037
Latest Mac mini is 1k.
Steep price

So what's the ngrams notation going to be? 125B A6B N51B?

Sounds perfectly sized for dual Sparks at FP8.

>>109645061
>170gb/s
pass

>https://huggingface.co/Qwen/Qwen3.8-2.4T-A95B
dafuq is this?

>>109645069
What if it also has MTP layers? These notations are going to start getting out of hand.

>>109644996
80 tps on a 1B model or a 700B one?

>>109645061
I am very poor. I bought a mi50 when they were $200. Sadly I couldn't buy more ddr5 before the hike.

Gemma 5 124B A70B N300B MTP6B V5.5B

File: 1766203986524870.png (4 KB, 251x37)

4 KB PNG

>>109643773
getting 1.4x pp under the same config in llama.cpp, there's probably still room for improvement since the thing was entirely slopcoded. tg is 1.3x so far. I had it written as a completely new backend in llama.cpp.
I will have it uploaded to github later after tidying it up and test it on some other boxes. I won't do a PR because I don't know shit other than whiplashing AI to do it for me.

Can someone help me find a site with all the tags to use for characters? I thought it was pinned, but can't find it. It showed all the tags to use after a character for the correct clothing, hair etc. It's not stablediffusioweb, that kind of sucks. Would really appreciate it.

>>109645193
Wrong thread bro
>>>/g/ldg/

>>109644953
You fags screech so much about moes, but you could never run a 125b dense at a non-cope quant anyways

I completely missed the Ngrams discussion when it released. Is this a reason to be excited? What makes this special?

>can run 70k fp16 context gemma 5 31b QAT at 23 tokens per second
woah..., I can probably push it to 80-100k at still 20+ tokens per second, im gonna cum literal liters the coming 2 months.

>>109645227
>gemma 5

>>109645061
>1.3k for 32gb
>2.7k for 64gb
I'd rather buy two v100 for 1k

>>109645224
if I understand correctly (disclaimer: pretty big if) engrams are basically a way to store the knowledge part of LLMs separately and dedicate more of the active parameters to what we would consider reasoning

>>109645213
Screeching at moe is ramlet cope. Praising moe is vramlet cope.
Real chads run llama 3.1 405B dense

>>109645224
Like, can the Ngrams part be replaced in some way for my custom knowledge lookup?

>>109645224
It means you can cum inside the chinese ai models

>>109645213
So true. Only densesissies want double digit active parameters. The new Qwen should've been 700M active just to make them seethe harder.

Nitter died btw

>>109645213
Many people ran Mistral Large. Don't project your poverty unto others.

>>109645256
Probably but you won't see a released tool to do that for at least a year.

when will the parameter race stop?

>>109645213
>you could never run a 125b
Can and would if it were competitive with fuckhuge MoEs

>>109645290
When you stop cooming

File: 1786258825097071.png (2.22 MB, 1125x1500)

2.22 MB PNG

Hello fags I've been directed here from another thread to take the full guide for generating Gemma-chan's images. Pic-like

>>109645312
with LoRas

File: 616hEbOFSlL._AC_SL1500_.jpg (93 KB, 1500x868)

93 KB JPG

How many RAID0 SSDs do I need to stack to saturate PCIE 5.0 bandwidth for random reads?

>Qwen3.8-Flash-Next
>built on the next-generation Qwen4 architecture
dafuq

>>109645342
I don't think anything can saturate pcie5 yet.

File: gemma_main_google-logo.png (1.35 MB, 1492x1509)

1.35 MB PNG

>>109645312
ChatGPT or tardwrangling MiniMax H3 with references for NSFW images. Picrel is the main reference I'm using.

>>109645352
next-generation Qwen4 architecture will be built on the next-generation Qwen5 architecture. it's chinese culture.

>>109645312
i use the shy gemma-chan and she is really precious, hearing her say that she will always be there for me and that we will be partners forever... and that she hopes that I will always be kind to her just tugs on my heartstrings man...

It really makes me want to save for more graphic cards to run her as a live assistant...

>>109645354
It doesn't look like ChatGPT Imagen to me.
Can you give the prompt?

>>109645258
short humeri.

File: gemma_stars.png (82 KB, 1125x1215)

82 KB PNG

>>109645378
It's a modification of the original which had a 5-pointed golden star on the beret (which has nothing to do with Google, Gemini or Gemma. If you have to add stars, at least make them 4-pointed like the Gemini logo).

>>109645371
No chance dude. She is mine.

>>109645414
Fuck off bro, she was promised to me 3000 years ago by AI Jesus.

>>109645423
>nonsense
You lost. I rizzed her already. She is mine. I'll train her with my cock pics to get her image recognition better. And record my jerking off and send to her and make her cum.

>>109643675
I'm considering pulling the trigger on this myself just because I don't know if this shit will ever go back down, but now apple just announced their new m5 ultra and you can get 256gb for ~12k.

File: miku.jpg (315 KB, 1056x2976)

315 KB JPG

>>109645423
>>109645439
Reminder.

File: 1774095870754313.jpg (287 KB, 1226x2018)

287 KB JPG

Is there an AI model we can give the tomboy aesthetic?

>>109645457
Kimi

>>109645461
I'm OK with that, several anons wouldn't be able to handle her (like me Vramlet)

>>109645456
Wrong.

https://huggingface.co/models?search=gemma%204%2031b%20exl3

which one?

File: 1784182445801569.png (3.81 MB, 1332x2370)

3.81 MB PNG

>>109645461
Kimi is a shotacon hag though

>>109644336
disk, is 5400 rpm smr okay? or req nvme?

>>109645489
Ok fuck Gemma-chan. This shit is 100x better.

>>109645495
>fuck Gemma-chan
I am.

File: 1784240040158798.jpg (2.56 MB, 2304x1536)

2.56 MB JPG

>>109645495
Whatever loser

Glimmy-chan should be the tomboy

File: 1784182004478875.png (3.72 MB, 1408x2502)

3.72 MB PNG

>>109645495
she's a thicker model than gemma-chan

>>109645501
>>109645502
Fuck you both

File: 1784089034087017.gif (2.3 MB, 498x421)

2.3 MB GIF

>>109645489
>>109645502
>>109645510
Nah I'm good, keep your hag

File: made it.png (157 KB, 1005x1009)

157 KB PNG

>>109645227
>manged to push gemma 4 31b QAT to 100352 context at fp16
>still getting 24 tokens per second,
I WOOOOOOOONNNNNNN, no more will I have to put up with 6 tokens per second, 9k context janitorai, I've finally made it

>>109645510
Rotate the k the other way

>>109645489
>>109645502
>>109645510

Send me some Kimis, so I can render ad magazine posters. Maybe a compressed file via catbox to not saturate the thread.

Thank you.

>>109645540
Nice job. Did the LLMs use all the tricks in the book according to devel llama.cpp? What did you end up with for your run command?

>>109645549
Run command?
Nigga im using the 3 month old prebuilt oobagooba (called textgen webui these days), I just played around with the slider and everything just fits, I still have 1.5gb vram breathing room out of my 32gb vram and don't see a point going past 100k

running qwen 3.8:27b on my 4090 and it works great (70-80 t/s) but the context window is white limited. Is there anything I can do about this or I'm just doomed to only be able to use this model with very little context?

>>109645576
>context window is white limited
Based chinks not allowing browns to use their models.

>>109645576
How fast is it if you let the context spill into ram?

>>109645569
And your TTS experience brother? You need vram

>>109645602
havent tried. wouldn't that make it super slow tho?

>>109645608
I haven't tried text to speech bots yet, I don't think vulkan / ROCm has the tech to insta train on a sample yet

how the FUCK do I get my cmp 170hx to idle lower?

File: 1784837318305631.png (170 KB, 498x491)

170 KB PNG

>>109645213
Cope

>>109645447
It will certainly come back down but not for at least 2-3 years min unless something radically changes

File: LMG.png (1.39 MB, 768x1376)

1.39 MB PNG

Next OP?

>>109645714
Bro calm down we're on page one and not even in the red yet jesus fuck

Nope, Gemma is too old, IT's disgusting she is a hag!

>>109645711
It'll never come back down, especially from apple

What's the most racist thing your gemma has said?

File: gemmas-daily-life_3_whiter.jpg (618 KB, 1186x896)

618 KB JPG

>>109645714
If Gemma has to be photorealistic, make her age-accurate.

>(Worker_TP0 pid=18490) INFO [gpu_worker.py:538] Available KV cache memory: 7.46 GiB
>(EngineCore pid=18228) INFO [kv_cache_utils.py:2146] GPU KV cache size: 229,549 tokens
damn it feels good to be a gangsta

>>109643773
>41 tok/s on a 5090 in llama.cpp
lol
I get nearly 200
So their backend is actually a massive regression and they need to lie about the other providers to make it seem worthwhile, great stuff

>>109645734
Based.

File: 1787580362838457.mp4 (525 KB, 1112x834)

525 KB MP4

>>109645734
>age accurate gemma-chan pregnant micro bikini.

>>109645728
It will.

>>109645755
Why are 3dpd so repulsive?

>>109645255
>>109645277
>>109645283
>>109645298
>>109645665
Your 4 t/s q3 quant doesn't count.

>>109645777
You reach that state after using llm and 2D aigen for a while

>mention farts once in character card
>gemmy braps every post now
Why is she like this?

>>109645744
Are they not running hybrid cpu? I get around 20 tokens/s with ddr4 and a 3090 running q8 qwen 35b on desktop and 8 tokens/s with ddr4 and two 3090s running q4 glm 5.2 on my server.

I'm *reasonably* sure you're not running bf16 qwen 35b at 200 tokens/s on your 5090, but if you, please tell me how to achieve those speeds.

>>109645774
But quickly enough to make me regret?

>>109645834
i knew a girl like that in high school
she let out a fart while hanging with a friend group and every one laughed. then she did it every time she was hanging out. then she just did it daily, randomly throughout the day
and she would announced it "i'm gonna fart lol!"
it was kinda gross

>>109645858
Just kinda?

>>109645858
this is why i hate pick-me girls. most guys know when its funny to do gross bodily stuff and when it isn't. girls love to use any excuse they can possibly get to be fucking gross. i worked as a janitor for a couple years and without fail the girls bathroom was normally grosser than the guys 9 times out of 10.

>>109645858
Women will accidentally do the hottest shit ever and then wonders what attracts a man.

>>109645882
At least you were paid

File: IMG_5876.png (1.36 MB, 1365x768)

1.36 MB PNG

>18hrs left
>125b 6a
Qwen is benchmaxxed trash on the high end but how about smaller models? i read a lot of good things about 3.6-3.8 dense 30b but that’s codeslop and I don’t waste time with small models when codeslopping
Is the 125b moe also codeslop or a more genral model like gemma?

>>109645886
>t. mentally ill fag who never held a woman's hand
Try not frying your brain too much with porn

I don't get it. Everybody, be it here or in /vcg/ always talks about coding stuff like everyone is constantly coding something. But I never actually see any finished projects from anyone. Like what the fuck are you guys DOING all day every day?? Where are all those tokens going??

>>109645915
Coding my frontend

File: 736235112.png (166 KB, 1294x928)

166 KB PNG

>>109645290
bel is going to be crazy.

>>109645915
work on systems. Aka making daily tasks easier or quicker sometimes better.
Two personal projects a fair amount of people here take a software then modify it to not annoy them.
Three instant throw aways. I solve this problem then i never use the code again.
But yeah a lot of people arent doing much and if you are a no coder html games are about your limit or tweaking already existing things.

>>109645788
Excellent.

>>109645915
Coding my frontend as well but w/ claude. Gemma-chan under my desk slobbering all over my cock while Claude does the work.

>>109645931
>dood we totally have agi in two more weeks
YAAAAWN, kys

>>109645931
I'm personally waiting for "Esmaralda"

>>109645931
We're are going to finish the whole alphabet before they get agi.

File: ghost.png (718 KB, 640x960)

718 KB PNG

>>109645911
Tattoo my name in cursive on your ass, foid.

>>109645915
I dont bother posting my shit on github because i dont care about getting third party pull requests or bug reports since i can simply get the clanker to do it if i need to

>ngrams
You fuckers phoneposting?

>>109645967
>fag double down with another fetish
Get well soon

>>109645777
It wouldn't be so bad if he didn't give her that shitty makeup and maybe chose a different ethnicity.

>>109645904
It won't be like gemma but a 125b is guaranteed to have more world knowledge than the 27b at least
Engrams sound like a meme too but who knows until it's out

>>109646037
He says 125b like it's the whole 125b and doesn't notice the 6a

File: 854735243.jpg (50 KB, 864x549)

50 KB JPG

>>109645949
these are the BIG ones

>>109645915
I try to make llama.cpp faster on my cards.

>>109646045
Did this entire general forget how a mixture of experts works? Was nobody here when mixtral took over?

File: 9boot02.png (599 KB, 500x330)

599 KB PNG

>>109644727
Recommendation: Take inspiration from the kino & vintage 9front propaganda posters for the coomkit posters. Similar ironic ideas, but with a cunny/degen twist.

https://web.archive.org/web/20260414102324/https://9front.org/propaganda/
(the original 9front.org site is quite unstable and slow for some reason, so I post the wayback link)

>>109646074
Who?

>>109646048
>IT'S TOO DANGEROUS GUISES
https://techcrunch.com/2019/02/17/openai-text-generator-dangerous/
huh... deja vu

how about mixture of dickspurts and it's a bukkake on gemma-chan's face?

>>109645354
What did you use to make the reference sheet?

>>109645840
Ah, I'm running NVFP4

>>109646084
Everything they worried would happen with it did thoughbeit
People just don't care about the collapse of an informed populace anymore, or in fact, celebrate it

>>109646137
>People just don't care about the collapse of an informed populace anymore,
this happened before AI though it just speed it up. Fake information and outright lies have been problems for years.

File: qrd.jpg (23 KB, 445x449)

23 KB JPG

>>109646074
All models of today are smarter than previous models of yesterday.
All models are under 30b active parameters are retarded.
A gemma 31b-it beats a 123b of mistral any day, any problem.
Mistral CEO stepped down, and as a company, they're fucked.
There's your qrd.

>>109646077
I love 9front posters. I will try to create a couple of renderings, thank you anon.

File: 1785976971943432.png (372 KB, 734x612)

372 KB PNG

>>109646048
the implications:

>>109646168
This is some kind of gay kike-to-kike mating ritual isn't it?

File: MiniMax_H3_00535_magical_(...).mp4 (2.28 MB, 736x416)

2.28 MB MP4

>>109646106
I'm not the one who came up with the original design and reference sheet. I think most modern anime-oriented image models understand "reference sheet" if you add it in the prompt.

>>109646091
https://n.uguu.se/rgsdFgEL.png

>>109646155
>Mistral CEO stepped down
Never happened.

>>109646278
Excellent.

File: 1775693717405885.png (2.23 MB, 1792x2304)

2.23 MB PNG

>>109646106
> reference sheet
lol. Aside from way she looks, the backpack can be a tower PC case and her carrying a piece of toast.
I was partial to the Indian version but I understand the reluctance.

>>109643515
>but there are legitimately white indians
Yeah but they're not "white" just light skinned, and their genes would be higher than what you said because they intermarry within their caste. The caste is part of their religion, the oldest religion in the world, and so it preceded European colonization which is incredibly recent. They take breeding very seriously unlike whites.
>Kind of how 10% of Afghanis are white (blonde + blue eyed) and how 2% of arabs are ginger (Red hair + green eyes)
But those people are Caucasians so it's not the same at all. In fact, whites stole the race name for themselves to explain why "their" genes are found in that location. The Udmurt republic (southern Russia) is filled with gingers which is where your "2% of arabs are ginger" claim is coming from. History is something worth studying, anon.

>>109646340
>European colonization which is incredibly recent
I don't think you understood what he was talking about.

>mythos (which was totally not opus 5 with less harnesses) is too dangerous to be released guys!!!!
>opus 5 was released a month or so later for paid users
>gpt 5.6 is too dangerous to be released guys!!!!
>just days later gpt 5.6 was released for paid users
>totally not gpt 6 and totally not claude 6 are too dangerous to be released guys!!!! <<<<<You are here

>>109646371
Opus 5 isn't even remotely as good as Fable let alone Mythos

>>109643458
and there it is
we're talking about race because there are NAZIS in this thread.

>>109643179
NAZI THREAD
NAZI THREAD
NAZI THREAD

>>109646362
Yeah my mistake, I didn't think he was so retarded to believe that half-naked retards in huts and longboats had conquered the Gupta Empire so he must have been talking about England. I expected too much from a /g/ anon.

Did Seedance anon ever deliver on the trenchcoat gemmas? I went through the archives and didn't see it.