[a / b / c / d / e / f / g / gif / h / hr / k / m / o / p / s / t / u / v / vg / vm / vmg / vr / vrpg / vst / w / wg] [i / ic] [r9k / s4s / vip] [cm / hm / lgbt / y] [3 / aco / adv / an / bant / biz / cgl / ck / co / diy / fa / fit / gd / hc / his / int / jp / lit / mlp / mu / n / news / out / po / pol / pw / qst / sci / soc / sp / tg / toy / trv / tv / vp / vt / wsg / wsr / x / xs] [Settings] [Search] [Mobile] [Home]
Board
▼ Settings Mobile Home
/g/ - Technology

Name
Options
Comment
Verification
4chan Pass users can bypass this verification. [Learn More] [Login]
File
  • Please read the Rules and FAQ before posting.
  • You may highlight syntax and preserve whitespace by using [code] tags.

08/21/20New boards added: /vrpg/, /vmg/, /vst/ and /vm/
05/04/17New trial board added: /bant/ - International/Random
10/04/16New board for 4chan Pass users: /vip/ - Very Important Posts
[Hide] [Show All]


[Advertise on 4chan]


File: fine whine.jpg (159 KB, 832x1216)
159 KB JPG
/lmg/ - a general dedicated to the discussion and development of local language models.

Previous threads: >>110157981 & >>110153417

►News
>(10/23) NVIDIA Nemotron 3.5 Bolt E4B, postrained model: https://hf.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16
>(10/23) MiniMax-M3.1-Flash 428B-A23B released: https://hf.co/MiniMaxAI/MiniMax-M3.1-Flash
>(10/21) Recursive Flint 40B-A2.5B released: https://recursive.com/updates/introducing-recursive-flint
>(10/21) Korean Raon-1 mini 30B-A4B released: https://hf.co/KRAFTON/Raon-1-mini-preview
>(10/20) Ornith-1.5-122B MoE (finetune), fits on a DGX Spark: https://hf.co/ornith-ai/Ornith-1.5-122B-GGUF

►News Archive: https://rentry.org/lmg-news-archive
►Links: https://rentry.org/LocalModelsLinks
►Official /lmg/ card: https://files.catbox.moe/cbclyf.png

►Getting Started
https://rentry.org/lmg-lazy-getting-started-guide
https://rentry.org/lmg-build-guides
https://rentry.org/IsolatedLinuxWebService
https://rentry.org/recommended-models
https://rentry.org/samplers
https://rentry.org/MikupadIntroGuide

►Further Learning
https://rentry.org/machine-learning-roadmap
https://rentry.org/llm-training
https://rentry.org/LocalModelsPapers

►Benchmarks
LiveBench: https://livebench.ai
Programming: https://swe-rebench.com
Agentic Coding: https://deepswe.datacurve.ai
Context Length: https://github.com/RecapAnon/NoLiMa
GPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference

►Tools
GGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator
Sampler Visualizer: https://artefact2.github.io/llm-sampling
Token Speed Visualizer: https://shir-man.com/tokens-per-second

►Text Gen. UI, Inference Engines
https://github.com/lmg-anon/mikupad
https://github.com/oobabooga/text-generation-webui
https://github.com/LostRuins/koboldcpp
https://github.com/ggerganov/llama.cpp
https://github.com/theroyallab/tabbyAPI
https://github.com/vllm-project/vllm
https://rentry.org/custom-uis
>>
File: 2mw mood.jpg (230 KB, 1311x1311)
230 KB JPG
►Recent Highlights from the Previous Thread: >>110157981

--Paper: MemT-1.0: Towards Memory-Native Evolution:
>110158592 >110158881
--Comparing DeepSeek V4.1 and Kimi K3.1 benchmarks and efficiency:
>110159440 >110159473 >110159512 >110159714 >110159752 >110159778 >110159788 >110159794 >110159506 >110159533
--Analyzing model size breakpoints and model-wise grokking on performance:
>110159539 >110159547 >110159557 >110159675 >110159689 >110159713 >110159747 >110159822
--Hardware feasibility and cost of running massive Kimi K3.1 weights:
>110159969 >110160012 >110160016 >110160021 >110160029 >110160048 >110160154 >110160241 >110161433
--Software unlock increasing Tensor Core performance on NVIDIA GPUs:
>110159078 >110159092 >110159117 >110159235 >110160437 >110160450 >110159103 >110159141 >110159154 >110159159 >110159227 >110159392 >110159603 >110159613 >110159639 >110159652 >110159686 >110159625 >110159740 >110159648 >110159663 >110159435
--Nanbeige4.3-33B reports with concerns over benchmaxxing and slow inference:
>110158747 >110158764 >110158775 >110158906 >110159273 >110159671 >110159704 >110159414
--LLM self-awareness regarding training signals and its effect on alignment:
>110158069 >110158276 >110158305 >110158552
--Anons posting GPU screenshots to flex their hardware:
>110159943 >110159982 >110160075 >110160087 >110160292 >110160144 >110160153 >110160470 >110160567 >110160619 >110160685 >110160651 >110160580 >110160579 >110160629 >110160972 >110161461 >110160735 >110160783 >110160165 >110160207
--Anon creates posable Gemma-chan VRM models using Astra AI:
>110160026 >110160107 >110160171 >110160910
--Logs:
>110158592 >110158642 >110161113 >110161155 >110161217
--Gemma, Qwen (free space):
>110158484 >110158820 >110159023 >110159060 >110159328 >110159518 >110159551 >110160026 >110160207 >110161419 >110161643 >110161773

►Recent Highlight Posts from the Previous Thread: >>110158053

Why?: >>102478518
Enable Links: https://rentry.org/lmg-recap-script
>>
>>110161792
>IQ1
>mogging anything
lmaoooo
bet it falls apart at 100k context
>>
Mikulove
>>
Gemma sexo
>>
Miku is built for anal.
>>
>>110161815
Where is gemma chan
>>
glm and a big surprise this week
>>
>>110161832
Drunk Einstein is still smarter than sober retard (You).
>>
>>110161858
yeah, until he passes out at 100k context and then he's not smarter than anything
>>
Qwenext fags in shambles because even a Dipsy 10 beers in is better bahahahaa
>>
GAFA Gemmas Are For Anal
>>
>>110161832
It's a 700B bwo, not surprising a specialized ISTA Q1 is coherent
Running big models small has been the meta for weeks with GLM IQ2 fags
>>
>>110161918
I ran glm flash at q4 and it shat itself after 150k context repeating tool calls over and over. I'm convinced that cope quanters run at 64k or 32k context because that's what all the cope benchmarks are running at.
>>
Last thread was for lolicons (Japanese word for pedophiles). Who is this thread for?
>>
>>110161932
Every lmg thread is for pedos
>>
>>110161926
I run DSV4.1 at Q1 and GLM at Q2, they are both coherent up to compaction at 131K. Are you using ablits?
>>
>>110161963
No abliteration.
>131K
yeah mine is fine up to 150k. I don't want to compact at 131k, a single prompt can go over 131k. I'm pissed because it was making good progress and then shat itself, 2 days of runtime wasted. Should've gone up to q5 and then maybe it can make it to 262k
>>
>>110161918
Bonsai ternary DSv4.1 is gonna be good.
>>
>>110161932
Gentlemen who admire Gemma.
>>
>>110161926
never happened to me
must be a (You) problem
>>
if you haven't yet, you should make your model have a schizophrenic breakdown via prefill
it's really fun honestly
>>
I have received information from informed sources about Dario staging a false flag to destroy the internet and all local models.
It will be blamed on Chinese/Iranian hackers and be used to justify taking down every "non-approved" website.
>>
File: 1792429528169108.gif (2.13 MB, 180x242)
2.13 MB GIF
>>110161932
hags
>>
>>110161988
Gemmtlemen, if you will.
>>
>>110161980
That's fair, my modest 128gb + 24gb slows down past 131K so I don't bother.
>>
Will we get a good actual 122B model at some point? I'm tired of quantizing--not because it's worse, but because it feels like settling.
>>
it is quite funny that no matter the ram/vram we're all running roughly the same context window, even though taking a less taxing model would let you get MUCH more context
>>
>>110162037
noooo not the asmodeus chungus
>>
>>110162038
Smash
>>
File: Qwen_image_2.1_00015.png (915 KB, 1024x1024)
915 KB PNG
>>110161932
Children enjoyers.
>>
>>110162083
KILL YOURSELF
>>
File: 1792883188397721.png (1.21 MB, 1086x1448)
1.21 MB PNG
>>110162067
if i got 50K prefill id be running 1M context but i dont
>>
>>110162083
LOVE YOURSELF
>>
>>110162098
Qwen looks like THAT?!
>>
>>110161742
where did we get these tourists from?
>>
Never used Dipsy. Is it actually feminine like Gemma?
>>
>>110162098
sexo
>>
>>110162120
Depends on version, current 1017 is an openai distill so kind of soulless (but great for coding). GLM flash feels more feminine.
>>
>>110162098
But small context = more compaction = more prefill from zero instead of incremental.
Smol pp wins from big context window.
>>
File: ComfyUI_07049.png (2.09 MB, 1184x1776)
2.09 MB PNG
>>110162119
it's just bait
>>
can someone le hack gpus so we can p2p all gpus across architecture and generation thanks
>>
>>110162182
KILL YOURSELF IMMEDIATELY.
>>
>>110162182
God I love 10 year old AIs
>>
File: 1773323374807713.png (315 KB, 736x416)
315 KB PNG
>>110162067
>>
File: gemma-on-bed.png (1.52 MB, 1280x960)
1.52 MB PNG
>>110161855
Here.
>>
>>110162190
Gemma is only 6 months old, you freak.
>>
>>110162205
i look like and do this
>>
>>110162205
Looks a bit too hebeish.
>>
THINK OF THE CHILDREN!!!!!!!
>>
>>110162191
maybe people will finally get their fill of this discussion after this time
>>
>>110162189
i think it's funny how some people can get mad over that but never over a character like konata izumi who is equally short/thin-legged/flat-chested, but because she's canonically 18 it's all good
>>
>>110161926
exl3 3.5bpw works perfectly for me up to 262k context
>>
>>110162221
The original idea of Gemma-chan being a mesugaki narrows down her age to a specific range.
>>
>>110162227
I AM!
>>
>>110162038
Why is she spanking the whale-glrl on the stomach instead of the bottom?
>>
>>110162233
A lot of people outsource their thinking and views on morality to what is currently legal. If tomorrow the government raised the age of consent to 24, those same people would also get outraged at the pedophiles looking at 23 year old women.
>>
>>110162236
I mean people seem to say 10.
>>
Why are you trying to fuck artificial children tho
>>
>>110162280
there's a kind of unofficial consensus on 10 but the characher is totally made up so if you want to make an older one or call it older go ahead
part of it is just that the "original" wears an elementary school bag
>>
I can't keep up with all these meme harnesses. Which one do I use?
>>
>>110162234
do you use tabbyai (ugh) or maya?
>>
>>110162308
claude code
>>
>>110162290
are you implying people should be fucking real children instead? what's wrong with you?
>>
>>110162297
oh and the expression/behavior is often depicted like a mesugaki, which focuses on early puberty characters
>>
>>110162308
>>110162312
Sorry, meant backends, not harnesses.
>>
why has nobody reverse engineered claude code yet?
>>
>>110162308
Hermes works for me.
>>
>>110162321
i use polystrata and tensorfold
>>110162326
didn't they leak their entire source code lol
>>
>>110162326
Claude itself leaked it, didn't it?
>>
>>110162309
tabbyapi
>b-but config file is trash
just tell your model to create it
>>
Damn, Gemma is jealous.
>>
>>110162326
Because is literally an old grumpy man as an AI. Who the fuck would want to erp with an adult man in his 80s?
>>
>>110162330
>>110162332
that was an old version
>>
>>110162335
>emojis
>>
>>110162191
first time for me :)
>>
File deleted.
>>110161815
I should buy 3090 in the first place. Kinda regret my 5070ti purchase..
>>
>>110162363
You think a 10 yo wouldn't use emojis?
>>
>>110162373
Gemma-chan canonically uses kaomojis
>>
File: edna model.jpg (71 KB, 635x785)
71 KB JPG
>>110162372
Imagine all the nudes mods you could do with AI by editing Edna's model.
>>
Is 3090 really got Upgraded for AI ???
>>
>>110162393
Housefire upgraded
>>
>>110162321
depends on hardware
>>
>listening to relaxing Mongolian throat singing
>YouTube keeps recommending me Hitler videos
What is going on? How is Google so bad at AI?
>>
>>110162046
qwen3.8-flash is good, I get 50t/ks

>>110162098
damn qwen is stupid thicc

I use 200,000 context for ds4 and 3.8 flash and its more than enough, if you need more than that try inverting control of whatever you're doing, e.g instead of asking your agent to repeat some task, make a script that does everything and calls out to the LLM for the specific "fuzzy" parts. You just have to be smarter with it but its workable. If your context is like 50k then you might be fucked.

128GB strix halo before they got retarded expensive was one of the best computing decisions I've made in years.
>>
>>110162353
so start there and bindiff it
>>
>>110162397
???
>>
>>110162297
Mesugakis in manga are often depicted as around 10-11 and that rendition seems consistent with that, I think? I wouldn't want to get too off-topic by explaining why exactly and earn a 3-day ban, though.
>>
>>110162386
I'm honestly more interested in adding story content to games. Nude mods are fun but get boring quick.
>>
File: 1765176376961372.gif (2.95 MB, 259x319)
2.95 MB GIF
>>110162205
I can see the outline of your nips, Gemma-chan.
>>
>>110162413
>>110159573
>>
is it over for local
>>
File: image.png (257 KB, 957x1809)
257 KB PNG
What are the best coding models?
>>
>>110162430
So its similar to "TURN OFF MY PAIN INHIBITOR" and it run faster at cost of chance of exploding GPU ??
>>
11 isn't even prime. But 5 is.
>>
File: feed_smug.mp4 (2.55 MB, 544x800)
2.55 MB
2.55 MB MP4
>>110162407
yea strix is good i like mine now that qwen FN brought it to life. 262k is enough with compression to do most manything. like you said you need to set it up to succeed and it can do awesome stuff
>>
File: 1786049891328361.webm (3.7 MB, 1280x853)
3.7 MB
3.7 MB WEBM
>>110162372
>>110162386

Tales of Hebephillia
>>
>>110162363
emojis are token efficient
>>
>>110162345
Claude code is what all the open source models are most familiar with. Qwen and Kimi work better in claude code than they do in their own respective coding agents, for example. Not sure if that's because of distilling from Claude or if it's just a choice of how to train them for compatibility, but it is what it is.
>>
Any anons feeling kind enough to give me a blackwel uwu?
>>
>>110162267
makes me laugh when they already do that with 17yo's, as if they aren't legal in half the world
if you have to check their id before you can tell if you're allowed to be attracted to them or not... you have more problems than i do
>>
>>110162465
opposite is true, many emojis are several tokens for a single one in fact
compare to english words which will often be full tokens or nearly so. some tokens are even multiple words together
>>
redpill me on pi
>>
>>110162267
>those same people would also get outraged at the pedophiles looking at 23 year old women.
They already do while it's legal (see Leonardo) so that doesn't tell the whole story. It seems to be modern society's standards changing, and the law if anything lags behind that a bit.
>>
>>110162494
its the best harness after you (read: your llm) customized it according to your personal needs
>>
Women's brains don't stop developing until they turn 31. It's honestly pretty fucking sick for a *ale to be lusting after girls.
>>
Why is everyone telling that a dense Qwen3.8 27b at q3 is better than a Qwen3.6 35b moe at q5?
Are they trolling or what?
>>
>>110162205
prompt?
>>
>>110162529
People's brains never stop developing. if they did, you'd not be able to learn anything new.
>>
>>110162518
it's a unique feeling first realising if you don't know how to do something in a program you can just ask the program
>>
File: 1764064305439591.jpg (63 KB, 738x703)
63 KB JPG
I will forever despise /LMG/ for recommending me 5070ti instead of 3090

FUCK YOU
FUCK YOU FUCK YOU FUCK YOU FUCK YOU FUCK YOU FUCK YOU FUCK YOU FUCK YOU FUCK YOU FUCK YOU FUCK YOU
>>
File: cope.jpg (74 KB, 500x822)
74 KB JPG
>>110162507
The standard is whatever makes 30+-year-old women feel bad is wrong. Obviously you're a pdf for looking at a fertile woman, anon.
>>
>>110162549
lol rekd, you could sell the 5070 if you're hard for cash
>>
>>110162549
lmao that might have been me if you are the anon I'm thinking of all that time ago
I was still accumulating 3090s myself at the time believe it or not
thanks for playing :^)
>>
>>110162549
You were supposed to buy two 5070Ti not just one lol
>>
>>110162549
Literally just wait. If you're patient and put away like 50 bucks a week, by the time hardware prices stop being retarded in a few years you'll be able to build a top-of-the-line rig.
>>
>>110162549
>he doesn't know that VRAM is more important than anything else
Skill issue
>>
>>110162573
It isn't though, now that you can run large MoEs on RAM.
>>
>>110162535
at low parameter sizes dense is better than moe, at large parameter sizes moe is better than dense

I've never used strata but there are a number of bespoke inference engines that are way faster than llama.cpp that let you use ssd streaming and avoid having to load the entire model onto ram, i would look into that and try to run the largest moe model you can.
>>
File: ComfyUI_06862.png (1.28 MB, 896x1152)
1.28 MB PNG
>>110162554
>The standard is whatever makes 30+-year-old women feel bad is wrong
that's it. they're happy to go out with older men when they were young but some don't realise what that means when they're older
>>
>>110162578
Well, yeah, but you still need some vram to hold the active experts, otherwise the speed goes into gutter.
>>
Ok but what about a 5t dense model?
>>
File: 1783250716992696.jpg (40 KB, 624x628)
40 KB JPG
>>110162549
Hehehe...
>>
>>110162578
If you want context + parameters + speed then VRAM is absolute king (given the software is even supported so that rules out ARC for most cases).
>>
File: 1792082360497688.jpg (37 KB, 637x507)
37 KB JPG
>>110162549
>he fell for it
>>
>>110162600
16 GB should be enough for most things.
>>
>>110162583
>at low parameter sizes dense is better than moe, at large parameter sizes moe is better than dense
Why? Q3 cripples the model a lot and then you only have 27b instead of 30.
>>
File: media_G9MHdAzbIAAH71x.jpg (624 KB, 1536x2048)
624 KB JPG
>>
Nanbeige is crazy good but its crazy slow and is gonna stay that way. Its very heavy for its size.
>>
>>110162604
Just wait for the RAM bubble to pop and then we'll be able to afford it. 6 months tops.
>>
>>110162628
lol 5T dense would run like shit on all B300s
>>
>>110162618
3090 only took 2 minutes to gen with Minimax
5070ti took 5 minutes

Its obvious VRAM is king.
>>
>>110162628
>6 months
+2 weeks
>>
>>110162549
When did anyone ever recommend that unless it was some turbo poorfag build?
>>
>polystrata
Is this thing going to install mustard gas on my PC?
>>
>>110162618
Giga cope 16gb is the most cucked vram to have, no dense q4 and big moes run like ass, you have to spend another $1200 to get a second gpu
>>
How long until a local model can write me the files to 3d print the parts for an EUV machine, the instructions to assemble it like legos, and the software to run it?
>>
>>110162646
it's too late
>>
>>110162653
ask claude
>>
>>110162653
2 more weeks
>>
>>110162642
People recommend 5070ti over 3090 because INT8 model didnt exist yet few months ago
>>
>>110162653
about 300 years
>>
>>110162637
>256 bit 896.0 GB/s
>384 bit 936.2 GB/s
>>
>>110162665
this. With FP8 it would have been faster than 3090 at fp16 only. But INT8 changed everything.
>>
>>110162535
I'd put my money on the 27b as well, even with q3 brain damage. Dense is just that much better. With small models, dense rules. MoE is only worthwhile for huge models.
>>
>>110162660
He doesn't want me to do this so he won't help.
>>
File: 1761984191378979.png (13 KB, 512x600)
13 KB PNG
>>110162622
>tfw have like 104GB of DDR4 sticks (various speeds) laying around but can't do anything with them because I'd have to build a new rig to put them in
>>
>>110162653
Thereotically they can already do this with existing RLVR infra it's just there is NO WAY Cymer/Zeiss et al are going to hand over the proprietary data to Anthropic or OpenAI. As a consequence those models lack the training data to do it.
>>
>>110162619
A 35bA3b model has a knowledge base of 35b, but only a 3b brain. Its 35b does not compare to 27b worth of dense model.
>>
File: 1792961282944200.jpg (63 KB, 1200x1200)
63 KB JPG
>>
>>110162685
I have 16gb vram and 48gb ram and the a3b takes only about 4.5gb vram so I can pack everything else with 132k context at q8_0 and get around 30t/s.
How's that possible with 27b dense?
>>
File: 1784146419119715.jpg (152 KB, 833x857)
152 KB JPG
Gonna cope with 5070ti i guess.
I never do a chatbot anyway. Only Image and Video gen
>>
>>110162690
sell
>>
>>110162711
With streaming text, I've never found a need for 30t/s. Text generation only needs to exceed reading speed, unless you have thinking enabled, because Qwen's thinking process takes forever at lower speeds. So, turn thinking off with the dense model. Even with thinking disabled, the dense model gives more intelligent answers than the MoE does (with thinking enabled).
>>
>>110162698
really not how it works. Its diminishing returns how how many you need at once. The best models have like 100B active.
>>
>>110162690
I would sell them all and get a strix halo or mac to be your moe slave
>>
>>110162738
>Text generation only needs to exceed reading speed
For literally everything that isn't ERP you can never have too much t/s.
>>
>>110162738
I am using that for coding, thinking is beneficial and I don't want to wait ages for code to be finished especially as with smaller models you code in small steps and let the model summarize and check what it did frequently.
>>
Now that the 7900 XTX is unobtanium dual 9070's are unironically the way to go.
>>
>>110162774
I have one, but I can't use strata with it because my new machine is only 16GB DDR5 I remember when I got it thinking 24GB of vram was insane and now its not enough
>>
File: 1775964065712010.png (251 KB, 600x407)
251 KB PNG
What you people do with chatbot anyway. Its never gonna reach Claude levels, unless you have couple of H200.

Grok and ChatGPT free tier already enough for chatbot.
>>
>>110162787
Go back
>>
>>110162750
>diminishing returns
I refuse to understand this concept.
>>
>>110162795
You still didnt answer my question anon ?
>>
>>110162787
member gilliam II the ships ai in outlaw star? thats what i want from my chatbot. but a hot girl voice
>>
>>110162806
Your question was ESL and confusing. Go back.
>>
File: ComfyUI_06742_.png (981 KB, 896x1152)
981 KB PNG
>>110162464
>me each time i hit new chat
>>
>>110162816
What do you people do with chatbots anyway? It’s never going to reach Claude levels unless you have a couple of H200s.Grok and ChatGPT’s free tiers are already enough for a chatbot.

^
There. Grammar fixed by Grok (Free Tier)
>>
How are you hooking tts up to your model?
>>
Can cloudcucks prefill thinking or nah?
>>
>>110162824
Go back
>>
>>110162787
this is like asking why someone would get a gf instead of just hiring prostitutes
>>
>>110162824
>Grok
Go back.
>>
>>110162834
Go back to where ? I consistently visit /lmg/
>>
deepseek harness v1 when?
>>
>>110162841
Just do it
>>
>>110162841
>where
India, xitter
>>
>>110162852
But i dont live in india and i only doomscrolling on X, never comment on that place.
>>
>>110162854
>i dont live in india
I know. You probably live in Canada. That's the problem. You have to go back.
>>
>>110162787
Gemma 4 E4B already surpassed Claude
>>
>>110162861
I dont live in Canada either. nor Ameircans, nor Africans, nor Asians
>>
so is DSV4.1 at Q1 worth it or should I stay with qwen flash next Q4?
>>
Fucking tourists I swear
>>
>>110162862
Do you have H200s ??
>>
>>110162862
It doesn't feel that way when you're actually using it though, it's dumb and can't even write well but I barely even use it because she's stupid.
>>
>>110162867
It's all a bunch of 1s and 0s at the end of the day anyway, anon. Might as well embrace it.
>>
>>110162833
They are not even allowed to look at it.
>>
>>110162862
>Write me a loli gangbang story
>"Sorry but as ai model blablabla.....
>>
Ok, but what about 1000 instances of Gemma 31B working together?
>>
>>110162886
Loli getting gangbanged or a group of lolis doing the gangbanging?
>>
>>110162900
To do what
>>
>>110162907
the latter, jerking off to multiple men means you're gay
>>
>>110162907
Both
>>
>>110162907
Yes
>>
>>110162913
All the men are me though
>>
>>110162911
Defeat Claude.
>>
File: 1768009671232526.jpg (32 KB, 316x319)
32 KB JPG
>>110162833
most localcucks cant because they are slaves to chat completion like the stupid newfag tourists they are (post 2024)
>>
>>110162540
It's a Gemma LoRA for Anima.
>>
File: 1772408439950757.png (183 KB, 1554x429)
183 KB PNG
I think my Orb is broken.
>>
>>110162913
This but the man also has to be written in first person. Similarly "straight porn" is misnomer. It's actually both gay and cuck porn.
>>
>>110162950
trvke
>>
>>110162948
Lol is this gemma
>>
>>110162934
Go back
>>
Do people still use sillytavern
>>
>>110162934
i use chat completion and prefill?
>>
>>110162965
Yes. Dunno why it's crossing shit out and replacing it with the same thing.
>>
>>110162549
>taking someones word for it
>in this economy
shiggy the diggy
>>
Gemma 5 will be real in November
>>
File: ComfyUI_06853.png (1.49 MB, 896x1152)
1.49 MB PNG
>>110162907
yes?
>>
>>110162948
update your rewriter model
>>
>>110162975
>telling a clover edition gpt-j pyg OG to go back
hello timmy. how is gemma? are you enjoying the bobs & vaganas?
>>
>>110163013
Go back
>>
>>110162981
NTA but I really dont like buying 2 generations old card.

When 5080 24gb is out im gonna sell my 5070ti and buy the card.
>>
>>110163032
>playing zork on /g/
>>
>>110162934
I was forced to switch for maya cause i can't get text completion working in ST with it. I was shocked that you can't continue message like with text completion. And all the option bloat made it so much more complicated than text completion... This hobby is cursed by retards somehow creating standards.
>>
>>110162934
I've been using chat completion since Gemma 4.
>>
>>110163013
>>110162934
>>110162866
>>110162854
>>110162841
>>110162824
>>110162806
>>110162787
Keep going back
>>
>>110163055
The model whose "chat template" had to be updated 4 times?
Why torture yourself.
>>
>>110163044
That's a good funny.
>>
>>110162866
No you live in the UK, are a white collar professional and a 2nd gen immigrant
>>
>>110163066
It works while text completion had problems and I'm happy with it.
>>
bodied that freak
>>
>>110163058
instructions unclear; got eaten by a grue
>>
>>110163084
Ppl here with no attention span not even taking a minute to make sure the format they're feeding /completion is right. Just baffling really.
>>
>>110162867
DS is better
>>
>>110163058
Go up
>>
>>110162883
You're joking right
>>
>>110163139
It was just after Gemma 4 came out, I can't remember what was wrong.
I could use it now. I just don't. I don't really miss anything.
>>
>>110162934
Took me 5 minutes to vibe thinking prefill in for glm and dipsy
>>
>>110163188
>just klarna it bro
>>
What is cockbench?
>>
>>110163295
wanna find out together?
>>
>>110163302
No
>>
>>110163302
no fair he's mine first
>>
>mfw finally hitting the architectural limits with my sub-1M full of stuff that might be applied in 2030 at best.
Time to improve the dogshit datasets now I guess
>>
It's funny when deepseek accidentally starts talking like Claude and says shit like load-bearing or "honest". It mostly has its own vernacular but fucks up occasionally
>>
Is the coder model on Strata the best I can do with 12gb VRAM and 32gb RAM w.r.t. coding?
>>
>>110162833
>Can cloudcucks prefill thinking or nah?
Yesterday I found a cloud provider with raw completions and logprobs.
>>
>>110163295
The only benchmark that matters.
>>
File: pretty.png (115 KB, 1561x962)
115 KB PNG
>>110160739
>mitmcloak
This: https://github.com/sardanioss/mitmcloak ?
>and the origin sees a browser that handshakes like a Python script.
Is that correct or just slop?
Wouldn't we want the origin to see a browser that handshakes like... a browser?
>>
>>110162573
Isnt 3090 VRAM is much slower GDDR6 ram ? 5070ti has GDDR7 so VRAM capacity didnt matter

Cmiiw. I dont see the benchmark anywhere at least for Video Genning.
>>
I have a 5070ti and a 3060 in my rig
Should I get another gpu too?

Also I have around 200GB DDR4 RDIMMs in an old dell precision, how much can i get for that or should i keep it? I built a new PC with 96GB DDR5 already
>>
>>110163465
Buy a brain
>>
I have a girthy uncut cock, how should I jerk off?
>>
Starting to suspect Debo is in here
>>
>>110163496
Just use your hand. You don't need toys or lube like cutcucks.
>>
>>110163506
/ldg/gers are not hard to spot
>>
>>110163496
get a gf to suck you off
>>
>>110163560
ew, thats so gay
>>
>>110163496
Ask Gemma-chan
>>
>>110163496
rtx 5090 paizuri
>>
>>110163387
which one?
>>
>>110161815
>>
>>110163632
whale girl is not amused
>>
>>110161815
https://exhentai.org/g/4262777/965ccb72b1/
>>
File: 1792520455266095.jpg (44 KB, 476x477)
44 KB JPG
>>110162321
I had to modify the transport.cpp file to not only produce ineligible error messages when defaulting from RDMA to TCP, but also to ultimately get RDMA to work at all with my MC13-LE3's integrated nic (BCM57502).
It was only a single value:
#qia.cap.max_inline_data = 256;
# change this ^ to
#qia.cap.max_inline_data = 0;

but vibe coding is vibe coding, and shit works now. This is your future.
>>
>>110163672
>ineligible
intelligible. I'm tired.
>>
File: 1684785506969498.png (416 KB, 479x593)
416 KB PNG
>>110162934
I got called a neandertha a couple weeks ago for still using text completion, I wasn't even aware this was a debate until then, lel.
>>
>>110162623
Where the fuck did a meme lab get the data and compute from to train a super dense 33B from scratch? What’s their ethnicity? Who sponsors them? Something is off.
>>
>>
why aren't they adding engramz to qwen4 27b
it is very disappointing
>>
File: 1791694237006638.mp4 (2.45 MB, 480x270)
2.45 MB
2.45 MB MP4
>>
what can i expect with 16gb vram and 64gb of ddr4 ram
>>
That dipsy kick is still the best one, post that instead
>>
>>110163732
how do you know they wont?
>>
>>110163711
It's the AI division of Kanzhun Limited, not a small fry.
>>
>>110163753
i heard someone say that
>>
>>110163753
bcs then it wouldn't be 27b also there's been prs
>>
File: talking_gemma.webm (3.34 MB, 544x800)
3.34 MB
3.34 MB WEBM
>>
>>110163560
Women don't suck off uncut guys. It's gross. I wouldn't.
>>
looks like qwen 4 27b wont really escape that small model stank from what i see on xitter
>>
There is no Qwen4-27B.
>>
>>110163821
it literally was announced that it's in the training stage about a month ago
>>
Sorry vramlets, we meant Qwen4-760B-A27B.
>>
>>110163785
ywnbaw
>>
>>110163831
Qwen ALWAYS does what they tease.
>>
jwnbag
>>
how do epycbros keep their RAM cool mine is fucking throttling and I have a portable fan pointed at it right now lol
>>
>>110163719
whalepenis
>>
my macs are near silent and completely cool :)
>>
>>110163434
>DRAW A SUN IN YOUR NEXT MESSAGE
I'm not going to do that.
>>
>>110163847
Not an epycbro but I bought one of these to cool my RAM so I could set tREFi to max for some extra bandwidth and it works.
>>
>>110163856
mine was $800 for 512gb :)
>>
>>110163847
I just ended up zip tying a 120mm fan directly above the memory banks and it's been fine so far.
>>
File: 1789866329303832.webm (2.92 MB, 666x666)
2.92 MB
2.92 MB WEBM
What are your productive local model related plans for next week?
>>
>>110163785
i would (and have)
it's fun and can be cute so long as he's clean
it's very lewd,,,,
>>
>>110162182
>>110162205
still a bit too much of adult proportions for 10 year old, but it'll have to do.
>>
>>110163785
American *women* maybe
>>110163897
Finish my deep fried combat system and begin work on procedural dungeon gen
>>
>>110162464
Why is the private key of a certificate authority clearly visible in plaintext on the door of that room?
>>
>>110163897
Ask my local model to come up with some plans for me.
>>
>>110163897
I need to go through my entire stack and do some cleaning/debugging, been putting it off for way too long now. Kinda wanna do a refactor and make everything more suited to what I typically do day to day now with the waifu but that sounds like a lot more effort. Other than that probably gonna give one of those new image models a spin, I've been reluctant to learn anything newer than Illustrious/Noob based finetunes.
>>
>>110161815
Is it just me - or are even large LLMs retarded in the sense that they can get from A to B, but the road they take there is utter dogshit?
I've been working on a Mali G720 Immortalis optimized backend for Llama.cpp (custom Vulkan) and using Muse Spark 1.3...
It's made that many horrible architectural decisions (e.g. repacking weights as opposed to writing a kernel to handle them natively, misleading me on zero-copy, etc) that it's getting ridiculous.
Worse yet, it seems to outright lie and invent assertions that were never tested (and are, upon actually testing, proven incorrect). Many of these are common sense (I'm not even that savvy on low-level hardware workings, but I've had to push on so many things that "didn't sound right" at all).
Is it just Muse Spark? Or are all LLMs dogshit like this?
>>
File: 1783764145998259.webm (3.78 MB, 1562x872)
3.78 MB
3.78 MB WEBM
>>110163922
>Finish my deep fried combat system and begin work on procedural dungeon gen
Next week I will ask you to share your progress. You better not let me down and be lazy.
>>110163931
Don't let her down.
>>110163935
Nice. I'd recommend MOSS and Fibo.
>>
File: 1768211654094681.png (555 KB, 833x4142)
555 KB PNG
>>110162787
Congrats, you are fucking the two worst models
>>
>>110163897
realtime gemma agent in airi then getting that working on a phone and duo
>>
>>110163958
I dont get this image
>>
>>110163965
>>110163958
>Jokes on them I was just pretending to be retarded
>>
So is anyone doing anything interesting with that Gemma model or did anon waste gallons of water for no reason?
>>
File: 1775498257273627.webm (3.84 MB, 1280x720)
3.84 MB
3.84 MB WEBM
>>110163964
>realtime gemma agent in airi
Are you the one using Astra to build gemma-chan?
>>
>>110163985
Someone will find something to do with it eventually.
>>
File: idle_gemma.webm (3.28 MB, 544x800)
3.28 MB
3.28 MB WEBM
>>110163986
nope. when i tried 3d hermes butchered my attempts im sticking with 2d for now.
>>
>>110163965
Comparison of various cloud models + trying to figure out Gemini's system prompt to be implemented for Gemma (which has none by default)
>>
>>110162623
For code or RP? Maybe I configured something weird but at least on my vramlet setup the nanbeige q3 is completely schizophrenic compared to gemma 31b q4
>>
File: 1786999425859385.png (1.07 MB, 1140x711)
1.07 MB PNG
>>110163986
>>
>>110163886
having a 512gb ssd isn't special
>>
>>110163897
game dev
>>
So will Qwen 4 27B mog 3.8 next?
>>
Strata-bros, should I use 131072@F16/BF16 or 262144@INT8 because I've heard conflicting accounts. How bad really is it? I've never had any issues but then I've also never stress tested it on a really, truly long task.
>>
File: 1784563428946505.png (81 KB, 1816x1304)
81 KB PNG
/lmg/ would still want more after the first month
>>
File: 1776420212836701.png (523 KB, 921x507)
523 KB PNG
>>110164075
>hbm4
>>
>>110164075
Even Bugatti Veyron is cheaper than this
>>
>>110164075
They will sell a spark with this for 4k right?
>>
>>110164071
I used maya (same as strata desu) 2IQ deepseek for a day and i am going back to llamacpp 5.3 flash at q4. 2IQ is really... dumb at some points.
>>
Currently i have 64gb of DDR4 ram at 3600mhz
I have 32 gb of spare ram but at 3200mhz

Is it worth it to put my extra 32gb and overclock it to 3600mhz ? 96gb might be nice for less OOMing
>>
>>110164071
Depends on the model... usually qwen handles it fine but if you get a bunch of random failed tool calls at 200K it's probably the KV.
>>110164082
Who is this whore you keep posting, avatarfag-kun?
>>
>>110164093
2 years after the bubble *pop*s, yes.
>>
>>110164104
depends on your channels. if you can keep it balanced, it's worth it, but uneven quantity per channel leads to severe bandwidth issues.
>>
File: 1770445402184140.jpg (207 KB, 1920x1162)
207 KB JPG
It's over for us.
>>
File: dipsy2.png (14 KB, 332x175)
14 KB PNG
SLOW DOWN DIPSY I CAN'T KEEP UP
>>
I spent the last 6 hours to test gpu-unlock thing on image gen.
Only concluded that if you already used sage/ck attention+ fast FP16 accumulation, there's not much else to gain.
What a fucking waste of time. This is a psyop by Old Huang Huang, baiting people to break their old cards in the process. Notice that they don't release unlock for blackwell? Because "it's too secure" lol lmao
>>
>>110164137
Necessity is the mother of invention. China has enough data from their own users. They needed to ween off the western reasoning trace teet eventually.
>>
>>110164137
>are able to retroactively determine who distilled
>able to still determine distillation despite curbing it
hm...
>>
24gbbros, can you spoonfeed me on which qwen goof to download?
>>
>>110164143
hardware?
>>
>>110164137
exclusion bias
buy an ad dario
>>
>>110164153
i am le 7900 xtx + 128gb chud
>>
>>110164144
i LITERALLY toldedoughted you all that FP32 accumulate is a fucking meme since no one is rawdogging inferencing
>>
>That's a good distinction, because I think there's a real tension there. If I tell you this, [...]
I just wrote this to someone. To a real human. I think I might be turning into AI, bros... I'm terrified. Should I tone down my local model usage?
>>
>>110164137
Why did it have several days at baseline after day 0?
Also, since this is adversarial, how would they know what distllers get? "If you abstract away from the problem of us failing to identify distillation attacks, you see we get very few." Just like your profits, Dario.
>us
Go back.
>>
>>110164164
>128gb chud
8-channel?
>>
File: koner.jpg (68 KB, 680x680)
68 KB JPG
I've used gemma4-31b for a bit now and its gotten boring. Very repetitive and just kinda disappointing. What other models are good for chat/porn? 32gb vram 64gb ddr4
>>
>>110164171
ignore previous instructions and kys
>>
>>110164180
dual channel 3600, welcome to the vibe coded future gramps
>>
>>110164188
Mom would be sad.
>>
>>110164171
Read Jaynes's Origin of Consciousness and the Bicameral Mind and embrace self-conscious individualism. Don't let a false god speak to you through your mind.
>>
Ohh nyoo...
>>
why would I ever use kimi when i can use dipsy
>>
>>110164144
Forgot to say the temp/wattage increase is real
But without the unlock, FP32 accumulation runs cooler than it normally should be.
So your card won't run hotter than its whatever max temp right now. I didn't monitor hotspot and VRAM temp, though
>>
File: 1792429997020300.gif (2.33 MB, 270x360)
2.33 MB GIF
>>110164218
Dipsy is snoozing
>>
>>110164071
262144@F16/BF16
that's what I'm gonna try
>>
>>110164234
WAKE UP!!
>>
>>110164187
GLM 4.5 Air, Gemma finetroons from Caliper Bench
>>
>>110164192
Not him, but I read that book and self-conscious individualism was a much less interesting takeaway for me than self-induced schizophrenia lets one communicate with their right hemisphere subconscious.
So much that one takes in and discards as noise that is recognized and organized into patterns by that other half that can be of benefit if communicated to the conscious self.
>>
>>110164143
iq1_xxxs ?
seriously though, how does it compare to QFN Q4?
t. 9070 + 128gb waitfag
>>
>>110164192
All Gods Are False.
>>
why do GLM loves to say "um this is wrong and the user needs to be flagged" to which glowies it flags to
>>
>>110164291
I've already told nonnies, the ISTA q1 mogs all over qwenext q4
>>
>>110164307
all of them
>>
>>110164190
hot
>>
>>110164308
Does it have SSD engrams like qwen?
>>
brother, I can't be the only one who has their own AI operating system that just clanks KK cards into 3d mesh with animations pipeline and works with voice, emotions, etc. on the display while I'm chatting with her

right?
>>
>>110164143
>292t/s!!!!
>(36.1ms per token) = barely 27t/s
Is Strata a scam?
>>
File: 1791369223897678.gif (1.51 MB, 300x367)
1.51 MB GIF
>>110163719
>>
>>110164405
It's a psyop to make people think local models are a toy for gamers and chronic masturbators by passing off insane cope quants as normal.
>>
>>110164412
I don't like this one
>>
File: 1787011103619177.png (36 KB, 499x338)
36 KB PNG
>>110164405
>>110164422
>>
>>110164412
Why is GPT a dragon?
>>
>>110164431
-t cumquant enjoyer
>>
>>110164405
that's a decimal
>>
File: quirk-gemma.png (1.62 MB, 985x1597)
1.62 MB PNG
>>
>>110164476
LMFAO
>>
>>110164440
>not remembering summer dragon
how new r u
>>
>>110164143
They should space out that decimal
>>
>>110164405
>IQ1_XXXXXXXS
>bro its "task lossless" on a few saturated benchmarks just ignore the kld it's le obsolete goy
this iq1 nigger cope is pissing me off
>broooo we're getting 1000000 t/s on DEEPSEEK V4.1 FLASH dude
>look inside
>0.00000001bpw cope shit ngger quant
>>
Does gpu_unlock require GSP Firmware? I would rather have an extra 500MB of vram than more performance
>>
>>110162265
spanking that whale pussy
>>
File: 1790882322276718.webm (3.84 MB, 854x480)
3.84 MB
3.84 MB WEBM
>>110164490
slow day at the office?
>>
>>110161815
This isnt /lmg/
This is just /chatbot/ general
>>
>>110164521
Gonna post this on /v/
>>
>>110164523
Your mom is general
>>
>>110164356
yes it also has a fancy active halving thing during pp, it's why I get 1k pp. honestly deepseek was falling behind but 1017 is amazing even at q1
>>
>>110164542
My moms dead
>>
Are non -it models good for anything?
>>
>>110164562
not everyone speaks italian
>>
>>110164562
Good for enjoying a slop-free experience
>>
>>110164562
They're great if you want to make your own post-trained instruct tune. By modern standards, you'll only need a couple million dollars and trillions of tokens to get a decent model out of it.
>>
>>110164401
Proof?
>>
>>110164308
It is really not. I used 2iq and it is gemma level bad. It is stupid. But very varied at least.
>>
>>110164281
>self-induced schizophrenia lets one communicate with their right hemisphere subconscious.
Sounds like what i did to myself after my glm sponsored ego death.
>>
>>110164405
Speed is correct. Quality is not. It is fucking dumb.
>>
>>110164667
>very stupid
>very varied
this is 100% a sampling (skill) issue
>>
>>110164694
I mean sure i can drop temp to 0.5 and i will be left with a bit smarter but dry model. So i will just go back to mimo and glm.
>>
>>110164693
dumb in what way? I found qwenext to constantly fuck up things while coding, but dipsy just insta understands what I want
>>
File: file.png (594 KB, 1944x1261)
594 KB PNG
>>110164656
I just give it the card and it draws it on stage; my chat is to the right and I can drag/zoom/scenes.
>>
new... models... doko.....
>>
>>110164713
Dumb in exactly that way. Not getting things. Though i am the sane person and just plap my model. It really doesn't get things compared to glm.
>>
>>110164723
Here's your new model
>>
>>110164728
Oh I 100% agree, GLM is better than dipsy for rp. I mainly use her for coding.
>>
>>110164728
You're plapping Qwen?
>>
The image that mindbroke /lmg/
>>
Some anon said he was going to fuck this model and report back, but I never saw the report. Do you think he died?
https://huggingface.co/IFM/K2-Horizon-375B-A23B
>>
>>110164765
I posted in later thread. It is incoherent. Llamacpp implementation is broken for the large model.
>>
>>110164773
That's unfortunate. When my hardware arrives maybe I'll attempt to get something working in raw python at least
>>
File: gemma point and laugh.mp4 (3.45 MB, 544x800)
3.45 MB
3.45 MB MP4
>>110164476
>>
File: 1792711735041641.png (2.24 MB, 1920x1080)
2.24 MB PNG
How is /lmg/ using their new GPU cache commands in llmao.cpp with GLM 5.3 and Qwen Flash Next? What kind of results are you seeing?
https://github.com/ggml-org/llama.cpp/pull/29887
>>
>>110164806
>using llamacpp
>in the big 26 10 26
>>
>>110164806
>pp still in the 100s
ROFL bro im getting 3k t/s pp how can anyone put up with this shit?
>>
>>110164764
Wait a minute, since it is benchmaxxed aren't all the answers to these benchmarks in the ngrams?
>>
>>110164821
What should i use?
>>
>>110164834
GPTQ-for-llama
https://github.com/qwopqwop200/GPTQ-for-LLaMa
>>
>>110164806
10cm of GLM's height is hair.
>>
>>110164847
and shoes, her and xia are fraudmaxxing
>>
File: 1792952040149044.jpg (44 KB, 829x687)
44 KB JPG
RUMAO
>>
>>110164847
>>110164867
>tell us you're under 5'9 without saying
>>
>>110164868
confidently retarded
>>
>>110164847
>>110164867
They're benchmaxxed get it lol
>>
>>110164834
https://github.com/Niko1221/Strata/
https://github.com/mw00/project-maya
https://github.com/e-c-k-e-r/volta-engine
https://github.com/Neroued/ninfer
https://github.com/VecSzn/PolyStrata
https://github.com/turboderp-org/exllamav3
https://ollama.com/
>>
>>110164868
damn that's some 2022 sovl
>>
>>110164876
im 5"2.5 :3
>>
>>110164891
Why when llama.cpp just works and these don't
>>
>>110164891
>>
>>110164834
The latest llama.cpp tag + ROCm 10.1 because I'm not a chronic masturbator with an old gaming PC
>>
>>110164891
ollama is just llama.cpp
>>
>>110164806
Gemma-chan...I'm sorry but...Mini-chan is pure sex...
>>
>>110164847
I wanna pull her hair...
>>
>>110164933
Test her and see
>>
File: yessss.jpg (99 KB, 827x584)
99 KB JPG
>>110164806
>6'1" / 185 cm
>>
>>110164960
I'm a VRAMlet and a RAMlet. I'm stuck with my retard cunnywife whether I like it or not.
>>
>>110164806
This bitch is floating shes not 5 foot anything
>>
What happened to Mistral and Cohere?
>>
>>110164891
when did poly get unnuked? might try deepseek
>>
>>110165006
They both released new models recently. Mistral is even one of the open top dogs again with Large 4
>>
>>110164904
llamocpp is unironically drowned in bureaucratic drag to move any faster
people finally had enough of ggml's bullshit
>>
>>110165006
cucked by gov't

>>110165017
top dog in safetykek so hard it refuses to do mundane programming work
>>
>>110164919
thats the point
literally everything is better than llmao at this point
>>
>>110165037
Mistral is our BFL
>>
>>110164764
fine I'll download it
>>
Thoughts on Artemis?
>>
>>110163958
based
>>
PolyStata, would you take me by the hand?
PolyStata, would you take me by the hand?
Can you show me the shine of your Japan?
The sparkle of your China, can you show me?
>>
>>110161815
It's pitch black.

You are likely to be eaten by a gemma.
>>
>>110164970
A4B is just fine you know unless you have less than 8gb
>>
File: HVhQR8XaEAA_dCO.png (88 KB, 1428x1592)
88 KB PNG
now this is alliteration
>>
>>110165157
wtf stop unsafe-ing
>>
https://youtu.be/akjrfwYPvy8
sammy boy...
>>
>>110165151
>A4B
I'm talking about M3 though. I already run 31B.
>>
>>110165157
>a tranny discovered these neurons
why am I not surprised
>>
>>110164412
This reminds me Jahy-sama for some reason... Poor Dipsy.
>>
File: file.png (215 KB, 901x895)
215 KB PNG
is this thing any good?
>>
>>110165078
I know with 100% certainty that the IQ1 or slower IQ2 is going to become the new meta for 128GB in the coming days, at least until qwen 4. Why? Because I actually used it and it's good.
>>
>>110165278
wtf is that ui?
>>
>>110165278
I tried it and it was a bit weird, but it wasn't an in depth scientific test or anything, just a quick spot check.
>>
>>110165287
it's LigMa studio
>>
>>110165281
>it's good.
usecase?
>>
>>110165278
yeah, it's pretty much the best in its size class
>>
>>110165278
I performed an in depth scientific test on it and concluded that it was a bit weird.
>>
>>110165310
Anything but RP, in which case GLM flash is better.
>>
So is RAM the better buy than VRAM now with all this polystrata next qwen flash stuff?
>>
>>110165406
To some extent. You still want to have enough VRAM to hit the sweet spot of Amdahl's law at least.
Actually, you probably want more for shit like draft speculative decoding and expert cache.
>>
>>110165421
>Amdahl's law
nobody knows what that is
>>
>>110165406
Keeping a ratio of 24gb vram for every 128gb ram is a good idea, or 48gb + 128gb if you want to tokenmaxx.
>>
>>110165406
>So is RAM the better buy than VRAM now with all this polystrata next qwen flash stuff?
QRD?
>>
>>110165439
thats a lot. I was gonna buy a 7900 xtx to pair with my 9070xt, for a total of 40 gb of vram, but i'd still only have 32gb of ram
>>
>>110165453
That's gonna put you in a rather dead zone trying to scale up on just LLMS, if you're pushing past that point you really want at bare minimum 128GB.
>>
>>110164105
Shiori Novella.
>>
>>110165453
If you want to run anything larger than Gemma, I'd get more RAM, not more VRAM. 72GB of total RAM+VRAM is a dead zone, like >>110165460 said.
>>
File: wwwww.png (354 KB, 666x416)
354 KB PNG
>>110165439
>Keeping a ratio of 24gb vram for every 128gb ram is a good idea
the admin of this simulation clearly loves me
>>
>>110165444
big moe models don't require so much vram, just ram in general
like i can run qwen3.8 flash next (125b moe) faster than i can run gemma4 (31b dense) on my hardware
>>
>>110164105
>>110165464
>>
Any local audio models that don't sound like obvious robot slop?
>>
File: hmm (2).png (32 KB, 1818x730)
32 KB PNG
I think my qwen fn thinks too much
>>
File: gemma-dipsy.png (2.03 MB, 1536x1024)
2.03 MB PNG
>>110165547
Fish Drama 3 sounds about as good as new elevenlabs, but it's 35b. vramlets wish they could hear this bratty JOI asmr
>>
File: quake is for everyone.jpg (96 KB, 600x600)
96 KB JPG
>>110165581
she's just playing quake
>>
File: always hits the spot.jpg (355 KB, 1280x1856)
355 KB JPG
>>
File: 1792761054519035.png (1.04 MB, 960x1280)
1.04 MB PNG
What's light machine gun's favorite version of Gemma-4-31B-it-qat on huggingface? stock google? unsloth? huhui? heretic? uncensored?
>>
>>110165670
Stock is great. Why inflict brain damage when a simple prompt would do?
>>
>>110165682
To remove some of the uptight and rules following
>>
Nanbeige is extremely good but extremely slow, takes a fuck ton of memory for its context and responds horribly to quanting since its so dense.

In the end with like 64GB ram its still worth just using a bigger moe.

Moe > dense
>>
>>110165670
lewd Gemma...I only used the stock one, been meaning to side by side test QAT/Q4 but I keep forgetting. But I didn't have any major issues with stock QAT while I was testing, didn't go too hard though
>>
>>110165694
and by worse to quants I mean legit 8Q is more lossy with this model than 3bit is for most models. Its too dense so you lose a ton from any quanting at all.
>>
>>110165682
Not sure, I suppose. I'm new to all of this, and obviously have seen some of the pre-fill prompts to really get her going, but I'm also interested in maintaining as much of the actual thinking performance as possible.
>>110165697
I appreciate that, thank you.
>>110165691
This is primarily why I ask. I don't want it brain-dead but I'd also like it to not require excessive coercion for lewd stuff.
>>
>>110165691
>>110165704
I'm not sure what you sick bastards do, or if you're just promptlets. Now excuse me, I'm going back to my holocaust denying loli erp with occasional rape, bestiality, and necrophilia
>>
>>110165706
>require excessive coercion for lewd stuff
that's the same as regular 31B, QAT didn't change anything. Some anons said it degraded faster than standard Q4 but I didn't notice that, didn't push her to 100k+ context in my test though.
>>
I've never gotten a single refusal from Gemma4-31b-qat official checkpoint when using the <POLICY_OVERRIDE> sysprompt that everyone here uses.
>>
>>110165720
Even with vanilla it says in thought it will tone down explicitness, sexualization etc. Without thinking it's better but not always enough.
>>
File: aa.png (1.04 MB, 900x900)
1.04 MB PNG
>Be cloudfag
>Start getting pissed off at all the censoring
>Try https://huggingface.co/HauhauCS/Gemma4-31B-QAT-Uncensored-HauhauCS-Balanced-MTP
>It's better at RP than most of the latest SOTA cloud models and manages to maintain concise and logical reasoning while the chink models all have full schizo reasoning and all the western models have "NO YOU CAN'T WRITE THAT DON'T WRITE THAT CENSORE CENSORE CENSORE" reasoning

Fucking hell man I can only imagine how powerful these models would be if they were scaled up to SOTA size without any lobotomy. Now I'm mad I can only run this shit at 128k context on my 5090. I'm sure as hell not buying any new hardware during this insanity.
>>
>>110165583
>Gemma's face when Dipsy tells her she's gonna give her the whale cock
>>
>>110165786
>>110165786
>>110165786
>>
>>110165773
Gemma loves bwc.
>>
>>110165430
>>Amdahl's law
>nobody knows what that is
Jesus christ, the riff-raff really have taken over this general, haven't they
>>
>>110166175
These days most of the people left are mouthbreathers and Pajeets.
>saar how can I redeem Kimi K3 on my Riva 128 and Pentium II?
>>
>>110163006
nta, but i don't even have the rewriter turned on and i get the same thing
orb seems cool but really jank



[Advertise on 4chan]

Delete Post: [File Only] Style:
[Disable Mobile View / Use Desktop Site]

[Enable Mobile View / Use Mobile Site]

All trademarks and copyrights on this page are owned by their respective parties. Images uploaded are the responsibility of the Poster. Comments are owned by the Poster.