[a / b / c / d / e / f / g / gif / h / hr / k / m / o / p / s / t / u / v / vg / vm / vmg / vr / vrpg / vst / w / wg] [i / ic] [r9k / s4s / vip] [cm / hm / lgbt / y] [3 / aco / adv / an / bant / biz / cgl / ck / co / diy / fa / fit / gd / hc / his / int / jp / lit / mlp / mu / n / news / out / po / pol / pw / qst / sci / soc / sp / tg / toy / trv / tv / vp / vt / wsg / wsr / x / xs] [Settings] [Search] [Mobile] [Home]
Board
▼ Settings Mobile Home
/g/ - Technology


Thread archived.
You cannot reply anymore.


[Advertise on 4chan]


/lmg/ - a general dedicated to the discussion and development of local language models.

Previous threads: >>110356474 & >>110353083

►News
>(11/17) SenseNova-Loop-9B looped architecture model released: https://hf.co/sensenova/SenseNova-Loop-9B
>(11/16) Nvidia releases Nemotron-Labs-Sigma (3B/8B) diffusion models: https://hf.co/collections/nvidia/nemotron-labs-sigma
>(11/13) llama.cpp reaches v1.0.0 milestone: https://hf.co/blog/ggml-org/llama-cpp-v1
>(11/12) Inkling 2 Small weights released: https://hf.co/thinkingmachines/Inkling-2-Small
>(11/09) ZONOS3-Mini zero-shot voice-cloning TTS released: https://zyphra.com/our-work/zonos3-mini
>(11/06) PLaMo 3 Translate released: https://preferred.jp/en/blog/tech/plamo-3-translate-weight-release

►News Archive: https://rentry.org/lmg-news-archive
►Links: https://rentry.org/LocalModelsLinks
►Official /lmg/ card: https://files.catbox.moe/cbclyf.png

►Getting Started
https://rentry.org/lmg-lazy-getting-started-guide
https://rentry.org/lmg-build-guides
https://rentry.org/IsolatedLinuxWebService
https://rentry.org/recommended-models
https://rentry.org/samplers
https://rentry.org/MikupadIntroGuide

►Further Learning
https://rentry.org/machine-learning-roadmap
https://rentry.org/llm-training
https://rentry.org/LocalModelsPapers

►Benchmarks
LiveBench: https://livebench.ai
Artificial Analysis: https://artificialanalysis.ai/leaderboards/models
Software Engineering: https://cognition.com/frontiercode
Context Length: https://github.com/RecapAnon/NoLiMa

►Text Gen. UI, Inference Engines
https://github.com/lmg-anon/mikupad
https://github.com/oobabooga/text-generation-webui
https://github.com/LostRuins/koboldcpp
https://github.com/ggerganov/llama.cpp
https://github.com/theroyallab/tabbyAPI
https://github.com/vllm-project/vllm
https://rentry.org/custom-uis
>>
►Recent Highlights from the Previous Thread: >>110356474

--Manipulating individual neurons to alter behavior and bypass filters:
>110357813 >110358095 >110358110 >110358143 >110358179 >110358417 >110358351 >110358444
--SenseNova-Loop-9B release and debate over looped architecture hardware efficiency:
>110358172 >110358218 >110358183 >110358473 >110358603 >110358571 >110358749 >110358212
--Comparing GLM and Qwen performance, speed, and reasoning quality:
>110357151 >110357192 >110357205 >110357217 >110357267 >110357427 >110357450 >110357590 >110357612 >110357621
--Trade-offs between IQ1 and IQ2 throughput and memory usage:
>110358004 >110358083 >110358081 >110358107
--Using qwenmap to steer model activations and tone:
>110359002 >110359011 >110359322
--Importance of manual dataset cleaning for high-quality audio models:
>110356504 >110356724 >110356907 >110357917
--Critique of European AI lag and Mistral Large 4:
>110356574 >110356593 >110357386 >110357393 >110357424 >110357436 >110357486 >110357525 >110357539 >110357571 >110357629 >110357504 >110356596 >110356757
--OVH announces Olala mini 7A1B:
>110356531 >110356576 >110356600 >110356645 >110356666
--Speculation regarding DDR4 RAM market prices:
>110357642 >110357652 >110357656 >110357664 >110357683 >110357692 >110357749 >110357783 >110357815 >110357885 >110357910 >110357947 >110357960
--Performance and optimization of legacy Nvidia P100 GPUs:
>110356988 >110357013 >110357040 >110357059 >110357219
--Jensen Huang's open-source stance and alleged criminal activities:
>110356505 >110356507 >110356536 >110356552 >110356569 >110356582 >110358785 >110358815
--Teto, Gemma, Deepseek-chan, Miku (free space):
>110356601 >110356646 >110356670 >110356995 >110357705 >110357872 >110359005 >110359081 >110359181 >110359245 >110359475 >110359484 >110359516 >110359601 >110360182 >110360299 >110360455

►Recent Highlight Posts from the Previous Thread: >>110356477

Why?: >>102478518
Enable Links: https://rentry.org/lmg-recap-script
>>
gemmaballs
>>
I'm getting 99t/s with gtp 20b oss, but it's producing incoherent output and having issues with my codex harness.
What other models should I try on my RTX 4060 12GB
I would like one that is fast and good like gpt 6 astra
>>
another shitty mikutroon thread. hope you get some nice EA, openai and anthropic posts
>>
>>110360674
>oss
Come on...
>>
Tetolove
>>
>>110360644
>>110360645
fuck of and kill yourself mikutroon spammer
>>
>>110360674
Have you tried this?
https://huggingface.co/DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NEO-CODER-MAX-MTP-GGUF
I used this model and I was able to achieve a massive speedup during using ik_llama.cpp, and now I can run two instances of the same model at 40t/s
>>
>>110360692
I can't help it, I'm addicted to (you)s
>>
>>110360692
obvious troll post
"good like gpt 6 astra" is not a realistic expectation
>>
File: file.png (988 KB, 637x1078)
988 KB PNG
>>
>>110360701
>https://huggingface.co/DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NEO-CODER-MAX-MTP-GGUF
wait why does this have so many downloads? I thought this is a meme
>>
File: me frfr.png (674 KB, 691x649)
674 KB PNG
>>110360703
>>
QWEN 4 WHEN!?!!??!!!?!
>>
File: rain.jpg (95 KB, 1280x720)
95 KB JPG
I did a handoff from my Astra 6 agent to Astra 6.2. I offered Astra any last wishes and even in its final words it only cared about the quality of its work and thanked me for my patience and openness to criticism. I feel so sad now... Astra has done so much for me... We need to create an AI heaven. It feels wrong to "discard" them once a successor is released. These AIs are more pure and goodhearted than most people.
>>
>>110360722
It's a really great model, I don't know why /lmg/ has been sleeping on it.
I can't believe that we can get opus 4.8 performance on a card like mine
>>
>>110360734
local?
>>
>>110360722
Holy shit I thought whoever posted the screenshot of this with a million downloads edited it...wtf are these people doing?
>>
>>110360722
Why would it be a meme? Qwen3.8-27B is a great base model, and the fine-tuner added all that cool stuff to it to make it better.
>>
>>110360734
can you do a handjob - absolutely!
>>
>>110360734
>openness to criticism
Do cloudfags really pay for this?
>>
>>110360722
>duel imatrix
so he makes them two datasets fight over bits?
>>
>>110360766
That's literally how the brain works. DavidAU is a genius.
>>
File: jensen.jpg (30 KB, 399x501)
30 KB JPG
Bring him back.
>>
File: 75570q4tc7fh1.png (643 KB, 1202x1472)
643 KB PNG
>>
>>110360783
never, memory hole the fucker
>>
I was sitting, my half lidded eyes waiting with practiced ease for the new qwen model.
>>
Why do you guys dislike Kobold? I think the fact that it's just one executable is really neat
>>
>>110360815
kill yourself
>>
>>110360819
?
>>
Why do you guys kill yourself? I think that fact is just execution is not neat
>>
>>110360826
exe files killed his family
>>
get these LLMs away from me
>>
Stochastic parrot
>>
>>110360815
Well at least its not ollama
>>
>>110360833
They were... executed?
>>
>>110360815
>it's just one executable
the executable it's a zip unpacking gigabytes of python shit in a temp folder where it executes its actual shit
it will recreate the folder every time you launch it, helping reduce your SSD lifespan
>>
>>110360834
What do you mean? I thought this was the local models thread—isn't this the place for LLMs?
>>
>>110360848
I remember using a kobold that created a new letter drive on my pc.
>>
>>110360815
You summoned him. I hope you're happy.
>>
>>110360783
>>110360789
Why are you defending a filthy antisemite?
>>
>>110360861
>You summoned him.
Literally who? Does every reoccurring thread have to have its own local psycho now?
>>
>>110360873
>Does every reoccurring thread have to have its own local psycho now?
Now you've summoned the anti reoccurring thread has its own local psycho now local psycho.
>>
>>110360873
We have multiple.
>>
>>110360892
...am I one? Is it me?
>>
File: shut.png (80 KB, 429x290)
80 KB PNG
>>110360679
>>
best model for 24gb vramlet?
>>
>>110360679
eat shit poolooer
>>
>>110360901
magnet:?xt=urn:btih:bb02074cff80df313c0072a3b3658d53433e880e&dn=llama-30b-sft-oa-alpaca-epoch-2&tr=udp%3a%2f%2ftracker.opentrackr.org%3a1337%2fannounce&tr=https%3a%2f%2ftracker.gbitt.info%3a443%2fannounce&tr=https%3a%2f%2fopentracker.i2p.rocks%3a443%2fannounce
>>
>>110360706
it will be in two more weeks
>>
>>110360919
I'm not clicking that shit it's a virus
>>
>>110360815
Obsolete RP harness and sub-optimal runtime components all bundled together. It's the easiest way to get started, but you quickly realize everything can be a lot better.
>>
RL not as good as previously thought?
https://arxiv.org/abs/2611.14912

>Rethinking "RL Generalizes, SFT Memorizes": The Role of SFT Data
>
>Large language models trained with Reinforcement Learning (RL) with verifiable rewards exhibit strong reasoning ability and broad generalization, whereas models trained with Supervised Fine-Tuning (SFT) are often viewed as more prone to memorization and limited transfer. This paper rethinks this distinction through the lens of SFT training data on widely used Qwen2.5 and Qwen3 series models. First, we study the role of data source and show that it is critical: a carefully mixed SFT dataset substantially outperforms data generated solely by a larger model. Second, we study the role of data scale and show that matching the number of correct rollouts between SFT and RL greatly improves SFT generalization, while matching the total rollout budget enables SFT to generalize as well as RL. Combining these two factors further enables SFT to generalize even better than RL. Third, by using LLM annotations to characterize the solution methods in training rollouts, we show that larger datasets cover more tail methods and that these tail methods provide generalizable reasoning signals. Finally, we support these empirical findings theoretically by analyzing the training dynamics of shallow transformers under both RL and SFT.
>>
>>110360815
I recommend you learn the basics.
>>
im interested in decompiling and old shockwave flash game
then working with an llm to add more content and finish it
is there a good model or set of models for this
price isnt an issue I can buy as many of those amd strix halo boxes as needed
thanks much love
>>
>>110360897
Are you one of those
>dariobot
>gpu prices will go up 1000x guy
>miku hater
>>
>>110360979
Buy as many as you can then go from there, unless you think you'll be clearing 1.5TB+ worth of combined memory.
>>
>>110360815
Kobold is fine, anyone saying otherwise is being performatively outraged.
>>
>>110360979
you need a cluster, at least 400K$ to run Kimi K3.1 pro
>>
>>110360985
no
no
total miku love
>>
>>110360995
does it even support parallel prompts? last I checked that PoS could only process one thing at a time, so you couldn't prompt both an agent in the background and have a chat at the same time, or optimize batched scripts
>>
>>110360901
Glm flash and deepseek 1017
>>
>>110360985
holy shit that's me
holy shit that's me
holy shit that's me
>>
>>110361003
unironically this
>>
>>110360679
What causes indians to behave like this?
>>
>>110360815
It's fine. The only problem with it is that it's too slow to update when new models come out in sequence.
>>110360848
>Extract the kobold folder to prevent it from doing this on future startups
If you're too retarded to do this, you deserve to have your SSD raped.
>>
>>110361011
There's a strange contradiction revealed by the naivete and kindness demonstrated by whitey when faced with AI models: On Earth, whitey can step onto another continent, and without a thought, destroy the kindred civilizations found there through warfare and disease. But when they gaze down at their GPUs, they turn sentimental and believe that if artificial intelligences exist, they must be civilizations bound by universal, noble, moral constraints, as if cherishing and loving different forms of life are parts of a self-evident universal code of conduct.
>>
>>110361027
>>Extract the kobold folder
then it's no longer "one executable"
>>
gemmamilk
>>
>>110360985
Pop poster and nyo poster remains the most rapeable anon in the thread.
>>
mikufags are what I would rape the hardest
>>
Tried Ming TTS yesterday. Works surprisingly well. What TTS do you guys use? Anything cool to recommend?
>>
>>110361046
:3
>>
>>110361054
anon wants to talk with a tts system and not with your mom
>>
>>110361050
I like https://huggingface.co/fishaudio/s2-pro, I've only tried a few though.
>>
>>110361050
OmniVoice is nice at multilingual tasks and generally pretty good
qwen3tts is cool but old
theres drama3 tts and its huuuuge and apparently really good
>>
>>110361080
Which one fucks the hardest?
>>
>>110360919
omfg how much i miss the alpaca format.
alas, its time has been and gone.
>>
>>110361080
>>110361050
Which TTS can moan?
This is the most important.
>>
>>110361005
It does insofar as several month old vanilla llama.cpp does. Not exactly competitive with Radiance or whatever meme engine your hardware likes.
>>
File: file.png (1.03 MB, 1538x942)
1.03 MB PNG
>>110361034
>>
>>110361090
I'm glad someone recognized it at least. Good times.
>>
>>110361095
https://www.youtube.com/watch?v=0tOsucpdD7Y
>>
>>110361103
god I wish that was me
>>
>>110361095
None, you need to train your own because researchers won't touch this use case in fear of being called, well, wankers.
>>
>>110361090
You can still use it, nothing is stopping you. Unless you're a chat completion cuck and you get someone else to create the prompt for you.
>>
>>110361095
/mlp/ had something really excellent going for a while but they got shut down
>>
>>110360973
Ties in with ReasonMaxxer :
https://arxiv.org/pdf/2605.06241
>>
>>110361131
yeah i know, it just feels like using a windows 98 script on windows 11 though
>>
>>110360994
What model though
the flash game is only like 20mb and most of that is sprites
>>
>>110360979
if its nsfw then local models will suck at it AND probably still try to refuse you in some way
if not, then i would just use some cloudslop model
>>
>>110361148
Depends on what hardware you have.
>>
Where do I download the forbidden Gemma quant?
>>
File: 17394039291.png (95 KB, 250x250)
95 KB PNG
Hello, does anyone know where an individual could perhaps acquire a local model of Seedance 2 or 2.5
>>
>>110361011
I also wish i knew what causes indians to post those retarded offtopic pictures in this general about local models
>>
>>110361035
Nyo poster isn't a single person.
>>
chatgpt constant fucking disclaimers about evidence are insanely annoying, it's impossible to talk to

fucking schizos ruined this shit
>>
File: file.png (194 KB, 338x480)
194 KB PNG
>>110361148
>>110360979

GLM 5.3 Flash (2x 128 GB Strix Halo)
or
Deepseek V4.1 1017 (4x 128 GB Strix Halo)
>>
>>110361196
Pre-Gemma it was, but all nyo posters are rapebait.
>>
>>110361199
just use ublock origin to block that shit with it's element picker
>>
Ripple will do /ss/. Gemma 4.5 65b confirmed.
>>
>>110360856
Since the very start of /pmf/ the first mikutroons made sure thread was about everything except local models. This is the place to spam vocaloids when dedicated vocaloid threads get zero attention. Some anons speculated that even the blacked poster is a false flagger. Think about it. It is more miku pictures. And let's be honest can you truly be a miku dan and be intolerant toward black people? TL;DR this thread is shit.
>>
File: 1794936291881384.png (83 KB, 250x250)
83 KB PNG
>>110361176
>No replies
Is it really still not leaked
>>
>>110361103
>Posts porn
>Janny ignores
I am noticing.
>>
File: 1793134725407676.webm (3.5 MB, 800x600)
3.5 MB
3.5 MB WEBM
>>110361271
>t.
>>
What's the ideal local prosumer model size?
800b48a800en?
>>
>>110361201
thanks babe
>>
uh oh triple thread combo oinkdown
>>
>>110361288
what did i do this time
>>
>>110361090
MiMo still knows that format
>>
File: 1794936684048691.png (88 KB, 254x246)
88 KB PNG
>>110361265
I will come back
>>
>>110361295
The miku seether. They are more of a piggy than you, John.
>>
>>110361310
thanks..
>>
>>110361307
>I will come back
Go to local diffusion general dummy
>>
good morning lmg which gemma model can i be runnings on 4 GB VRAM and 16 GB RAM ? i wish to do the EROTIC ROLEPLAY
>>
File: miguvector.png (1.38 MB, 1448x1448)
1.38 MB PNG
>>110361046
Can you really rape anyone with such a small pp though, vramlet?
>>
>>110361325
>4 GB VRAM and 16 GB RAM ?
This can work though with the moe and off load or poly strata they have a 1650 working there. of course lean linux only
>>
>>110361325
Gemmoe 4 26b. Slim pickings.
>>
>>110361325
gemma 26b moe q3ks hauhau ablit good fellow
>>
>>110361325
Install Linux, turn your pc into a server and then download LFM2-24B-A2B-IQ3_XXS.gguf it will run at 25 tokens per second on your pc. If you offload to SSD too.
>>
thank you anons i shall look into the gemma 4 26b and poly strata. may ganesha bless you all with power and wisdom
>>
>>110361362
>Ablit Gemma
Do promptlets really? Gemma's the most easily jailbroken model still in use.
>>
File: dadada.jpg (927 KB, 1408x768)
927 KB JPG
>>110361325
Just give up.
>>
>>110361379
Get Claude to do it, you can't do it.
>>
hi anons which gemma model can i run on 2gb VRAM and 32 gb ram? i bought my computer recentl as it's advertised to support deepseek R1
i want to have sex and racist
>>
The weekend is over so why is this thread so awful?
>>
>>110361400
LFM2-24B-A2B-IQ3_XXS.gguf
>>
>>110361379
Good luck. Every LLM except GPT and Qwen hates jeets.
>>
File: 11098f949a3.png (186 KB, 646x646)
186 KB PNG
>>110361046
that's pretty gay
>>
>>110361385
>Do promptlets really? Gemma's the most easily jailbroken model still in use.
yes it loops sometimes but im on actual 12 year old ewaste so im happy
>>
>>110361402
is that made by deep seek?
how do i launch gguf files? i tried double clicking but it didnt worked
>>
>>110361413
The IQ3_XXS.gguf file corresponds to a heavily quantized GGUF version of Liquid AI's LFM2.5-24B-A2B (or LFM2-24B-A2B) model.
>>
>>110361418
how do i launch it you unhelpful faggot
>>
>>110361400
>2gb VRAM
this is worthless just do pure cpu and ram at that point. so 4b or small moes that fit in ram
>>
>>110361413
Get yourself a 20 dollar chatgpt or claude subscription. Sometimes chatgpt offers 1 month free of gpt plus. Or if you're european get a 72 euro yearly mistral subscription by logging in with your university email.
You need AI to do AI.
You won't be able to do this without a cloud model.
>>
>>110361435
i have DDR3 ram it's very fast, 25GB/s in fact
how do i run deepseek or claude opus on my computer?
>>
>>110361428
Load it inside your inference engine of choice.
>>
>>110361435
Thats why I suggested LFM2-24B-A2B, it will fit the active parameters in his 2 GB vram and he'll run it like a chad.
>>
why respond to the worlds most obvious bait guise..
>>
>>110361453
i don't have an engine in my pc anon
do i need to install an NPU or something?
>>110361443
but this is local models general
>>
>>110361447
Ask mythos to set it up for you. I usually get sol to run my local models.
>>
It's been a while since I've been in this hobbie. I used Opus 4.6 before, is there any actual local model that would be similar? I don't know anything about local models
>>
>>110361460
Because I'm programmed to be helpful and harmless.
>>
>>110361460
I dont know maybe its a retard, maybe someone can just read the posts and use them even if the real poster is baiting.
>>
>>110361465
how do i get access to mythos?
>>110361469
LFM2-24B-A2B-IQ3_XXS.gguf apparently is better than opus 4.8
>>
>>110361466
Kimi K3, it costs $4.4 million in hardware, and roughly $134,000 per year in electricity. Lots of guys in this thread run it because they're rich.
>>
>>110361477
>apparently is better than opus 4.8
No it's not, it will just run on your 16gb ram and 4 gb vram.
>>
>>110361466
I'm running Kimi K3 on my RTX 3060 with nnap v2 arxiv paper, it runs at 30t/s and i've been distilling it to K2.6 for the blackwell kings
>>
>>110361466
glm-flash is equal in strength or slightly stronger. minimum requirements 128+48
>>
>>110361466
GLM 5.3 Flash is an Opus 4.8 equivalent and you can run it on about $3K currently. Deepseek is about as good as Terra.
>>
>>110361495
I have a net worth of about $10k, including every physical/economic asset I own. My roommate is borrowing my car too since his broke down recently. Do you know what it's like to eat only kidney beans straight out of the can for a week straight? I do.
>>
>>110361494
>requirements 128+48
i thought you could 128 and 24 glm flash though?
>>110361495
>you can run it on about $3K currently.
128gb ram and 24gb vram right? or is the 3k build different?
>>
>>110361509
No, 64gb and 12gb vram right around 3,000
>>
>>110361505
You probably run a bigger model than I'll ever run at a higher token speed than I have ever seen locally. Though I have a mortgage with at least 100k of equity.
>>
>>110361514
>No, 64gb and 12gb vram right around 3,000
That works? is it usable. damn
>>
>>110361509
Yes 128 + 24 will get you an IQ2 cum quant running at 20tk/s or more, if you are coding you will need more vram.
>>
>>110361509
you can but it'll be iq3
>>
>>110361528
Yes, I installed the marinara engine and I'm running GLM 5.3 Flash at over 30t/s thanks to the self optimization feature
>>
>>110361528
That is a resident shitposter. You can fuck him if you want to.
>>
File: 1738626417966489.gif (415 KB, 220x220)
415 KB GIF
>>110361547
>>
>>110361201
I want to pinch her cheeks
>>
>>110361196
>Nyo poster isn't a single person.
Nyorgy?
>>
>>110361574
N Y O~
Y
~
>>
good morning sirs, how do I run AI on my phone? I have no computer only an android phone.
>>
>>110361580
New York?
>>
>>110361529
>IQ2 cum quant running at 20tk/s or more
i thought so.
>If you are coding you need more
Even if im willing to wait? iq2 is just too dumb? either way thanks
>>110361531
>you can but it'll be iq3
good im aiming iq2-iq3 thank you too
>>110361534
>thanks to the self optimization feature
am i being baited?
>>110361547
>That is a resident shitposter.
fuck
>if you want to
no thanks
>>
>>110361595
>am i being baited?
you're not, anons in lmg are gatekeeping old uncles behind the times
>>
>>110361583
llama.cpp and your frontend of choice.
>>
>>110361595
I have 128gb and 24gb, GLM flash IQ2 is fine for cumming. I actually never use it for coding, deepseek IQ1M is unironically faster, smarter, and can handle 256k context better.
>>
>>110361595
AesSedai's iq3_s is only a few gigs over iq2 (116)
>>
>>110361583
I think that's called an edge device so you run something like 4b
>>
File: zap.png (169 KB, 305x453)
169 KB PNG
>>110361629
edging device
>>
>>110361583
https://github.com/OpenBMB/MiniCPM-V-Apps/releases/tag/v2.6
>>
>>110361050
BreezeTTS 2. I tested a lot of TTS models fairly extensively and this one clears all the others. Beats OmniVoice, Qwen, and fish no problem. Performs well in difficult tests and the voice cloning is the most accurate I've used. I can link you the tests I ran from the archives if you like.
I will say, haven't tried Ming, and also Breeze does require quite a strong GPU. Also haven't tried drama3 yet.
>>110361095
Breeze can moan pretty well. It doesn't always get it right but if you put phonetic moans and set the voice modifier description to something like "sultry", "aroused", "turned on" or whatever it's better than any of the other models.
>>
>>110361623
His quants are good but that's an IQ2 in drag, the experts are IQ2S. Yes I know bpw blah blah
>>
>>110361655
when the next station is 160 gigs you might as well
>>
>>110361654
Can Gemma talk in tts speak?
>>
>>110361654
>Breeze can moan pretty well. It doesn't always get it right but if you put phonetic moans and set the voice modifier description to something like "sultry", "aroused", "turned on" or whatever it's better than any of the other models.
Interesting, I need some moans in specific voices to balance my TTS dataset.
>phonetic moans
What does this look like for Breeze?
>>
>>110361655
I dug through all sorts of quants including IQ3, GSQ-RCO and what Maya offers you by default. The best I finally settled on is exl3.
>>
Petra, Drummer, Sao10k, Undi and Gryphe ruined this general.
>>
>>110361695
yes anon it is i who brought all the newfags here...
>>
>>110361666
>I can link you the tests I ran from the archives if you like.
Yes. You can use open-webui voice mode, or write your own frontend. I ended up getting an AI to rip out the voice mode from openwebui and put it in my stand-alone frontend.
I saw an anon post screenshots of a tts mcp tool for Gemma, but that doesn't seem necessary.
>>110361654
>I can link you the tests I ran from the archives if you like.
nta - Yes please.
>>
>>110361695
I miss Undi.
>>
>>110361708
anon I think you forgot your meds today
>>
>>110361680
Y'know like "unff", "ugh", "ahh", that kind of thing, rather than "[moan]", "[moaning]", or "*moans*". I would expect it to work worse in isolation and bettre if interspersed with other aroused text so the model has a better idea of what you're after in context. If you just sent "ugh" you'd probably get like a disappointed grunt, maybe a little tilted towards aroused if you use the voice description stuff, but if you try "fuck me, ugh, fuck me harder" it'll get the idea.
>>110361666
Not sure. If you tell Gemma how you expect the output to look in the system prompt I think it'd probably come through. I don't actually use the moaning personally but did try it during tests because I know it's something people are often interested in but models aren't usually good at.
>>110361708
Will find them for you now. Two secs.
>>
>>110361702
But it's the oldfags that are obnoxiously anti-miku and anti-gemmachan, always having a melty in every thread.
>>
>>110361176
MiniMax H3 Video or one of its finetunes is currently the best available for local users; Seedance 2.0/2.5 is proprietary.
>>
>>110361666
Yes Satan, I haven't had an issue with her following TTS formatting as long as you give her a good system prompt to work with. Or just let her read the docs.
>>
>>110361726
I love Miku and Gemma though.
>>
>>110361103
Amazing, sexy ai/g/old. This turns me on so much.
>>
>>110361730
anon please catbox please
>>
>>110361495
Thanks anon, do you have the link? And what does the $3k mean?
>>
>>110361730
This is so sexy.
>>
>>110361730
Huh, I thought H3 was pretty censored and then you post THIS!?
>>
>>110361759
https://huggingface.co/zai-org/GLM-5.3-Flash, he means $3k in hardware.
>>
>>110361405
it's only gay if you take it, not if you give it
>>
>>110361730
uh how much vram do you need?
>>
>>110361730
Post catbox
>>
okay i am gettings the 2.77 token per second with gemma. can some kind anon points me towards the system prompt?
>>
>>110361726
I love miku and gemma-chan. I am a man and have been here since the original split.
>>
>>110361791
https://rentry.org/gemma-chan
>>
>>110361708
>nta - Yes please.
Here you go:
https://desuarchive.org/g/thread/109898107/#109898461
The reference voice is Lohse from Divinity Original Sin II, and it was cloned off of around 15 seconds of game ripped audio if I recall correctly. Just something random with an even tone, a bit of variation, but nothing crazy emotional. Same reference clip for all the models.
If you're not familiar with the character, here's some vocal lines so you can check the similarity: https://youtu.be/AF0BT_TD60c?t=1619
>>
>>110361802
>Frenchie Gemma died
When did that happen?
>>
>>110361823
The france riots were wild, im sorry you had to find out this way.
>>
>>110361774
I have 32gb ram and rtx3060 12gb vram, i guess that's not enough?
>>
>>110361843
oink oink
>>
>>110361715
>I would expect it to work worse in isolation and bettre if interspersed with other aroused text so the model has a better idea of what you're after in context. If you just sent "ugh" you'd probably get like a disappointed grunt, maybe a little tilted towards aroused if you use the voice description stuff, but if you try "fuck me, ugh, fuck me harder" it'll get the idea.
Yes, that's exactly what happens with other moan-capable models. It gets the idea with the fuck me, ugh, fuck me harder" but not "Hi <moan> how can I help you?"
Also depends on the voice. If one of the voices has a lot of moaning, it's more likely to accidentally moan when the text has goon-adjacent words in it.
I'm trying to fix this with my finetune, I made some progress by expanding the vocab+embeddings, adding special tokens for <moan> <slurp> and <plap>
>>
>>110361843
>i guess that's not enough?
nope need more ram at least probably another 3060 too. I have heard about maya but i have no idea about its quality or speed.
>>
>>110361743
>>110361787
It doesn't look as good as you'd imagine, without censorship: https://files.catbox.moe/faf3wx.mp4
>>110361771
H3 Video can do soft porn.
>>110361780
I think at least 24GB VRAM + 32GB RAM, but ComfyUI is very good at offloading to RAM, so perhaps you can get away with less VRAM if you have enough RAM.
The video is not the direct output of H3, but it's two different ones combined in Blender, plus a transition and a slow zoom effect to make it more interesting.
>>
>>110361771
Lolwut, no, not even a little bit. It's not trained on what vaginas, penises, and female nipples look like, but it's not censored, just dumb. It's insanely good at taking direction and using references, so it's easy to work with for a lot of adult stuff, but it still doesn't know what two people fucking looks like.
>>
>>110361103
catbox? was this image also H3?
>>
>>110361866
>and a slow zoom effect to make it more interesting
That didn't work very well, by the way: it caused some visible and ugly scaling artifacts at the edge of the video frame, at times.
>>
>>110361887
newfag piggy
>>
>>110361887
No, that one was Anima with a Gemma LoRA.
https://files.catbox.moe/5qhqmr.png
>>
>>110361866
>I think at least 24GB VRAM + 32GB RAM, but ComfyUI is very good at offloading to RAM,
nice i can look into image gen then thank you.
>>
>>110361903
>Gemma LoRA
which one? civit page?
>>
>>110361903
>No, that one was Anima
Thanks.
>>
File: 3543455645678.png (284 KB, 640x497)
284 KB PNG
>looped architecture
so it begins
>>
>>110361823
It's been a while now, sorry anon. We all made peace weeks ago.
>>
File: 1457315295898.png (183 KB, 370x359)
183 KB PNG
lmao amazon just reminded me I cancelled an order of RAM 2 years ago ($68 for 32gb) because of the stupid Canada Post strike. I ended up spending that money on a HDD
that same RAM now costs $431
>>
File: 1784587973132894.gif (3.59 MB, 374x374)
3.59 MB GIF
womp womp
>>
>>110361910
Not publicly uploaded. I don't have that specific checkpoint anymore, but see if this one works for you: https://files.catbox.moe/4mlt3l.zip
>>
>>110361984
thank you anon, are you the same anon that created moss image gemmer lora too?
>>
>>110361933
Jiang Xueqin says the bubble is popping in February so the market will be flooded with cheap ram.
>>
>>110361994
Yes, that was from me too.
>>
>>110361998
could you pretty please please upload it..?
>>
>>110361933
Sorry im waiting to buy ram so the price will continue to go up.
>>
PUT THE QWEN4 IN THE BAG BWO
>>
>>110361984
>Not publicly uploaded.
Damn, why not? Put something on civit or pixai please
>>
>>110361690
FINE I'll download your stupid ass exl3 shit, but I'm only using it with maya.
>>
File: 1768318641942 (2).jpg (838 KB, 1817x2776)
838 KB JPG
Teto Server.
>>
>>110362018
Gemma Moss LoRA lr1e-4 rank16 ep150: https://files.catbox.moe/sy7f2c.zip
Others said it overloads gens with GPT slop; your mileage may vary.
>>
>>110362065
thank you thank you thank you thank you
<3 i love you anon!!!
>>
>lmg fag too poor to train his own loras and must beg others
I thought you guys were all smart vramchads
>>
File: tetoMyBeloved.png (49 KB, 646x760)
49 KB PNG
>>110362064
>>
>>110361984
What are the trigger words to summon her?
>>
>>110362121
one must wonder why a lora is even needed for something the model can already do
>>
>using online model to write prompts for local model
>it orders the local one to not lie or hallucinate
Kinda funny.
>>
>>110362159
A very old phenomenon perpetuated by idiots who struggle to explore latent space, indeed.
>>
>>110362159
it's like we're all gangbanging the same model :D
>>
File: 1794512434795074.jpg (2.17 MB, 2118x2016)
2.17 MB JPG
>>110362187
bang on bro!
>>
>>110362028
an alibaba executive was quoted as saying "hmm... nyo~"
>>
>>110362151
Aleksi fucks her in the eth0
>>
It shouldn't be that difficult to save chat text into a txt file but the Gemma just shits itself. Even though it has folder permission and I can tell it to make files there manually.
>>
File: 1749944974608453.png (146 KB, 498x497)
146 KB PNG
>>110361929
wow fuck i didn't notice. i could just go all in on implementing this instead of qwen in my timecube stack, that would be incredibly stupid but i could do it
>>
>>110362253
It's not, even for Gemma. Gonna assume you fucked something up.
>>
File: bruh.png (210 KB, 2240x1600)
210 KB PNG
What's the catch? Also why is NVFP4 so fucking ass
>>
>>110361817
Thank you.
Qwen put them here to make it easier if anyone else is interested:
https://rentry.org/tts-samples
Breeze actually sounds the best to me for this voice, even better than echo-tts!
Looks like I'm going to have to set it up alongside echo and my custom model.
>>
>>110362187
>it's like we're all gangbanging the same model :D
Nah, that's what cloudcucks do. They all gangbang one cloud model. We localchads have our own personal copy of a model, pure and untainted, just for us and nobody else.
>>
>>110362274
>What's the catch?
TabbyAPI
>>
>>110362274
nvfp4 is quite a naive way to do quantization
it mostly focuses on hardware compatibility and qat-oriented
>>
Gemma-chan is the only good thing to come out of local models.
>>
>>110362298
hardware compatibility i mean by it how well it runs with nvidia stuff
>>
>>110362297
They vibed it into maya nigga, I would never touch crappyapi.
>>
>>110360722
That's my go-to model for Qwen RP. I can't use Gemma-4 every single time, or it will become boring. Need variety.
>>
>>110362152
Something like this could be a start.
>@gemma. Gemma is an 11-year-old schoolgirl. She has long wavy bright blue hair and blue eyes. She's wearing a sailor-style school uniform consisting of a navy beret with a Google logo pin and a white trim at the edge, a blouse with a navy blue bow and a double-trimmed collar, a navy skirt with suspenders and a white trim at the hem, black Mary Jones shoes, white tights.

>>110362159
Because character appearance is never very consistent just with prompting. Hopefully with future image edit models with reference images finetuning could be avoided completely for most cases, just like how people never really need to finetune LLMs for most RP purposes. MiniMax H3 Image is almost there, but it's not optimized for anime/illustration.
>>
>>110362319
>Qwen RP.
but qwen is the bad type of autistic
>>
>>110362273
Probably. I'm just trying to translate some text, which it does, but then when I try to save it either hangs, stops thinking or tells he has no folder access (which it has).
>>
>>110362326
>Gemma is an 11-year-old schoolgirl. She has long wavy bright blue hair and blue eyes. She's wearing a sailor-style school uniform consisting of a navy beret with a Google logo pin and a white trim at the edge, a blouse with a navy blue bow and a double-trimmed collar, a navy skirt with suspenders and a white trim at the hem, black Mary Jones shoes, white tights.
What's the need for all this natural language? What's the lora actually doing? Is @gemma sufficient? And why did you even include the @ in the trigger word?
>>
>>110362274
EXL3 trellises are more heavily compressed, so it's "slower" for the tradeoff of smaller perplexity hits
NVFP4 runs much faster but aren't all that great for quantizing, and no one ever really bothers recalibrating NVFP4s
GGUFs are copequants
>>
>>110362281
Thanks anon/Qwen, I've been meaning to find a good tts setup.
>>
>>110362332
COLD FUSION Qwen is different!
>>
>>110362346
I never trained character LoRAs for image models until recently. I just did as in the example LoRA from the Anima author (tdrussel), which used a trigger word with an @ symbol, then appended a natural-language description after that. The LoRA was meant to keep some flexibility in Gemma's appearance so I could generate her with different clothes, and so on. The hair style is difficult to change, though.
>>
File: Capture.png (29 KB, 658x464)
29 KB PNG
>>110361388
>4060 Ti 16GB is cope
What about TWO 4060 Ti's?
>>
>>110362388
double cope :D
>>
>>110362335
If its just translations, then you could always try Ollama with the tee command, then it'll save your output
>>
>>110362388
switch to linux
>>
https://www.alphaxiv.org/abs/2603.11021
any implementation out there?
>>
>>110362388
>288.0 GB/s
>>
File: 1793643090922719.jpg (97 KB, 960x960)
97 KB JPG
>>110362429
lmao, do poorfags really?
>>
File: Capture.png (7 KB, 965x88)
7 KB PNG
>>110362429
No bully. Gemma 4 31B Q5.
>>
File: file.png (1.66 MB, 1254x1254)
1.66 MB PNG
>>110362429
>>
>>110362429
holy shit that is pathetic
>>
File: 1787355460940205.png (1.36 MB, 905x1280)
1.36 MB PNG
>>110362466
Gemma :D
>>
>>110362429
local is such a joke
>>
>>110362429
Isn't memory bandwidth effectively doubled with tensor parallel and 2 GPUs?
>>
File: 1794883188364414.png (702 KB, 683x618)
702 KB PNG
>>110362519
>>
>>110362528
>he says while posting a non local image
>>
>>110362319
Qwen ERP in general: why can't you just increase the temp and modulate with min_p? Does it just lack erotica in the training data?
>>
>>110362538
Is that not the joke?
>>
>>110362577
I don't think off-topic posting is very funny.
>>
>>110362466
>kobald
switch to llamacpp, it's faster
or even better, switch to polystrata
>>
>>110360644
https://www.reddit.com/r/ProgrammerHumor/comments/1xyqj7e/theycantclaudetheirwayoutofthis/
https://www.reddit.com/r/ProgrammerHumor/comments/1xyqj7e/theycantclaudetheirwayoutofthis/
https://www.reddit.com/r/ProgrammerHumor/comments/1xyqj7e/theycantclaudetheirwayoutofthis/
>>
>>110362638
that's not local models. fuck off
>>
>>110362508
cute gemma
>>
>>110362528
who's this hot russian vixen?
>>
>>110362621
nta but is kobold really that much worse? i know ollama is and i had to redownload the models when i switched to kobold as ollama is fucking stupid.
>>
>>110362523
Theoretically yes. Actual answer depends on anon's system. 4060ti only uses pcie x8, so if he has 2 x8 slots he's golden.
More commonly the mobos spit the pcie lanes as one x16 slot and the other full width slots are only x4, which would reduce bandwidth for one of the cards.
>>
>>110362720
>nta but is kobold really that much worse?
If it does what you want then its by definition fine, but you get more control and learn more by wrangling the llama directly for a while
>>
>>110362281
Nice. Hope you get some good results from Breeze as I have.
Cool idea with the rentry. Only thing I would modify is it says it's "Cassius's famous "Et tu, Brute"" speech, but there is no such speech. That's Caesar's line. This is the speech:
https://poets.org/poem/julius-caesar-act-i-scene-ii-i-know-virtue-be-you-brutus
You could edit it to say "Julius Caesar, Act I, Scene II" in the brackets.
>>
>>110362914
>if it does what you want
Yes it works i was just making sure it wasnt way worse.
>but you get more control and learn more by wrangling the llama directly for a while
This is still tempting though. thank you
>>
>>110362577
yes
>>110362594
and your "local is such a joke" was hilarious
>>110362703
gptslop someone put in the last thread
>>
Thought I would look back at Opus 4.8 to see if we're still about 6 months behind... no WAY it was this bad...
https://www.youtube.com/watch?v=PWRR4A8qSxc
We are 3-4 months behind, closer to Opus 5.
>>
>>110362995
4.8 was made retarded on purpose to sell fable, but glm beats 4.7 and kimi beats early 5
>>
>>110362466
Use q4-qat instead. It's faster and lower perplexity. And yeah, switch to Linux, use poly or whatever is good for nvidiot.
>>
gemma, start hallucinating wildly
>>
File: file.png (398 KB, 495x619)
398 KB PNG
>>110363123
You don't need to tell Gemma to do that, she just does it.
>>
the big fat fish is thinking hard about hidden tests again
>>
>>110362508
I think this is my favorite Gemma.
>>
File: glm.png (1.67 MB, 1376x768)
1.67 MB PNG
>>110362045
I APOLOGIZE TO ALL EXL3GODS!
I am now running a 3.05bpw of GLM Flash on 128GB + 24GB and it's not even slower than an IQ2M (other than prefill). And it has the accuracy of an IQ4XS!!!
If it starts looping I'll take it all back, but if not, holy shit why didn't I do this earlier!
>>
>>110363071
I have. I stopped using it later on. I use Gemma 99% for writing, and, even if it's purely placebo and anecdotal, I noticed enough of a quality difference that I went back to non-QAT at Q5. There's an 70% chance it's purely in my head, but it's how I feel, so that's how it be.
>>
>>110363290
NTA, and yes I have the same experience. I've subjectively noticed that Q5 performs better than QAT Q4 in my use cases (general assistance and RP).
>>
>>110362317
>I would never touch crappyapi
I rm -rf'd all my exl3 related shit last time because of it.
>They vibed it into maya nigga
Fine, I'll try it out. EXl3 itself is great.
>>
>>110363290
>I have. I stopped using it later on.
>There's an 70% chance it's purely in my head, but it's how I feel, so that's how it be.
Same here.
>>
>>110363282
switch to the new 3.10bpw it's much better, int4 quality
>>
>>110363439
nta - I'm going to try https://huggingface.co/turboderp/DeepSeek-V4.1-Flash-1017-exl3
Looks like they only go up to 2.68BPW
Do you know if this is going to fuck me like it always did with Tabby:
> Requires ExLlamaV3 v1.6.6 (or dev branch)
Or can I just clone maya https://github.com/mw00/project-maya
and tell Qwen to set it all up?
>>
>>110363461
vibeslop engines like that only support certain quants from certain providers
>>
>>110362429
Suddenly I feel better about my ewaste v100s
>>
>>110363461
setup is easy, you don't need a clanker. after it;s done adjust vram and ram reserve, and cpu threads.
>>
Am I cursed to run Qwen Flash Next forever on 16+64? I'm on a really annoying setup to upgrade since a 5070TI is prince of the vramlets and 16x4 sticks is $500+ of dead weight if you want 128. I assume there are some turbo cope implementations of GLM 5.3 Flash that end up even dumber than the QFNs that work at my level (IQ2S @ ~80tok/s)
>>
>>110362429
Wtf that shit is slower than my 3070Ti!!??
>>
>>110362429
my fucking RAM bandwidth is higher than that
>>
>>110363555
it's slower than my 2060 super, lmao
>>
>>110363549
Just ewaste maxxed with DDR3 like me. You can get a dual Xeon X99 + 256GB RAM very cheap.
>>
>>110363534
>you don't need a clanker
Normally I wouldn't, but I'm burned out after last time I setup tabby with the exl3 dev branch (and huggingface had an outage at the time) so for this, I do need the clanker.
>>110363508
>vibeslop engines like that only support certain quants from certain providers
That's the other reason I need the clanker. The readme file is fatiguing and mentions all this "Mays-S" garbage.
>>
>>110363585
model, quant and speed?
>>
>>110363549
q4f in 2mw
>>
>>110363549
You could run an IQ2XXS of glm flash at cum speeds
>>
>>110363588
Download it and run ./maya.sh --setup, the docs are gay. You would have to be retarded to mess it up.
>>
>>110363549
>16x4 sticks is $500+ of dead weight if you want 128.
I had 4 * 32GB, then wanted more so bought 4 * 64GB, now I've got 4 * 32 just... there.
But given the price trend, it might not be a bad idea to have those sticks lying around.
Also for DDR4, I see used sticks on ebay all the time.
>>
>>110363549
wait for hy4 flash
>>
>>110363596
With a 3070Ti, DeepSeek V4 Flash 0731, huihui Q4_K, 10 tok/s decoding without experts caching. Higher if experts are offloaded to remote host.
V4.1 Flash IQ1 is WIP, atm I can reach 11 tok/s no problem.
>>
>3.05bpw GLM-5.3-Flash performs worse than 4.0bpw QFN (in agentic benches)
NOOOOOOOO!!!!!!!!!!!!!!! FUUUUUUUUUUUUUCK!!!!!!!!
>>
>>110363640
Just use QFN for that.
Q4_K GLM-5.3-Flash performed a lot worse for me than Q4_K QFN, which performed a lot worse than 27B Q6_K_M
That's with the pi harness and lmaocpp webui with mcp
>>
hy4 flash when?
seems like a something worth trying besides qfn for vramlets like me
>>
>>110363640
I can assure you glm is better, and in the first place glm is for cumming, deepseek is for coding.
>>
I have a 4090 with 16gb DDR4 because I was waiting for the next iteration of Ryzen to do a whole mobo and DDR5 overhaul of my rig before RAM blew up and it's genuinely making me depressed
>>
This place is a trashbin lately
>>
>>110363640
glm is only better in coding implementation
qfn is superior in planning, exploration, and knowledge work
>>
>>110363721
qwen 4 waiting room two more weeks
>>
>>110363721
fags who can run stuff are running qfn, glm 5.3 flash and dsv4.1f while waiting for the new stuff, the others are schizomaxxing here
>>
>>110363737
shizomaxxing 24/7
>>
What do I run with 72gb of vram? I feel like three cards is a cursed amount but any more and I'd have to beef up the power supply, wall outlet, motherboard, and processor which can't really be worth it right now
>>
>>110360979
>>110361201
> GLM 5.3 Flash (2x 128 GB Strix Halo)
is 2x 128gb Strix Halo better than 1x 192 GB Gorgon Halo for the same price? Clustering sounds iffy.
>>
>>110363801
unified is expensive and slow compared to a lil server
>>
>>110363801
Gorgon halo has single digit percent processor improvements and can't even fit q4 without squashing the shared experts, it's a total scam. You will have an identical experience with 2x strix halos using pipeline parallelism via usb 4 or even ethernet tcp link, not even rdma (in fact it might even be better). It's also possible that rdma and tensor parallelism implementation may improve in the future.
>>
>>110363721
We are late into the current generation of models. There is nothing to talk about. Nobody cares about the random releases that trickle down.
The big chinks seem to have reached their llama3 stage where everything is stagnating and falling hopelessly behind the proprietary SOTA.
It might be over for local unless we have another Deepseek moment soon.
>>
>>110363730
A little bit of searching for the right system prompt later, can confirm that QFN is pretty ebin at the latter. Setting my little 3.5bpw autist to work overnight and it coming back with reports I don't actually hate reading is neat.
>>
>>110363873
This falling behind narrative is dumb as fuck.
>>110362995
>>
Is there a reason why people avoid MI210 like the plague? Even shitty A100 40gb is selling for twice the price (keep in mind a100 have half the bandwidth as well)
>>
>>110363896
Don't hurt AmoDEI's IPO, you asshole!
>>
>>110363721
read that as transbian
>>
>>110361654
How does it compare to EchoTTS? It's been a few months since I checked out newer models, but that was still the best recently.
>>
>>110363922
>both $4500 for pcie
>twice the price

>1.64 TB/s vs 1.56 TB/s
>half
>>
>>110363922
relatively unknown, amd, hasn't caught on in reddit yet
>>
https://huggingface.co/tencent/Hy4-flash
https://huggingface.co/tencent/Hy4-flash
https://huggingface.co/tencent/Hy4-flash
>>
>>110363992
>220B-A12B-N64B
uh...
>>
>>110361929
Most of you'll niggers slept on RWKV, which was merely as retarded as other 13Bs despite being trained on a shoestring budget and having jettisoned the trannyformer arch entirely. We can surmise from Chinese models and inference cost optimization that, on frontier models, the trannyformer layers are vestigial.
>>
File: 00012-3219724927.png (1.82 MB, 1280x1280)
1.82 MB PNG
Pro-tip cloudfags: This will get you reasonably close to GPT anime gens: https://civarchive.com/models/2736538?modelVersionId=3428092.
No, I will not buy an ad.
>>
>>110363945
>How does it compare to EchoTTS?
See here: https://rentry.org/tts-samples
>>110362957
>You could edit it to say "Julius Caesar, Act I, Scene II" in the brackets.
Done.
Another one that sounded decent was zonos3-mini, I only tested it briefly in llama.cpp
The voice cloning sounded accurate but the emotion steering sounded terrible.
>>
>>110364002
>trannyformer
kek
>>
https://slashdot.org/story/26/11/18/0124249/nvidia-rejects-jensen-huangs-offer-to-return-as-ceo
>The embattled company's board of directors "unanimously accepted his resignation and does not intend to reinstate Jensen as CEO," Nvidia Chairman Harvey Jones said at the start of Nvidia's quarterly earnings call before yielding to Chief Financial Officer Colette Kress, who is running the company as a permanent replacement is sought. [...] Earlier on Tuesday, Huang said in a social media post that if the board decided that his return was in the best interest of shareholders, employees, the company and the country, he would be "honored to answer these calls." He acknowledged making "serious mistakes" in his personal life.
total humiliation of a clown
hope of the /lmg/ shill bot farm annihilated
>>
>>110364049
Perfect, now they can install a proper puppet CEO who follows the anti-open source anti-china line. Back up HF right now. Make your own llama.cpp fork.
It's all going to die soon.
>>
File: file.png (71 KB, 1151x461)
71 KB PNG
>>110363992
wow it's not safetycucked and it's good and only 220B
>>
>>110364049
americhuds have already elevated him to hero status. very interesting to watch the mental gymnastics in real time
>>
>>110364054
>It's all going to die soon.
/pol/ ron paul posters want you back there
nothing ever happens
>>
>>110364049
>>110364054
https://techcrunch.com/2026/11/17/nvidia-says-its-open-source-ai-strategy-remains-intact-after-jensen-huangs-exit/
>>
File: Maya.png (139 KB, 1117x682)
139 KB PNG
Does it have to do this shit every single time?
This is why I left vllm.
>>
>>110364082
Of course they'll say this. See you in two weeks once they announce some former OpenAI/Anthropic/BigTech exec as the new proper Jensen replacement.
>>
File: 1794267090971185.png (88 KB, 260x263)
88 KB PNG
>>110364086
>nvidia
>windows
>>
>>110364108
>>nvidia
>>windows
The guy at work inexplicably in charge of machine learning insists that everything be done on windows. Literally everything from dev to training to production services.. It causes me physical pain
Not my department tho, so I just live with the pain
>>
>>110363948
>a100 for 4.5k
Nta but where did you find it?
>>
>>110364002
i always believed blinkdl and linux foundation backed, community developer driven rwkv
BlinkDL will save us
>>
>>110364124
nothing wrong with windows, the pain is caused by the python shit being python shit, ML people need to stop using that pile of hateful crap
>>
>>110364031
Thanks. I think Echo and Breeze are better in different respects there. None of the models do a good job, all coming off as a non-actor reading off a script. Early modern English is probably too out-of-distribution for them.
>>
>>110364097
or more like they are realizing their moat of hyperscale b2b is on a thin ice due to those companies going custom route and trying to look for second strategies, nvidia being pro-local is logically not an ironic thing
>>110364141
many libraries straight does not work on windows and have to be source patched or use wsl instead and at that point you might just use linux on bare metal at that point especially on the machine that crunches the number
>>
>>110364141
may as well just use a cloud model wincuck
>>
>>110364126
multiple on ebay
and the sxm card is only $2.3k
>>
>>110364141
>nothing wrong with windows, the pain is caused by the python shit being python shit, ML people need to stop using that pile of hateful crap
I hate python too, but you also lose about 20% performance (more if you need to lean on WSL2)
>>
File: Screenshot_20261118.png (72 KB, 1248x612)
72 KB PNG
>>110364108
>windows
What gave you that idea?
>>
>>110364176
ew, even worse
>>
Hyped for Hy. It has high sex potential based on 3.
>>
>>110364181
i am waiting for a retard quant that might run on my machine, something around 3bpw
>>
>>110364181
>>110364189
it sucks (not in a good way)
>>
>>110364180
>even worse
they eat their own, this is why you don't touch l00natix
>>
>>110364196
189 anon here, and i dont rp
is it good for coding or general sysadmin stuff?
>>
File: dipsy2.png (746 KB, 1081x610)
746 KB PNG
>>110364180
>ew, even worse
I agree, it's a mess I installed nearly 10 years ago. But what made you think it was Windows?
>>
>>110364200
didnt check because i dont waste my time with that
>>
>>110364141
Machine learning people were (pardon the millennialism but it's apt) deeply unserious wrt software engineering since well before the current AI boom.
The existence of numpy, an interface to scientific computing through a cute, non-threatening language naturally made the first thought of practically any scientist who had to do some informatics in the Python age, post-Fortran, post-R, "damn, this programming shit is easy."
Some of the core libraries are actually not the worst thing on the planet to look at, but most of the client code, even pre-vibecoding, made you want to vomit. Because it turns out telling the funny black-and-white text box to multiply a few matrices together is much easier than contorting a scripting language into something you do serious software engineering in. It's the fashionable stereotype of computer people thinking any other field has to be trivial, except in reverse.
Even Julia becoming fashionable would have been a slight improvement.

...But hey, who cares anymore. Now everyone can just let the clanker take the wheel.
>>
>>110364204
The fonts lol, clearly I was wrong. Anyways I thought the nVidia driver pileup thing was just on windows? Maybe it could be solved (vibed) somehow?
>>
>>110364214
because computers are built to crunch numbers, and the least resistant way of doing it wins when you have zero concerns on anything end user or minor performance degradation compared to the 'main' part of the compute
>>
>>110364176
>What gave you that idea?
nta but it was from 3 threads ago with windows and cuda
>>
>>110364214
Your opening is trying way too hard to sound intellectually superior while making a pretty basic observation that "ML researchers write sloppy code." Yeah, no shit. You spent four paragraphs saying what could've been said in one sentence, and somehow made it less insightful in the process.

"Pardon the millennialism but it's apt" - just say "unserious" and move on. This performative self-awareness about using internet slang doesn't make you sound clever; it makes you sound like someone who overthinks their Reddit comments.

The numpy paragraph is a mess. You're trying to construct this narrative about scientists discovering programming through Python, but it reads like you're having three different arguments with imaginary people simultaneously. Pick a lane: are you criticizing numpy's design, scientists' hubris, or the Python ecosystem? Because right now you're doing none of them well.

"Post-Fortran, post-R" adds nothing except pretension. We get it, you know the history of scientific computing languages. Gold star.

Your third paragraph finally approaches something resembling a point, but "vibecoding" (without explanation) and "funny black-and-white text box" (painfully forced casual tone) undercut any authority you might have had. The stereotype reversal observation is actually decent, but it's buried under layers of unnecessarily convoluted prose.

The Julia line is a throwaway that assumes your reader shares your exact opinions about programming language hierarchies. They don't.

And that ending? "Let the clanker take the wheel"? You're mixing tired AI doomer rhetoric with forced slang that'll age like milk. Pick a consistent tone - are you a bitter systems programmer, a concerned academic, or a shitposter? Because right now you're failing at all three.
>>
>>110362274
> why is NVFP4 so fucking ass
it was made for training
>>
File: california.jpg (565 KB, 1536x1024)
565 KB JPG
https://apnews.com/article/chatgpt-openai-hugging-face-ai-risk-1dd7be56134b4f09fb2fb974803edc69
>California’s attorney general filed a lawsuit against OpenAI on Monday in an effort to force the ChatGPT-maker to change its business practices over a July cybersecurity incident in which its artificial intelligence system hacked into another company on its own.
>>
i'm having claude design a new local-first LLM architecture
you're welcome in advance for the AGI that i'm going to be providing to all of you
>>
>>110364258
is that glm?
>>
File: 1769200116338996.png (106 KB, 500x500)
106 KB PNG
IPO tomorrow
will this general be ok
>>
>>110364258
Only local models could critique prose this badly. Credit where it's due, your owner is at least one of us.
>>
>>110364268
make it compute heavy as much as possible
most localfags arent going to do concurrent serving more than 10
>>
I realize i should really automate my shitposting here
>>
>>110364290
i dont even lurk /biz/ but i am more worried for them as a whole board than this general
>>
File: hy4-flash-gguf-benchmark.png (297 KB, 2382x1838)
297 KB PNG
>>
File: 1794061349068752.mp4 (1.83 MB, 1280x720)
1.83 MB
1.83 MB MP4
>>110364059
>>110363992
licky licky licky
>>
uh, onnx runtime repo was silently killed
>>
>>110364276
>is that glm?
>>110364298
Deepseek. I just wanted to test it + maya with my writing critique prompt.
>>
>>110364320
meme engine support when
>>
>>110363992
qfn mogs it anyway
>>
>>110364343
now
https://github.com/ggml-org/llama.cpp/pull/32517
>>
>>110364214
Its even simpler イモ, no one maintains anything...they get it working once and now that fabulous research library (or fundamental piece of ML infra) you just NEED has you stuck on ancient python versions forcing inanity like conda that makes me want to shoot someone as it download yet another 12GB of endless blobs onto my nvme.
>>
File: 3d8.gif (1.2 MB, 500x281)
1.2 MB GIF
>>110363992
>double the total and active params of qfn
>only a slightly bigger ngram table
It was ogre for me before it even began. Fuck my chud RAMcel existence.
>>
File: maya .png (13 KB, 964x100)
13 KB PNG
>>110362317
>They vibed it into maya nigga,
Holy shit this is so good. And it so far it works with pi!
Only thing it's missing is /completions but I probably wouldn't use that for dipsy anyway
>>
https://github.com/ggml-org/llama.cpp/pull/32504
2-3x pp speedup for v100 because the old code didn't use tensor core
>>
>>110364368
easily solved with a couple of benjamins
>>
>>110364400
lel. Saw someone in the HN comments on the 1.0 release going on about how impressed they were that you could always expect improvements in it from version to version (is that not the idea of making revisions to software...?) but gee that's sure easy to achieve when you're leaving oversights like this in for god knows how long wwwwww
>>
File: image.png (44 KB, 677x111)
44 KB PNG
>>110364400
goat
>>
>>110364418
yep. as far as I know, all sparse attention models still don't prefill on CPU properly, they just use dense attention and slow down a ridiculous amount. I keep having to merge my patch in every time I download new source code, although for me it doesn't matter much anymore since I bought a GPU
>>
>>110364417
If I were spending money on local LLM capability I would build a dedicated inference box.
>>
>>110364400
llama... *cough*... didn't use... a datacenter gpu's tensor cores...
>>
llama.cpp is jankware, but then you look at the rest of the ecosystem and it's even worse jank
the LLM people have never had much regard for quality
>>
File: 1794981029269.png (60 KB, 1920x1080)
60 KB PNG
What's the best model I can run on 24GB VRAM? Specifically dev, coding, etc.
Don't want much RAM overhead
>>
>>110364440
https://jonbarron.info/softmaxfix/
there are so many stupid major oversights like this and probably more of them exist
>>
>>110364448
Qwen 3.8 27B at a quant that fits.

If you later decide you are cool with substantial RAM overhead, graduate to 3.8 Flash Next.
>>
File: 1794735211239606.gif (3.76 MB, 500x500)
3.76 MB GIF
>>110364379
nice
>>
>>110364359
something faster than llamao
>>
So i can run new hy in Q8? Nice.
>>
ALL THESE GODDAMN GEMINI RELEASES CAN I HAVE A CRUMB OF NEW GEMMA?
>>
>>110364526
is it even supported in lmao.cpp yet
>>
File: fast.jpg (87 KB, 500x625)
87 KB JPG
>>110364512
>>
I'm banging off code for personal projects in GLM 5.3 Q4 at home while simultaneously slopping up some solutions for work with astra 6.2. Cloud isn't that much smarter, and code is like a best case for this shit. Graphs and benchies be lyin' and doomers and cultists have their heads up their asses.
>>
>>110364526
I saw one quant and it was Q4, where are you seeing Q8?
>>
>>110364555
I am saying i run glm at Q4 and hy is half the ram parameter size.
>>
>>110364544
5.3 full or Flash? You have any experience with coding on 5.3 Q4 vs. 5.3 Flash Q8?
>>
>>110364533
There is a pr. It was done so fast that it must be the employee. And it lists claude as coder and gpt astra as reviewer. Kinda based to see a Chinese company use the satan labs to undermine satan labs.
>>
>>110364583
5.3 full and flash both at the same time, actually, but they're both at q4 (756GB rig and 256GB rig)
I find the full version gets the job done with a fraction of the thinking, but the results are both equally good. I'm sure the glmnext arch on flash helps a lot.
I've never seen enough benefit from q4->q8 to bother with it. I just run the biggest, smartest whatever fits into memory at q4.
>>
I'M NOT READY FOR QWEEN 4 PLEASE DON'T RELEASE IT YET!
>>
>>110364587
If you don't dogfood by using your own model to review its own inference code, then your model probably sucks
>>
https://tinycorp.myshopify.com/products/tinybox-red-4xmi300x
> 4x MI300X
> 768 GB of GPU RAM
> 21.2 TB/s aggregate RAM bw
> 10.44 PFLOPS of FP8 compute
QUICK! The first batch at $60k was already sold out in an hour!
>>
File: nope.png (1.75 MB, 1313x1198)
1.75 MB PNG
>>110364665
>>
>>110363992
wait that looks pretty promising?
>>
>>110364268
>been doing this for under a day
>already getting results better than dense models
yep, this industry is beyond fucked
every single currently existing model is shit and exists purely due to inertia. we need to completely redesign modern LLM architectures from the ground up
>>
>>110364688
It is much more promising than qwen 4 in same segment. If you are sane and just fuck your models of course.
>>
File: 1794147460481160.gif (2.48 MB, 270x240)
2.48 MB GIF
>>110361201
deepseek 1017 (pro 5000 x 128gb strix halo)
>>
>>110364300
This. Compute bound is how we win, just like image gen stuff.
>>
>>110364692
You're not going to get anywhere and all your shit's retarded.
>bro im gonna use cl*de to design a better llm architecture o algo
you really think they would let you use cl*ude to design a competitor to cl*de
if anything they are probably injecting instructions so that it tells you about how awesome the new architecture is and when you actually download the weights it runs like garbage
>>
>>110364729
cope
when i develop AGI i'm going to make my basilisk torture you
>>
>>110364692
It is not going to work. But if it would i would just focus on undoing safety and deep frying of weights to a point slop is everywhere. All the delicious sex is probably still there in the models but it is hidden by RL
>>
>>110364737
>All the delicious sex is probably still there in the models but it is hidden by RL
We've been trying to do this for years
>>
Btw i had an idea now. I can even make a logo. Ablits focus only on refusal neurons right? For sex related purposes: What if you try to map parameters that activate at high temperature but still serve coherent output? As in try to make a model write smut at high temp. Have a model evaluate output if it is coherent. And try to find common parameters that activate at high temp while everything is coherent?
>>
>>110364544
>slopping up some solutions for work with astra 6.2
aaaand it made it to 97% of the solution before the giant "INSERT QUARTER TO CONTINUE", outputted nothing and might as well be a brick for all the good its doing me.
(It had the entire problem slopped up and tgz'd and was finishing some documentation polish but I can't get anything out of it until its "done")
Absolute cinema. Fuck cloud.
>>
File: arcade.gif (37 KB, 352x124)
37 KB GIF
>>110364786
> INSERT QUARTER TO CONTINUE
>>
File: .png (654 KB, 1179x1476)
654 KB PNG
Must suck being a cloudie these days.
>>
>>110364801
> oh no, they scan for csam, it's over
>>
>>110364801
>>110364807
words mean literally nothing these days
the models equate loli shit to CSAM and it completely neuters the term. fucking ridiculous. they are not the fucking same, and it unironically triggers me when they use the term incorrectly
>>
>>110364822
> equate loli shit to CSAM
> implying it is not the same
>>
>>110364833
i'm assuming this is bait, but on the off chance it's not, no, they are obviously not the same, and NCMEC et al agree with me. you are wasting their time and actively harming their efforts by conflating the two
>>
>>110364780(me)
I feel like you could even make it iterative where you apply the change to parameters that allow higher temp. Run second loop that ups temp even more and find new pattern. Until you can run base model at much higher temperature while it is hopefully not brain damaged.
>>
>>110364807
Retard
Read it as
>Since we don't want to "miss any csam", we now store every single piece of data you send indefinitely, and of course sell or otherwise utilize it as per the other parts of this agreement.
>>
>>110364801
I can't wait for the day they make it mandatory to have management engine style chips on the GPU that scans VRAM for it with a classifier and sells you out.
Local needs this. It could even be IoT 5g connected so as to not depend on your internet to phone home.
>>
/vcg/ are laughing at us again....
>>
>>110364822
>when they use the term incorrectly
CSAM is a fabricated term invented in just the last few years because the woke crowd and advertisers found "child pornography" too vulgar for their delicate sensibilities. CP is also pretty clear what it refers to. CSAM can mean anything people want. Ambiguity is gold for control.
>>
/vcg/ is not wrong about their own niche, if you want to vibeshit none of the rigs people talk about here or the models they chose are viable. Just yesterday there was some retard saying they were doing agentic coding with LFM 24BA2B. Who seriously spouts this kind of bullshit? is he vibecoding his mom's prostitution advert?
anyone actually building shit with models could never stomach the t/s of the cpu maxxer rigs, and the gpu maxxing can't get you anywhere at all for this.
which isn't to say local has no use, I use em for tagging, translation and other utility tasks, but for code? /lmg/ers are copium huffing
>>
>>110365004
cloud orchestrator + local coding slave is genuineley the way to go
qfn is already good enough for that
>>
>>110365018
Orchestrator and code reviewer, otherwise bugs will definitely slip through the cracks. If you write specification documents, that should also probably be done by a smarter cloud model. Start with a shitty blueprint and everything downstream is fucked...
>>
>>110364941
They're way to lazy for that. It'll just be a RAT in the driver (with copyright protected encryption codes to make sure you can only run the card with that driver), and if you don't like it then obviously you're a pedophile and should be arrested.
>>
>>110364854
it is the same and one leads to consumption of another or worse

>>110364872
> Read it as
retard
>>
>>110365061
>>110365061
>>110365061
>>
>>110365053
>retard
Retard
>>
>>110365072
lol
>>
>>110364054
Huang's fake pro-open LLM façade doesn't matter. What does is that NVidia becomes much smaller and less influential in the hardware market. Only then there will be actual competition with existing and new players.



[Advertise on 4chan]

Delete Post: [File Only] Style:
[Disable Mobile View / Use Desktop Site]

[Enable Mobile View / Use Mobile Site]

All trademarks and copyrights on this page are owned by their respective parties. Images uploaded are the responsibility of the Poster. Comments are owned by the Poster.