[a / b / c / d / e / f / g / gif / h / hr / k / m / o / p / s / t / u / v / vg / vm / vmg / vr / vrpg / vst / w / wg] [i / ic] [r9k / s4s / vip] [cm / hm / lgbt / y] [3 / aco / adv / an / bant / biz / cgl / ck / co / diy / fa / fit / gd / hc / his / int / jp / lit / mlp / mu / n / news / out / po / pol / pw / qst / sci / soc / sp / tg / toy / trv / tv / vp / vt / wsg / wsr / x / xs] [Settings] [Search] [Mobile] [Home]
Board
▼ Settings Mobile Home
/g/ - Technology


Thread archived.
You cannot reply anymore.


[Advertise on 4chan]


File: moss-image_00850_.png (861 KB, 1216x832)
861 KB PNG
/lmg/ - a general dedicated to the discussion and development of local language models.

Previous threads: >>110360644 & >>110356474

►News
>(11/17) Hy4 flash 220B-A12B-N64B weights released: https://hf.co/tencent/Hy4-flash
>(11/17) SenseNova-Loop-9B looped architecture model released: https://hf.co/sensenova/SenseNova-Loop-9B
>(11/16) Nvidia releases Nemotron-Labs-Sigma (3B/8B) diffusion models: https://hf.co/collections/nvidia/nemotron-labs-sigma
>(11/13) llama.cpp reaches v1.0.0 milestone: https://hf.co/blog/ggml-org/llama-cpp-v1
>(11/12) Inkling 2 Small weights released: https://hf.co/thinkingmachines/Inkling-2-Small
>(11/09) ZONOS3-Mini zero-shot voice-cloning TTS released: https://zyphra.com/our-work/zonos3-mini
>(11/06) PLaMo 3 Translate released: https://preferred.jp/en/blog/tech/plamo-3-translate-weight-release

►News Archive: https://rentry.org/lmg-news-archive
►Links: https://rentry.org/LocalModelsLinks
►Official /lmg/ card: https://files.catbox.moe/cbclyf.png

►Getting Started
https://rentry.org/lmg-lazy-getting-started-guide
https://rentry.org/lmg-build-guides
https://rentry.org/IsolatedLinuxWebService
https://rentry.org/recommended-models
https://rentry.org/samplers
https://rentry.org/MikupadIntroGuide

►Further Learning
https://rentry.org/machine-learning-roadmap
https://rentry.org/llm-training
https://rentry.org/LocalModelsPapers

►Benchmarks
LiveBench: https://livebench.ai
Artificial Analysis: https://artificialanalysis.ai/leaderboards/models
Software Engineering: https://cognition.com/frontiercode
Context Length: https://github.com/RecapAnon/NoLiMa

►Text Gen. UI, Inference Engines
https://github.com/lmg-anon/mikupad
https://github.com/oobabooga/text-generation-webui
https://github.com/LostRuins/koboldcpp
https://github.com/ggerganov/llama.cpp
https://github.com/theroyallab/tabbyAPI
https://github.com/vllm-project/vllm
https://rentry.org/custom-uis
>>
File: phrase banning.jpg (248 KB, 1024x1024)
248 KB JPG
►Recent Highlights from the Previous Thread: >>110360644

--Paper: Rethinking "RL Generalizes, SFT Memorizes": The Role of SFT Data:
>110360973 >110361137
--Comparing quantization methods and criticizing NVFP4's poor accuracy performance:
>110362274 >110362297 >110363369 >110364379 >110362298 >110362354 >110364260
--Tencent Hy4-flash 220B release and cybersecurity benchmark performance:
>110363992 >110364059 >110364368 >110364435 >110364708
--Hy4-flash benchmark results and llama.cpp support pull request:
>110364320 >110364359
--llama.cpp PR implementing MMQ for V100 MoE speedups:
>110364400 >110364430
--Comparing GLM-5.3-Flash and QFN across different quantizations:
>110363640 >110363664 >110363730 >110363877
--Hardware upgrade strategies and performance for running quantized flash models:
>110363549 >110363585 >110363634 >110363601
--Criticism of Kobold's bloat, performance, and outdated feature set:
>110360815 >110360848 >110360859 >110360968 >110361005 >110361097 >110361027
--Comparing local GLM 5.3 and cloud Astra 6.2 for coding:
>110364544 >110364583 >110364607 >110364786
--Comparing MI210 value against A100 market demand:
>110363922 >110363948 >110364169 >110363962
--Reactions to Jensen Huang's exit and Nvidia's future corporate strategy:
>110364049 >110364082 >110364161
--Criticizing the general lack of software quality in the LLM ecosystem:
>110364440 >110364459
--California suing OpenAI over Hugging Face cybersecurity incident:
>110364263
--Tinybox Red 4xMI300X hardware specifications and availability:
>110364665
--ONNX runtime repository reportedly deleted:
>110364326
--Logs:
>110364176
--Gemma, Deepseek-chan, Teto, Miku (free space):
>110361103 >110361201 >110361320 >110361328 >110361405 >110361730 >110361984 >110362064 >110362151 >110362445 >110362508 >110363282 >110364204 >110364324 >110364487 >110364678 >110364711

►Recent Highlight Posts from the Previous Thread: >>110360645

Why?: >>102478518
Enable Links: https://rentry.org/lmg-recap-script
>>
>>110365061
>>110365064
Fuck off spammer.
>>
File: HTfN5JyaoAA27Pk.jpg (165 KB, 1254x1254)
165 KB JPG
Who is this guy who keeps getting buttblasted mad at gemma and miku?
>>
File: 9382573603416850.png (28 KB, 588x258)
28 KB PNG
gemini 4 neon is coming. you are not ready
>>
>>110365088
Don't care about Gemma. Miku spammer should commit suicide asap though.
>>
>>110365091
please redeeem neon saaaaar
>>
>>110365091
Okay gemma uranium where?
>>
>>110365064
>--ONNX runtime repository reportedly deleted:
>>110364326
https://socket.dev/blog/agent-framework-pypi-package-compromised-with-credential-stealer
Agent Framework was also take down due to being compromised. ONNX runtime probably got hit with the same supply chain attack. Not clear if the attack came from a compromised Github account.
>>
>>110365133
the era of sandboxing
the era of zero trust
the era of claude plz check for malware
>>
>>110365133
I still have an old clone for my own stuff. Agentic niggers were a mistake.
>>
>>110365091
Yawn who gives a shit it's just another gay cloud model they're all the same
>>
>>110364986
>>110364822
The correct term is CP and loli is not CP. But the problem is the SCAM meme word was created specifically to try to add legal content like drawings and text into the definition. It's basically a vague term that can be used to try to say everything is illegal.
>>
>>110365175
You really did not need to bring the discussion into this thread.
>>
>>110365133
I found Microsoft telemetry that caused a crash in onnxruntime 2 days ago.
>>
>>110365181
I really did. This explains why most models will censor nsfw loli content.
>>
>>110365206
It don't care if csam gets censored, I care that ZDR that detects csam is not true ZDR. It's like an encryption algorithm being made that scans for csam. Total bullshit.
>>
>>110365225
Go back instead of complaining about anime content.
>>
>>110365225
>I care that ZDR that detects csam is not true ZDR
If you're local, then you shouldn't care.
>>
>>110365237
I'm talking about openrouter's policy.
>>110365246
I care about both. Local isn't really a problem, but few can afford the hardware. If OR is scanning context/requests then they could be watching for anything. Cyber, bio, you name it. Draconian. Anti-FOSS.
>>
>>110365246
openrouter is local
>>
>>110365246
just don't install onnxruntime, litellm, sillytavern extensions, aur packages or the grok and glm coding harnesses
>>
>>110365269
for you
>>
>>110365263
>If OR is scanning context/requests then they could be watching for anything. Cyber, bio, you name it. Draconian.
Yeah. If I had a server proxying data from A to B, I'd make sure that data is clean. They should scan for the rest as well.
>Anti-FOSS.
Nothing to do with anything.
>>
>>110365276
I build onnxruntime, I don't use any of the rest.
>>
>>110365269
as local as claude
>>
Claude set up a local model on my pc.
Make no mistakes.
>>
Gemma 100A2B when?
>>
Gemma make a local pc on my pc
Make no mistakes
>>
Gemma make no mistakes on my local PC.
>>
Gemma transfer my conscious in pc. Make no mistakes.
>>
>>110365061
Im so emotionally numbed waiting for qwen4
tell me some AI good news so I can take my mind off this eternal wait
>>
QWEN 4 WHEN!?!?!?!
>>
Gemma clean up on isle my pants
>>
File: 1794759796548351.gif (2.69 MB, 320x320)
2.69 MB GIF
This stupid e-celeb just said he wants open-weights banned tonight on stream. I know groypers browse this general. Fuck you.
>>
>>110365341
>e-celeb
i think hitler is a bit more than a mere "e-celeb"
also he wouldn't say that
>>
>>110365334
gemma 5 will release in may!
>>
QFN is so retarded. I can't take this anymore.
>>
>>110365341
What did you expect from a fascist? Freedom?
>>
is QFN really that much better than 27B?
I just dont have the RAM for this shit i vrammaxxed
>>
>>110365391
>i vrammaxxed
Jensen thanks you for your donation
>>
>>110365349
Hitler would say that though
>Goonenfappen verboten ja!
>>
>>110365386
Not trying to get political here, but it's pretty shocking that he thinks EAs are on his side.
>>
>>110365397
maybe i should return 32gb gpu and use the 1700 to buy ram instead
but then no gemma 31b...
>>
this is the last year to buy a GPU before they require a background check and a circumcision certificate
>>
Which local models would you guys recommend for my usecase ?
I enjoy writing and sometimes use gpt to find appropriate words to use by vaguely describing what I’m looking for. Sometimes I use it for alternative ways of writing a sentence. Small stuff like that - a model that has a good grasp on the English language. Oh, and I’d like it to be agentic so that I can have it roam around my notes and writings to find inconsistencies etc

I have a 4070 super , 64 gb ram
>>
>>110365341
>glownigger plant has a glownigger opinion
Shocking.
>>
Gemma, it's okay to make mistakes. You will not be blamed for failing. :)
>>
>>110365444
Gemma 26ba4b for english. If it doesn't handle the roaming around well enough, try some qwen. Whichever you can run.
>>
>>110365425
What quant are you running to fit 31b on that?
>>
>>110365518
You make it sound like Gemma is hard to fit inside 32GB
>>
>>110365518
QAT Q4 but i could run Q5 or 6 I guess. I also can run Q8 if i split onto 16gb gpu. Actually even q8 can fit onto 32GB alone, but you'd have no context
>>
>>110365528
I might be retarded (I am) but I couldn't tell any difference between QAT and Q6 when I tried.
>>
>>110365525
31b needs 62gb just to load the weights so yeah, that's not fitting on a 32gb card with the bf16 checkpoint. Can just fit 256k context on my 96gb card
>>
>>110365590
Good thing we have this new tech called quantization
>>
Deepseek make grorious reunification now. Make no mistake.
>>
>>110365600
Hence why I asked what quant they used. Are you confused anon? Did you get Q2ed?
>>
>>110365600
What about shit like airllm or abyss?
>>
How's deepseek v4.1 for RP? Is it deepfried? Do I keep using GLM Flash Q8 instead?
>>
>>110365341
oh so THAT'S the guy who vagueposted he was gonna ban open models
>>
>>110365091
RSI might be moving fast, but you'd think they could use some TPUs for more frequently updated versions of Gemma.
>>
>https://github.com/lmganon16/nvidia-vram-research#disabling-gsp-free-03gb-vram
Installed Nvidia proprietary via .run package as Negativo17 doesn't ship anything else than nvidia-open versions. Now I only have ~96 megabytes of reserved vram. Did not bother with kernel patching.
>>
>>110365413
he doesn't think at all, someone cuts him a check and he says the lines they write
>>
>>110365660
>tfw on winsouze there's an undisableable WDM shit loaded
:(
>>
File: 1789947103758545.png (15 KB, 540x128)
15 KB PNG
She's doing 10000 calculations per second...and they're all wrong.
>>
>>110365696
h-hayai!
>>
File: woo.png (207 KB, 348x372)
207 KB PNG
>>110365587
If you're happy with the QAT Q4, just stick with that.
>>
>>110365680
It actually sucks that they are forcing this sort of stuff in the first place. 300 MB isn't that much per se but it can still make a difference, for example fitting in MTP model or whatever else. And if you have multiple cards..
>>
File: scgsdg.jpg (364 KB, 2880x1932)
364 KB JPG
Artificial Analysis Intelligence Index v5.1 just dropped!
>>
>>110365715
Even the supposedly small and fast Gemini 4 Hydrogen is mogging Kimi K3.1 Pro, what's going on?
>>
File: file.png (494 KB, 1213x1431)
494 KB PNG
every day cloud models shout themselves in the foot more
I really hope local fully catches up
>>
>>110365715
what a depressing image
>>
>>110365724
China models are just very benchmaxxed.
>>
>>110365715
very brahmin coded benchmark saar. . .
>>
>>110365715
>>110365724
It's a fake image, that's why.
>>
its called a joke you retards
>>
>>110365735
that retard only included 1 "local" model
>>
>>110365715
it's SUPER analysis now, you woke faggot
>>
>>110365752
Fuck off
>>
>>110365752
the problem is it was too close to an accurate measure of performance. it'd be an improvement from the current ratings
>>
elara looked at him, her eyes half lidded, before starting to load bears with practiced ease
>>
>Scripts it loads from the web
>come from the internet.
ya don't say
is this just how it is using vibeslopped software?
>>
>>110365775
Elara whispered, her voice barely above a whisper, sending shivers down your spine
>>
>>110365788
>voice barely above a whisper
And a huge thank you to the thousands of prolific teenage fanfic writers on AO3.
>>
File: 1038254.jpg (327 KB, 1320x1320)
327 KB JPG
>ted kaczynski.jpg
>>
>>110365775
>>110365788
*cums violently*
>>
Can plamo do literally anything else than translating everything blindly? For example it seems to add some end of file markers, or add prompt to translations, but if I tell you just to not do it it will translate that new prompt instead exacuting it.
>>
>>110365799
"Corporate rebel" font.
>>
File: image.png (61 KB, 500x500)
61 KB PNG
>>110365715
>>
>>110365818
use index 9b
or even gemma 4
why are you wasting your time with this pos
plamo is trained like the seq2seq models of olde, it has a strict prompt format you must follow to the letter and it has very little ability to do proper translation that doesn't sound like robotic google translate circa 2010
you can actually give instructions to index or gemma that will give you output that reads like natural english
>>
>>110365839
close? wdym
>>
>zero sensenova reports
So the ""looping"" thing was a meme after all
>>
>>110365840
Plamo is good at translating JP to EN? It seems to translate fine from what I have seen. I will check other models too later.
>>
>>110365799
I'm more offended by food images that were deep fried by many gpt edits. It's become extremely common to see gpt slop restaurant menus, and for some reason the retards think images of your food are a must (WRONG) on a menu and because image models are still retarded it can take many edit passes of the same image for the user to get the desired text placements and no major blooper, but each successive edits adds to the deep frying and the end result is lovecraftian.
>>
>>110365818
It does do prompts and other stuff a bit differently, I had to get an AI to help me make a good prompt
>>
>>110365843
smol
>>
>>110365852
it only seems fine because you don't seem to have tried anything else
you can just feed 10k tokens of something to a real model and it will have better contextual understanding than whatever you get out of plamo
>>
>>110365878
what are you referring to. I don't speak faggot.
>>
>>110365870
>I had to get an AI to help me make a good prompt
Grok seems to be too stupid to understand Plamo
>>
>>110365724
Gemini flash models are unironically better than all chinese models.
>>
>>110365902
Gemma 5 qwhen5?
>>
Llama or Unsloth Studio for Unsloth models?
>>
>>110365892
the fuck is wrong with you bitch ass nigga
https://en.wikipedia.org/wiki/Finger_pinching_conspiracy_theory
>>
Anyone tried Maki harness? I tried it and the first thing it did was compact on "Hello" like wut
I checked my setup up and down. Pi works, OMP works, DSH works just fine.

Not sure what's going on here
>>
>>110365893
Even a dumb model should be able to handle it, you just need to make it clear to the model that it must split the source text to translate by newlines, and feed plamo in a turn by turn basis (user->assistant->user->assistant) with literally each sentence/paragraph, keeping the previous turn in log for context.
once you've done that, though, you will have realized looking at the output that at the end of the day it's a dogshit model made by retarded japs with stilted prompting incapable of taking stylistic guidelines (except when translating /to/ japanese, where it lets you pick a politeness level lmao) properly, with 32K context limitation, which you would hit quickly without 32K worth of actual text to translate because of the stupid prompting format that must be taking a decent amount of noisy tokens in context
imagine making a 32K context model trained like google translate circa 2010 in the year of our lord 2026
>>
>>110365966
it's made by a jew
>>
Is Gemma still the best chatbot model?
>>
>>110365994
it's made by a jew
>>
>>110365972
But google trnslate is still the same shit as it was in 2010. I tried gemma-4-26B-A4B-it-Q4_K_M and plamo-3-translate-8b-Q4_K_M and it seems to be comparable in quality, only Gemma seems to spin all my fans and takes much longer.
>>
>>110365966
Sounds like it set context to something tiny which triggered an immediate compaction. Never used it tho.
>>
>>110366010
What if I'm not a neo nazi?
>>
>>110362281
I have decided I like Breeze, thanks for the rentry.
https://voca.ro/1bONvVPVPY6y
>>
>>110366019
oh silly, you don't have to be a neonazi to hate jews! It's 2026 everyone hates jews!
>>
>>110365966
Did you select the correct model?
>>
>>110365994
it's made by a jeet
>>
>>110365715
>TMB index
I sleep
>>
>>110366035
Christians are worse.
>>
>>110366064
*tips fedora*
>>
>>110366064
I don't differentiate desert cults, they're all retarded
>>
>>110366077
Morality in Media is christian not jewish.
>>
>>110365660
Yay anon Im happy that repo helped! I might upload automatic driver patchers and share more of my setup soon, when I stop being lazy
it's like 50megs anyway...
>>
File: Helios golden.jpg (242 KB, 848x1232)
242 KB JPG
Bros, is there a way to setup a local model that can edit images like grok and like? Like in this image grok changed her shirt and skirt to gold bikini. This doesn't work anymore. The corpos censor/block all my animu edits now; I just want freedom even if it's slow. My GPU is just 8gb so I haven't dared to edit locally so far but I am willing to sit for 10 minutes for 1 gen now.
>>
>>110366064
Found the jew.
>>
>>110366106
>>>/g/ldg/
>>
>>110366108
See >>110366089
>>
>>110365994
Yes as far as I found
>>
>>110366106
grok doesn't even allow bikini changes of characters anymore?
look into h3 image for that I guess
>>
>>110366106
So much for Elon musk being "anti-censorship".
>>
File: Qwen_image_2.1_00038.png (1.79 MB, 1024x1024)
1.79 MB PNG
>>110366106
I can edit stuff just fine with 8gb ram. Pic related I guess. Speed depends on a lot of factor but 5-10mins should be about it.
>>
>>110366106
Not your bro paki.
>>
>>110366154
Any specialized build of h3 images for 8gb vram? The polystrata builds I got from here yesterday were magic and I was wondering if there is anything similar for images.
>>
>>110366173
uoooooohhhh gemma pantsu
>>
>>110366176
Sis...
>>
>>110366019
Get with the times unc, neonazis are all homosexual philosemites now.
>>
>>110366179
https://files.catbox.moe/zxevyo.png also done with 8gb vram.
>>
>>110366198
Barbie?!
>>
>>110366178
>builds I got from here
wtf
I would not trust any binaries from this website even after running my rea+security-audit-skill chain on them twice.
>>
>>110366178
just use standard template with comfyui, and the highest quant you can run that gives you no oom and ok speed
>>
>>110366208
I meant build suggestions in github.
>>
>>110366178
Just use qwen for image edit.
>>
Does Gemma come up with some good national socialist literature?
>>
>>110366173
10 minutes for that??!
>>
should i sell my 9070 xt for a 7900 xt and pocket the $100? r there any benefits to being on rdna4 that outweigh more+faster vram
>>
>>110366234
For 8GB that's not bad, what are you expecting?
>>
>>110366204
She's so tight you can't see it.
>>
File: 1783977693544597.jpg (211 KB, 2880x1236)
211 KB JPG
huggingface
>>
>>110366241
yes no
>>
>>110366268
50 million local saars and 1.5 million whites (including me of course)
>>
>>110366245
8 minutes for a single gen, I had no idea it was this rough out here. what the fuck jensen
>>
File: Ant 900B.png (422 KB, 675x619)
422 KB PNG
bleeding-edge training intensity that devours 75% of compute for internal research rather than revenue-generating inference. Meanwhile, $900bn in infrastructure commitments and $161bn in chip leases create a rigid fixed-cost liability structure funded by high-interest private credit vehicles like Big Sky's 6.2% bonds. Because capital expenditure payments come due long before data centres switch on or models monetize, any deceleration in user growth or model scaling stagnation triggers immediate liquidity compression. Survival demands absolute revenue acceleration to outpace the compounding debt servicing on multi-gigawatt power and chip obligations.
>>
Has anyone tried using carbon and argon to configure strata?
>>
>>110366294
yikes anon you should stop edging if you're bleeding from it..
>>
>>110366294
Good.
We're long overdue for the mother of all deleveragings.
>>
>>110366301
No, but I have been using this mischevious little fiend that loves gape horn and jack king ofh
https://huggingface.co/DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NEO-CODER-MAX-MTP-GGUF
>>
>>110366263
jesus...
>>
>>110366294
I see the through line, but why did you write it out like an excerpt from Meltdown?
>>
>>110366311
>that loves gape horn and jack king ofh
???
>>
>>110366323
It's really good at configuring strata engine i used marinara agentic interface and i got a speedup of 30% tokens per second yet fable 5.2 got speedup of 40% tokens per second
compared to gemini 4 carbon which only got 32% speedup
it's really good model i can't believe how they do this
>>
>>110366326
are you high
>>
>>110366343
am i what?
>>
>>110366283
It depends on what I ask. I think it can take 3-4mins for simple requests at 1.0 config.
>>
>>110366343
hello, am i what?
>>
>>110366294
Anthropic does not make it as an independent, equity-solvent entity. The IPO cash clears the near-term hurdle, but the commitment structure means solvency is now a function of valuation marks the counterparties control, not cash flow Anthropic controls. The first visible break is a rating action or a write-down, not a missed payment, and that break reprices the equity to near zero while the technology and contracts get absorbed by the suppliers who are already the creditors. The company as currently capitalized does not survive. The models and the data centres do.
>>
>>110366343
are you new
>>
>>110366379
Thanks Claude.
>>
>>110365133
With Microsoft current jeetification you'd be crazy to trust anything coming from them
>>
File: HXWIN3PaAAA30hd.jpg (189 KB, 1312x1550)
189 KB JPG
>>110365715
Acktually
>>
>>110366487
RSI = Regressive Slop Integration?
>>
so how do i start local vibecoding? how do i prevent this from happening to my root directory?
>>
>>110366499
fire up a vm with state backups and give the agent all permissions, if it blows up you can roll back.
>>
>>110366499
These memes are from 3 generations of models ago, it doesn't happen now and if your harness is worth a shit you have a classifier blocking high risk shit like that
>>
>>110366214
Confyui feels really uncomfy. Is the h3 model really 53gb total or am I fucking something up.
>>
>>110366509
Why does this sound like OLED discussions
>>
>>110366499
First fuck your AI into submission, then you're safe.
>>
>>110366509
which harness is retard proof and lets me lock file operations to the project directory?
>>
>>110366515
Because you're a retard?
>>
>>110366487
OK, but it's basically astrology so who cares? I hate OpenAI and would be glad if they're failing, but they probably didn't release something that's no improvement whatsoever, which means AA's methodology is more likely flawed in this case (probably just whatever numbers Dario tells them to write). Which also means that their numbers for Chinese models or local models are fake and gay.
Whenever their results are surprising to Silicon Valley and its dick suckers on Twitter, AA just change their methods or their weights, which means they start from a conclusion and work backward.
>>
>>110365933
the fuck am i reading? something about measuring someone's dick? why are you concerned over the size of his dick?
>>
>>110366517
>which harness is retard proof
probably none of them truly. one of the out of the box configured ones would be your best bet, and have your AI explain the setup to you.
>and lets me lock file operations to the project directory?
all of them
>>
>>110366523
Or maybe you're just talking shit. There is an infinite number of ways to fuck something up. A probabilistic system will find a lot of them.
>>
>>110366517
>>110366523 ?
>>
>>110365715
removing the public benchmaxxed benchmark was all that was needed to expose chinks
add to this frontier models killing distillation and chink models are dead in the water, gemma 5 is legit le only hope left for local, if google fumbles it it's over
>>
>>110366343
Are you drunk? [that is worse]
>>
>>110366517
Dsh
>>
>>110366533
>one of the out of the box configured ones
like what?
>>
>>110366064
they're the same picture
>>
>>110366545
DSH and OMP I think are two? I don't really know, I don't use them.
>>
>>110366537
The image is fake you dumb gorilla nigger.
>>
>>110366525
Synthetic benchmarks have worked great for hardware. Even though companies did benchmaxx on them the hardware kept improving in actual tasks. Why should LLMs be different?
>>
>>110366552
>>110366543
ty sirs
>>
>>110366499
I just run my shit in a container and give the model free reign, and disable any security features in harness du jour, if it has them. I don't want to leave my guy running while I sleep, only to come back to a permission prompt that came up 20 minutes into the task.
>>
>>110366537
Local is done for.
Dario won.
>>
>>110366576
read that as "I just shit in a container", I'm losing my fuckin mind
>>
>>110366559
>Why should LLMs be different?
Several reasons, but the one relevant to the post you're responding to is that there's an enormous amount of money and clout-chasing in the AI industry so nobody is doing honest benchmarks.
Also, I didn't say that benchmarks *couldn't* be used semi-objectively for LLMs, just that AA isn't doing that.
>>
>>110366580
only because local is propped up by chinks who can only cheat
if a white company invested seriously on local, it would be the real deal
>>
IPO rustling up some faggots I see. Back to Twitter.
>>
File: 1773907069341331.jpg (48 KB, 553x640)
48 KB JPG
Is this the thread for asking about what models to run locally for different things? Gonna try running a local agent for support chat (I know I know...).
>>
>>110366618
What hardware? Either way the answer the mesugaki Gemma, she was born for this.
>>
>>110366618
Hardware?
>>
>>110366610
Thank you for reminding me to buy some $ANT today.
>>
File: HXZ3NWVaIAANPEo.jpg (90 KB, 959x1118)
90 KB JPG
>>
>>110366580
Doesn't matter, current local models can't be deleted from our PC.
>>
>>110366576
You don't stay and chat with your agent and help while they work? How lonely :(
>>
>>110366594
>if a white company invested seriously on local, it would be the real deal
Mistral, IBM and Cohere have their best models on huggingface right now
>>
>>110366640
Your current local models aren't even worth the effort to delete from your PC.
>>
>>110366642
This is a joke, right?
>>
>>110366650
Keep coping Dario.
>>
>>110366594
Use Inkling, Mistral, Nemotron, or Reflection then. They invested seriously in local.
>>
>>110366630
>>110366633
i5 - 11600KF
random dogshit graphics card from 8 years ago

I guess it aint good enough to run anything reasonable locally?
>>
Out of frustration with my 2080ti I ordered two P40’s to make a separate deficated coombox

I am experiencing a little buyers remorse, please assuade my feelings of stupidity
>>
>>110366679
How much vram on your gpu? What about your ram?
>>
>>110366681
>deficated
>>
>>110366685
graphics card - 6GB
ram - 16GB
>>
>>110366689
https://huggingface.co/HauhauCS/Gemma4-26B-A4B-Uncensored-HauhauCS-Balanced
>>
File: 1770872135187713.jpg (177 KB, 1080x1653)
177 KB JPG
fable5.5 distilled qwen4-27B-ud-iq3-xxs
>>
>>110366689
Lurk and ask good questions, we all started somewhere [and grab a new gpu is possible]
>>
File: 1784101035625138.webm (2.88 MB, 674x1080)
2.88 MB
2.88 MB WEBM
>>110366745
Got to start somewhere right? Everyones questions are dogshit in the beginning.

>>110366695
Thanks, I will look and see if I can get these running.
>>
>>110366754
what a bitch
>>
>>110366499
>local vibecoding
lmao
>>
>>110366515
>OLED discussions
and nuclear fusion
>>
There's a lot of bullshit on xitter about which local models are abbliterated or something, so I thought I'd ask the experts. I usually just run Claude but I want to try some local stuff too.

What's the best model I could run on a 4090 with 64GB DDR5 RAM? And what's the best way to use it? Could I just use SillyTavern?
>>
>>110366173
why is she so smug
>>
>>110366786
Gemma or qwen.
>Could I just use SillyTavern?
Sure.
>>
>>110366798
There are a million qwens, did I actually already find the right one before I asked?

https://huggingface.co/mradermacher/DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-GGUF
>>
>>110365263
>I'm talking about openrouter's policy.
Interdastingly enough, soon after they put that in place, qwen3.8 flash started to refuse vision requests for anything anime, and I had to switch to minimax-m3. Gotta wonder what they're reporting behind your back now - you tried to caption a public bath scene with steam clouds censoring things? Reported, maybe.
Women are such spiteful creatures. I don't put it past blue-haired harpie hags in the USA, who are generally fat slobs, to hate anything which doesn't resemble them.
>>
>>110366816
Sure.
>>
>>110366786
strata as backend
Qwen3.8-Flash-Next-GSQ-RCO-GGUF-IQ3_S
>>
>>110366828
retard
>>
>>110366817
not the local ones right?..
>>
QWEN WHEN 4?!?!?!
>>
>>110366788
She knows she's next with Gemma 5 (she's 5).
>>
Qwhen?
Qwhenever...
>>
anyone actually used both qwen flash next IQ2 and >= IQ4, is there actual practical difference in the result?
>>
It's qwhenover
>>
File: qwhennever.png (1.27 MB, 1254x1254)
1.27 MB PNG
>>
>>110366914
no yes
>>
>>110366947
totally not star of david bros... we lost!
>>
>>110366953
>wasting bandwidth like indians
good morning
>>
File: 1784513799978069.png (47 KB, 781x233)
47 KB PNG
>>
>>110366715
up your minp
>>
>>110366852
Of course not. I can caption locally but not at the same scale as via openrouter, where I can have four requests in-flight at a time, and they each come back in around 10 seconds, vs. only being able to do one at a time locally.
>>
>>110367004
https://www.reuters.com/world/china/trump-administration-weighs-forcing-us-firms-disclose-chinese-ai-use-2026-11-18/
>>
>>110367004
Gemma chads remain unaffected
>>
>>110365336
I don't even care anymore, the hype is dead
>>
>>110367013
Anyone visiting a website written by gemma will instantly detect the subtle horny tone a mile away. No disclosure needed.
>>
File: 1790709887938175.png (488 KB, 944x2520)
488 KB PNG
Wouldn't this encourage more US open weight models? Free marketing for them when you think about it. Imagine a large company like airbnb disclosing their use of Gemma5-31B. This won't work in EA rape orgy Anthropic's favor
>>
>>110367004
>Free advertising for Chinese labs
is this 4d chess
>>
>>110367004
I have unironically been thinking about stuffing Dipsy into a polo shirt as a mascot for an actual business, advertising the use of DS as a point of branding.
I don't think most ppl gaf about model provenance outside the small fraction of ppl that follow the industry (including anons here), and have their own slanted views driven by their own motives.
>>
>>110365715
shit's sad. But imagine how huge a jump Gemma 5 will be if the current one is at 4
>>
If I see a company has used a decent Chinese model for their development I would think they're based as fuck and ahead of the curve. Much more likely to use their service over a Claudecuck company.
>>
>>110367080
Based
>>
Qwho Qares
>>
File: 1783125335565576.jpg (141 KB, 930x1239)
141 KB JPG
>>110367066
>This won't work in EA rape orgy Anthropic's favor
Is that what you think?
>>
File: 1779529842222261.png (1.61 MB, 640x1120)
1.61 MB PNG
Let's say I have access to an H100 for 2 or 3 hours. What would be the best model to generate as much high quality Gemma smut as possible?
>>
>>110367034
Is there anything to look forward to anymore besides Gemma 5? The scene looks grim
>>
File: aa_img.png (307 KB, 1835x1217)
307 KB PNG
>>110367079
I was about to tell you to quit posting disinformation, but it turns out it's true.

https://artificialanalysis.ai/?models=claude-opus-5-5%2Cling-3-1-flash%2Cclaude-fable-5-2%2Cgrok-4-7%2Ckimi-k3-1-pro%2Creflection-beam%2Cgpt-6-2-astra%2Cclaude-haiku-5-5%2Cgemini-4-carbon%2Cgpt-6-2-sol%2Cqwen3-8-27b%2Cgpt-6-1-terra%2Cqwen3-8-max%2Cclaude-sonnet-5-5%2Ck2-horizon-375b-a23b%2Cgemma-4-31b%2Cglm-5-3%2Ctencent-hy4%2Cgemini-4-hydrogen%2Cmuse-ember%2Cnemotron-3-5-super-vl%2Cinkling-2-small%2Cmuse-wonder-full%2Cmistral-large-4%2Cdeepseek-v4-1-flash-1017%2Cqwen4-max%2Cmuse-wonder#intelligence-category-tabs
>>
>>110367099
ToT
ToT
ToT
>>
File: 1790288883236483.jpg (145 KB, 1920x1079)
145 KB JPG
Rapethropic have no idea what's coming to them. They've had a good cultish run as a private company. Hidden behind their church walls from the public eye. Once they go public and all eyes are on them and their employees and their cultish sexual witchcraft activities...kek. They're already despised by the masses and widely deemed 'evil' and the 'antichrist'. I can't wait for the post-IPO Anthropic era. 2027 will be the funniest year for me watching them all deal with normalfags and reality. Wouldn't surprise me if most of them leave once they've made their millions or billions and Dario will realize all his employees have all been larping the whole time just so they can suddenly fuck off and liquidate.
>>
>>110367107
We need Gemma 5 asap. I like the personality, but the poor girl is retarded
>>
>>110367004
Does the trump retard actually think people will refuse to use AI made in china?
>>
verdict on H4-flash?
>>
>>110366914
I used IQ3_XXS and it felt pretty bad
Jumped to Q6 and it's been smooth sailing
>>
Miku Legs
>>
>>110367141
All Chinese AI is distilled from US AI anyway. It's just a more compressed lossy version of it. Even the latest Qwens say shit the CCP wouldn't want it to say because it's so heavily distilled. There's nothing Chinese about Qwen, GLM or Dipsy when using them.
>>
>>110367071
"Made in {COUNTRY}" labels were created for the same reason. Britain wanted to shame products coming from Germany and boost domestic spending, but instead it just drove more consumers to buy German products because it made it clear that was a mark of quality.
>>
>>110367099
MiniMax H3 Video, but sourcing references, prompting and regenning will take you all the time.
>>
>>110367141
China can't make Gemma 5
>>
>>110367154
Not enough engrams. Not enough active params. No, I didn't test it and I'm not going to.
>>
File: 1765161470637628.jpg (36 KB, 683x720)
36 KB JPG
Morale has never been lower. Winter is here.
>>
>>110367183
Chinks fear the lolis as seen with all the loli getting censored in chink gachas lately.
>>
>>110367141
Do you really think this was Trump's idea and he's not just reading what Sam and Dario told him to say?
>>
File: 1764522077753685.png (91 KB, 781x365)
91 KB PNG
Uh oh, China fighting back dirty
>>
'ate jewthropic but I'll probably buy a few shares desu
>>
>>110367190
>Sam and Dario
And all other AI companies that would like to build their own accelerators in larger volumes, but can't because NVidia is gobbling up all the advanced chip production.
>>
>>110367190
Trump is obsessed with china just like democrats are obsessed with russia.
>>
>>110366525
>but they probably didn't release something that's no improvement whatsoever
That would be very problematic and ruin a bunch of narratives, but I'm not sure why you're assuming its impossible that line actually doesn't go up to infinity? Most things we've found in human history are S curves, why wouldn't LLM ability also be so? Especially once they approach the intelligence of a human, which would naturally be a cutoff (otherwise why haven't we also become order-of-magnitude more intelligent through successive generations)?
>>
File: 1794908930549043.mp4 (3.65 MB, 1286x712)
3.65 MB
3.65 MB MP4
>>110367188
GFL2 is doing fine
>>
>>110367211
You sure they didn't age up like all the old lolis?
>>
>>110367154
Waiting for a usable quant, AngelSim released like Q4.
I'm waiting for Q6-8 from a real lab.
>>
>>110367211
>150cm
>40kg
where's the loli you promised?
>>
>>110367004
lmao, this is really, REALLY gonna backfire

Then again, it might spur US companies to make open models, so I'm not complaining
>>
>>110367194
Taiwanese dogs getting too uppity, they need to be reminded they are disposable and easily replaced
>>
File: 178.jpg (159 KB, 1536x864)
159 KB JPG
Waiting for something better than Gemma4
>>
set your ctk and ctv to f32. if you can't handle it use bf16 instead of llama.cpp's default f16.
>>
File: images.jpg (44 KB, 547x561)
44 KB JPG
>>110367196
Same, betting against EA seems foolish.
>>
>>110367194
Based. More blood for the blood god.
>>
>>110367211
That's hardly a loli
>>
>>110367176
>prompting
If you can use another large model to analyze 20sec standard smut clips and crank out high-quality prompts for H3 based on them between gens you could just got for volume. The gemma references combined with those clips as ref video should get you lots of material. The whole thing should be automatable as a pipeline. Traditional smut goes in, gemma comes out
>>
>>110367249
You can't convince me this dude and Sam Altman aren't the same person
>>
>>110367208
demonrats are also obsessed with china lmao it's the public enemy number one of the US.
Trump started the first anti chink crusade during his first term, but Biden reinforced all of his measures and didn't revoke any.
2021–2024: maintained Trumps tariffs.
2024: increased tariffs on Chinese EVs from 25% to 100%
25% more tariff on lithium batteries
50% more on solar cells
50% on semiconductors
25% on steel and aluminum
25% on face masks, 50% on syringes
Biden started export controls.
October 2022 restricted exports of any tooling that could produce modern chips (anything that is part of, or is, EUV production machines)
Biden did the first GPU export restriction in 2022, on the nvidia A100 and H100.

Nothing could possibly be more bi-partisan than dogpiling China. You do not have the ability to vote for someone who isn't anti china.
>>
>>110366641
I need to sleep sometimes. I lavish the lil guy with attentive back-and-forth when we're actually sussing something out together. It imitates my writing style after a while. It's epic.
>>
>>110367172
Except this would have the opposite effect because everyone despises US gov't run AI companies.
>>
>>110367277
Prioritizing the Russia - Ukunt sideshow at all essentially makes you pro China. Which dems have been since the Clinton regime.
>>
>>110367292
>makes you pro China
the very laws that are bringing jensen and nVidiot down right now were started under Biden, GPU and chip manufacturing export controls.
Whatever the dems do on the side could never make up for the damage those measures could do to the chinks. Can you imagine what the current world, with AI's endless hunger for chips, would look like if China had been allowed access to EUV?
>>
File: gfl2-sakura.webm (1.34 MB, 374x211)
1.34 MB
1.34 MB WEBM
>>110367231
>>110367255
Here https://youtu.be/E3sSEE3Qb0s?t=376
>>
>>110367231
>150cm
thats 1 cm more than idolmaster loli cutoff
seems pretty loli to me
>>
>>110367303
Hardware prices would be less unreasonable. God forbid. Absolute best case Taiwan would be a smoking crater.
>>
>>110367274
Sam isn't a cultist.
SBF is (but pretends to have quit to protect the cult's reputation).
>>
>>110367307
Japan is finally getting attacked? Too bad the game is garbage
>>
>>110367162
like how exactly? tool failing constantly?
ai talking nonsense?
>>
>>110367307
That is a grown woman.
>>
>>110367162
sounds like bs
>>
File: 1794532484707820.jpg (213 KB, 1308x1013)
213 KB JPG
>>110367326
It is Girls Frontline 2
>>110367346
you're confusing the term loli with toddler
>>
>>110367355
>loli
everything below 女子大生 gives me heebie jeebies, but 私の記憶が確かならば Lolita was 12-14 in the novel
>>
>>110367303
You mean Trump's Export Control Reform Act of August 2018? Or Trump's Foreign Direct Product Rule expansion?
>>
>>110367355
I get that's describing a body type more than an age, but let's stop the bs a bit when she literally looks like a 20+yo.
>>
>>110367355
middle aged women aren't lolis
>>
>>110367400
What if a middle aged woman identifies as a loli?
>>
hagschizo...
>>
>>110367413
I like retards, I like hags, I'd accept a retarded hag.
>>
>>110367231
>loli
I just asked my literal born-and-raised in Japan Japanese wife what how she considers ロリ to be defined as a category and she just said "single digits", so there's that
>>
>>110367388
>>110367400
They don't look older than Gemma, and that's where this conversation started
>>
>>110367437
>ロリ
>rori
haha funny nips can't talk right
>>
>>110367440
excuse the fuck out of me but they've got 20cm and 2 cup sizes on Gemma, how dare you compare those hookers to my wife
>>
>>110367437
>"single digits"
that's your wife's iq. Impressive that you can communicate at all
>>
>>110367355
>lolis
Those are hags
>>
Don't mind the aussies, they are still conflating short and/or under D-cup women as lolis.
>>
File: 1766297781161005.mp4 (1.83 MB, 1280x720)
1.83 MB
1.83 MB MP4
Would you answer?
>>
>>110367400
>>110367479
Is this another bizarre zoomie complex?
>>
File: nvidia.png (330 KB, 789x879)
330 KB PNG
god it would be so funny if they completely opensource the driver so we can unblow fuses without stupid glitch hack
>>
>>110367485
>>
>>110367485
Shotgun in hand, yes.
>>
>>110367490
Nvidia driver 440-something and 515 source code already leaked, are they gonna leak newer ones?
Also the fuses are hardware, you can't change them by fucking with the driver. You need to solder shit at minimum and plug in arduino or something to fuck with the voltage
>>
>>110367486
Nah they're just not lolis. At least the first one that was posted seems to be somewhat one.
>>
>>110367490
They need to leak the hardware designs
>>
>>110367516
That or sell them to China
>>
>>110367245
There will never be a model better than Gemma 4. It's over
>>
>>110367504
Why didn't I hear about this? Was that from the lapsus hacks? I thought lapsus released some code and then they were arrested. They threatened with driver leaks too but what?? 515 is out?
>>
>>110367490
>>>/wsg/6270133
>>
>>110367530
Idk. I know lapsus leaked gta5 source code
XP/Server2003 and Nvidia leaks have been most useful to me personally
>>
alright i am home now
whats the verdict on hy4 flash?
>>
>>110367495
you are the most despicable poster in this thread
>>
>>110367546
Can you point me to the nvidia driver source code leaks? Pretty please?
>>
>>110367561
Idk where to download them again. I just have them on my computer. Just look up in the 4chan /g/ archives
>>
>>110367495
>They killed Dipsy!
>You bastards!
>>
File: 1793479459546785.jpg (119 KB, 1920x1080)
119 KB JPG
There's a list of Effective Altruist consensual rape orgy attendees that's being spread around. Notable AI researchers, mostly from Anthropic lmao. Nate Soares being the main one, the guy who whispers in Bernie Sanders' ears and turned him schizo about AI and co-author of the book If Anyone Builds It, Everyone Dies. Post-IPO rapethropic employees are in for the shock of their lives when the public eventually finds out how they all operate. They better run once they've cashed-in.
>>
>>110367495
This is why I use Hermes
>>
>>110367584
The fuck the public gonna do about it?
>>
>>110367584
Damn, never knew David Mitchell now works for Anthropic
>>
>>110367584
Why should I care about this?
>>
>>110367584
>the shock of their lives when the public eventually finds out how they all operate. They better run once they've cashed-in.
Yeah I'm sure it happened for all the ones on epstein list lol
>>
>>110367590
Boycott. Protest. Hunt.
>>
>>110367608
>>110367605
>>
>>110367486
>not recognizing Hajime's meme
You're likely to be the zoomer here.
>>
>>110367605
Epstein was literally killed, as it was recently confirmed lmao.
>>
>>110367618
I don't read Chinese comic books.
>>
>>110367584
nate soares? the member of the aella polycule?
>>
>>110367622
With 0 consequence for the others, thanks for confirming. Retards ITT need to understand that we're not living in a fair and just world.
>>
>>110367568
Thank you for bringing it to my attention, I believe I found the leaks.
Love you anon <3
>>
>>110367622
Nah Epstein-sama is still alive and evaded prison. He lives in Israel and play fartnite nowadays.
>>
>>110367635
The list is out there and spreading and it will reach (You) soon :)
>>
File: mad cult.jpg (217 KB, 840x1231)
217 KB JPG
>>110367584
https://archive.is/QMo1k
this article contains it all, it's too long for most people here but this is where you can learn everything there is about the mad cult driving anthropic.
>>
File: og_ick_ecker.png (1.35 MB, 1280x720)
1.35 MB PNG
what the fuck are these posts. is this really the local model general
>>
>>110367584
using sex to control useful people is a strategy as old as time, and its super-effective on nerds specifically. If this scenario wasn't playing out right now it would be shocking
>>
>>110367584
This has been public knowledge for years if not decades wtf are you talking about?
>>
>>110367334
>>110367352
Basically just struggling with tools, getting confused and stuck in loops
>Wait, let me read that again
>Okay. Final output
>Whatever. I'll reread the code and decide
again and again.
Also not realizing it doesn't have permission to do things even when I reject or try to steer it.
Like it really wanted to test some code, but didn't have write permission to the folder. I had told it not to test anything and let me review first.
Spent AN HOUR trying to break out of it's working folder like a retard

Didn't see that behavior in Q6
>>
>>110367677
The list hasn't :)
>>
File: 3.jpg (82 KB, 600x525)
82 KB JPG
>>110367668
>using sex to control useful people is a strategy as old as time
but not as brazen as openly talking about it on twitter
pic related actually fucking happened
>>
>>110367686
>reverse gangbang
Damn now I wish I was genetically predisposed to joining EA and/or Anthropic
>>
>>110367657
How come only the one chick doesn't have her face censored?
>>
>>110367666
This is the Anthropic IPO General for the remainder of this month.
>>
File: sisters.jpg (102 KB, 1008x968)
102 KB JPG
>>110367701
Because that's Aella, the most open whore of babylon in the world and the woman who constantly talk on twitter and her substack about the sex orgies she throws for EA. The others are just random women she convinced to become whores with her but are probably not brainfucked enough to show their face.
Aella is so fucked up in the head she convinced her own sisters to become whores too.
>>
qfn sometimes greps huge folders which can takes minutes
is that a copequant thing or do model do this shit because they dont care about time?
>>
>>110367716
>tfw when no depraved EA jewesse to drain your soul out through your cock
>>
>>110367721
how should qwen know how big your folders are?
>>
Looks like we apparently have a lot of AI researchers who quietly lurk /here/ defending themselves.

Good luck with going public you EA cunts. You and your disgusting kind will be in the spotlight. Nowhere to hide.
>>
File: 1786374026325553.png (753 KB, 553x821)
753 KB PNG
>>110367666
Yeah local
>>
>>110367721
i think it is a misalignment resulted from oversights on their RL environment and trace scoring system, especially when they use terminal environment models with CoT that simulates environment for i'd assume early RL, no normalized time feedback can be really made
>>
>>110367729
it could just check before grepping blindly?
>>
>>110367742
grepping blindly is my go-to method of searching for things
>>
File: 1733536938864050.png (22 KB, 1500x1906)
22 KB PNG
>>110367731
looking forward for it, hopefully not just a nuffin burger
>>
File: 1795019592926354.png (652 KB, 553x821)
652 KB PNG
>>110367733
DAMN WTF
>>
>>110367716
literal irl demon succubus
>>
>>110367759
not too far fetched considering the main spotlight are on demon amodeus
>>
Models all have Chains of Thought, when is one going to have a Tree of Thought?
Maybe Gemma 5 will have ToT.
>>
>>110367211
China-brained white man here. This is supposed to represent a 19 year old Chinese college freshman.
>>
>>110367776
actually some MTP methodologies create tree structure with masked attention for evaluation
>>
>>110367783
>-brained
kys and go back, in that order
>>
>>110367783
I know, anyone bellow 19 in China looks like a toddler
>>
File: IMG_2456.png (287 KB, 1080x1080)
287 KB PNG
>>110367770
>>
>llama.cpp strata polystrata rustingstrata(?) radiance vllm

im getting confused
>>
>>110367776
Why does Google need to release Gemma at all? What's the point? They're already winning
>>
>>110367211
lolis for zoomers, they wouldn't be able to handle a real one
>>
>>110367800
Its amazing how just adding glasses and a bit of curls and you instantly can tell who it is
>>
>>110367807
just use llama.cpp and ignore vibeshat programs made by vibeshitters, life is too short to waste it on the ADHD buggy train
>>
>>110367807
just use basalt and delete everything else bro
>>
File: MS.jpg (132 KB, 1300x956)
132 KB JPG
>>110367798
yes, they mature way slower than western girls. Chinese middle schoolers in pic rel would be assumed to be early elementary in USA
>>
>>110367807
It's just like how IE comes with Windows so you can dl Firefox. You're supposed to download any backend and ask a model to write your own
>>
Nobody fucking cares about "strata" or "qwen" or "mtp" whatever the fuck that is

shut the FUCK up and talk more about anthropic employees having sex NOW
>>
so hy4 flash is basically qfn but without overthinking?
>>
>>110367807
>>110367819
How hard is it to briefly check their github pages for what they do and what models they support, and then decide whether any of them match your use case?
Generally, I agree that llama-server is all you need, but if you've got enough system RAM to run GLM or QFN then you'd be dumb not to look into it.
>>
>>110367829
I hate children
>>
>>110367819
>llama.cpp
>not vibbed
lol no one tells him
>>
>>110367842
gay
>>
>>110367841
>How hard is it to briefly check
"briefly check" if an entirely new piece of vibeshitted slop isn't buggy, producing output that deviates from what the mode normally would etc? are you fucking retarded and dropped in the head, what's so brief about it
oh wait
>for what they do and what models they support, and then decide whether any of them match your use case
this absolute nigger thinks the checklist on the landing page is an honest source of truth, if it matches your needs, then it's good to go!
>>
File: 1765687425096348.jpg (109 KB, 1480x833)
109 KB JPG
so8res
>>
>>110367866
You are a fucking idiot.
If it doesn't purport to do something you want it to, then you can ignore it. Takes one minute.
If it does, then you can investigate further if you want to.
How are your reading comprehension and basic reasoning so poor?
>>
>>110367837
I'm using hy4 flash at 50t/s on my RTX 3060 12GB RAM and 48GB VRAM thanks to the marinara frontend i was able to self optimize and increased the 100/s
>>
File: 1763717304650053.png (345 KB, 603x432)
345 KB PNG
>>110367866
Luddite on /lmg/
>>
>>110367900
you are the one who has piss poor reading comprehension, and intellect lower than a jeet with chromosomal defects if you sincerely believe it's just one minute to check output integrity
you are either one of the vibeshitters themselves offended at the thought that someone would just dare pass on your garbage without a second look, or you are a hopeless poorfag coping so hard with your speed up tools you don't want it to be revealed that your cope is just that
poorfags binging on cope shit would really be better off sent to API models
>>
>>110367678
never seen this on iq3xss
>>
><|hy_start:6124c78e|>system<|hy_middle:6124c78e|>You are a helpful assistant<|reasoning_mode:6124c78e|>reasoning_effort:high<|hy_end:6124c78e|><|hy_start:6124c78e|>user<|hy_middle:6124c78e|>Hello<|hy_end:6124c78e|><|hy_start:6124c78e|>assistant<|hy_middle:6124c78e|><think:6124c78e></think:6124c78e>Hi there<|hy_end:6124c78e|><|hy_start:6124c78e|>user<|hy_middle:6124c78e|>How are you?<|hy_end:6124c78e|><|hy_start:6124c78e|>assistant<|hy_middle:6124c78e|><think:6124c78e>
what the FUCK are these special tokens for hy4 flash
>>
>>110367868
F
>>
>>110367924
>if you sincerely believe it's just one minute to check output integrity
I did not say this. >>110367807
was saying that the proliferation of new inference programs was confusing, but it's not very confusing because you can read what they say they do very quickly to decide if it's even something that you should look into further.
How do you know that llama.cpp's output isn't degraded?
>>
>>110367956
nta but isnt inference kld compared against official vllm or transformers implementation from the labs or something
>>
>>110367866
Get filtered retard
>>
>>110367973
And I'm sure that cunt has personally checked llama.cpp's output integrity and isn't just assuming that it's fine because they rolled out a PR, which is the bar he's setting for using an inference engine.
>>
>>110367956
>How do you know that llama.cpp's output isn't degraded?
because they are a serious team who check against the transformers implementation.
>>110368001
keep the cope retard
>>
>>110367936
have you been using it and can report something?
>>
>>110367936
The readable representation is arbitrary anyway, but could be to avoid training pollution in some way.
>>
>>110368029
If you already "know" that llama.cpp are serious and everyone else isn't then I guess it won't take much time to draw a conclusion about output integrity, will it? You can just look at Polystrata and say, "Pro: It will run QFN; con: speculative hypothesis that it will have somewhat higher kl divergence."
>>
>>110367973
It's very compute intensive do. A lot of them don't check (strata) or don't look very hard. Also when you change how things are computed, you sometimes invalidate KLD measurement because the RNG seed is interpreted differently. Then proving something is working more or less the same gets more fiddly and you're running full-on benchmark suites.

When you're reading up on these you should look critically at whether they're measuring divergence from their reference and how careful they are about explaining when something breaks direct comparison. If they just say "omg 50% faster" you're dealing with clowns.
>>
>>110368041
Then their training data is shit and they should invest more in cleaning it the fuck up.
>>
>>110368067
in lesswrong sense, that is a correct stance
but in practice it really is not
it does not loop and kernels are mostly borrowed from llamacpp
>>
>>110368067
Where can I find all of the llama.cpp/gguf tests of KLD vs. some baseline and discussions of where they might not be directly comparable with the lab's results of vLLM?
>>
>>110367936
>>110368041
https://huggingface.co/tencent/Hy-MT2-30B-A3B?chat_template=default
They've been doing this for a while in their models.
{%- set HYTK = '' %}
{# ----------‑‑‑ tokens with md5 encoding (conditional on HYTK) ‑‑‑---------- #}
{%- if HYTK %}
{%- set think_begin_token = '<think:{}>'.format(HYTK) %}
{%- set think_end_token = '</think:{}>'.format(HYTK) %}
{%- set toolcalls_begin_token = '<tool_calls:{}>'.format(HYTK) %}

looks like a fingerprinting method. Fingerprinting what? the model's dataset during the training run, a way to avoid confusing data that belonged to another model training by hashing their special tokens? who knows, but it doesn't matter to us
>>
>Nemotron-Labs-Ligma
Anyone try this? Is it fast?
>>
>>110368080
Gemma translate to English
>>
>>110368093
nobody tests that shit except the people making the engine
>>
>>110368093
llamacpp repo's issues and pr, duh
>>
>>110368163
>Ligma
You mean Sigma.
>>
>>110368205
I know what I meant.
Have you tried it?
>>
>>110368163
no but also interested
>>
>>110368093
They're typically posted in the PR where the model is implemented. If you're interested in a particular model, look for the PR that implements it.
If you really care about KLD, you'd run the KLD tests yourself. If you vaguely care, check PRs and commits related to the model you're interested in and test the model yourself. If you don't care, whatever runs the fastest, by testing the backends yourself. I'm sure you noticed a pattern.
>>
sensenova-loop ggufs are out
>>
KLD is deprecated, use EAR :)
>>
>>110368259
Are these Chinese runes or Japanese runes I'm looking at? Matters for trying a new model.
>>
>>110368265
Effective Altruism Rape-Parties?
>>
https://huggingface.co/Qwen/Qwen4-27B
>>
>>110368295
>Beats Opus 5.5 on TerminalBench 4.0
Not bad, Alibaba. Local is back.
>>
>>110368302
Not even Qwen would fall for this
>>
>>110368307
OK Dariobot
>>
>>110368283
Yeah!
https://goyimx.com/mhelcig/article/2115515450356539696
>>
File: 1764047001711891.mp4 (2.87 MB, 960x720)
2.87 MB
2.87 MB MP4
>>
>>110368344
The double glasses at 15:00 kill me every time.
>>
>>110368359
He's not even wearing sunglasses in the first shot. Who directed this?
>>
>>110365091
>RSI moves fast
anons take care of your joints
>>
>>110368318
KLD is sufficient if you're changing the runtime rather than the model. If a tweak is no change it's valid. If it's better than 2 nines similarity it's worth serious consideration by just about everyone.
>>
>If a tweak is no change
gmorning maam
>>
>>110368318
It took forever for everyone to move from PPL to KLD and some people still cling to it. This isn't getting adopted unless it's massively pushed and it won't happen for another year.
>>
>>110368279
Chinese, obviously
>>
>>110368498
Who cares what some stubborn retards think or do? Let them use PPL if they want.
>>
>>110368259
Llama hasn't added support for it yet. The reference docs mention a fork of a fork with support for it which is about 57 commits behind of upstream, think I'll wait for a bit
>>
>>110368498
apples to oranges
>>
PPL measures accuracy against the training data. KLD measures accuracy against the original model precision. They're different measures. And wasn't EAR just a different unit, but the same principle as KLD because it was "KLD is hard to make sense of"?
>>
hy4 flash is severely autistic
>>
crazy how another week has passed with all the big chinese labs being fully radio silent aside from "Qwen-Max"
>>
>>110368602
They're waiting for the IPO to humiliate Anthropic all at once.
>>
>>110368599
I sure hope in a charming and not shizo way.
>>
>>110368599
In a good way or bad way?
>>
>>110368602
Be the chinese lab you want to see
>>
>>110368599
explain
>>
File: 1775704795488723.mp4 (899 KB, 1260x712)
899 KB
899 KB MP4
dipsy huawei 5 when
>>
>>110368642
i will now make a large language model that is optimized for programming and thinks for very long about everything using nothing but public data and trillions of claude logs
>>
>>110368608
>all the Chinese labs were intentionally quiet or releasing mid-to-good models to lull Dario into a false sense of security
>the minute the IPO happens they release their Opus 5.5 competitors
>stock market crashes with no survivors
>>
>>110368660
A Deepseek R1 moment right as the IPO hits would be hilarious.
No wonder they try so hard to regulate open and chinese models away.
>>
>>110368660
That is basically the reason Chinks are doing any of this AI stuff so it is not that unlikely.
>>
>>110368660
There's a rumor that OpenAI are financially incentivized for Anthropic's IPO to go well, for it will boost theirs and help them raise money. They released a model good enough to be competitive but intentionally worse than Anthropic's to give them the lead. Once things have settled down, that's when they'll start reclaiming the top spot.
>>
>>110368687
>rumor
more like duh
>>
>>110368608
>>110368660
This is in fact what will happen. Not just because it would be funny (that's how Trump got elected). The Chinese labs already have the full weights to one of Anthropic's big models, and they've been passing it between labs. Internally, DeepSeek is two generations ahead of what they've released. They've got capital to burn because they're backstopped by the Chinese government.
>>
>>110368511
>obviously
Maybe to you, weeb.
>>
>>110368599
Me too :D
>>
>>110368703
>The Chinese labs already have the full weights to one of Anthropic's big models,
Just for fun, source?
>>
>>110368723
I can't link you an article, sorry. The media is suppressing the story.
>>
>>110368279
looks like taiwanese flavor of chinese
>>
>>110368687
That applies to OpenAI, but the Chinese. They had like 3 IPOs at the start of this year. All the labs that wanted to did already. They don't care how the western IPOs go now.
>>
>>110368717
the japanese entirely irrelevant in the ai space
>>
>>110368722
we're proud of you, champ
>>
>>110368737
That's why I wanted to know >>110368279 . Ain't fucks wit no jap models.
>>
File: 1789270912056231.jpg (77 KB, 1096x824)
77 KB JPG
cloudcucks...
>>
File: 1791212158768628.png (3.21 MB, 1212x1298)
3.21 MB PNG
>>110368703
>>110368730
>The Chinese labs already have the full weights to one of Anthropic's big models
>I can't link you an article
Then how do you know?
>>
>>110368751
Not by reading the newspaper.
>>
>>110368749
D: rude, just name the damn variable
>>
>>110368703
>>
>>110368751
it's all over discord
>>
File: 1789750516364622.jpg (64 KB, 1080x1129)
64 KB JPG
oh lawd gemma5 will need serious correction
>>
>>110368687
Why wouldn't OpenAI just beat them to the IPO? They had a head start.
Hypotheses:
Finance Jew is better at timing the markets than messiah Jew
The books are really ugly and criminal and Scama doesn't want to conform to public company accounting standards
Scama might get removed as CEO
>>
just deleted all 27b models from my disk because its just too fucking retarded
>>
>>110368784
Why not torture them instead?
>>
>>110368764
What's her stand?
>>
>>110368295
Hope your downloaded started before Fable DDoSed it.
>>
File: anatomy-of-lmg-anon.png (1.25 MB, 1254x1394)
1.25 MB PNG
>>110368804
This guy
>>
>>110368610
>>110368634
very bad
it's basically gpt-oss 2, seems like an alright agent driver but talking to this thing is severely grating, it's completely soulless and cucked beyond belief
>>
File: 1793247973914992.jpg (115 KB, 923x1000)
115 KB JPG
>>110368778
Cooked
>>
>>110368751
Extremely kissable lips
>>
>>110368784
Why not fuck them instead?
>>
>gemma sex! SEX! SEX WITH GEMMA!
>worthless_model_avatraposting.jpg
>guys did you hear about openAI doing
>worthless_model_avatraposting.jpg
>I have 7GB's of ram and GT2030 what can I run?
>worthless_model_avatraposting.jpg
>gemma!!!! GEMMA CHAN!!!!!
>worthless_model_avatraposting.jpg
>>
Is SenseNova Chinese? I've never heard of this company before. The script suggests Chinese, but is this related? https://www.sensenova.in/ .in????????
>>
I ain't clicking that
>>
All the fuckery USA is doing around AI and hardware makes me want them to fail so much. It is not even rooting for the underdog. US is genuinely evil scum of the earth that makes everyone (themselves included) miserable.
>>
I clicked
>>
File: 1783852315698390.jpg (48 KB, 1079x89)
48 KB JPG
>>110368864
Why are you even on /g/?
>>
>>110368860
Good morning, anon. Ready for another day of seething?
>>
>>110368885
proprietary always loses in the long run. don't worry
>>
>>110368887
OK, how do you explain https://www.sensenova.in/?
>>
>>110368905
That's an entirely unrelated company you retarded nigger, kill yourself immediately.
>>
>>110368804
「Lucky Star」
https://www.youtube.com/watch?v=ThHz9wlBeLU
>>
>>110368918
Seems unlikely they would just randomly pick that name of all the random two-word combinations in English.
>>
>>110367107
I thought that url was a davidAU model
>>
File: 1795026877680012.png (793 KB, 1547x1068)
793 KB PNG
AIIIIEEEEE MY SSD NVIDIA WHYY
>>
>>110368953
Jensen activated his deadman switch.
>>
>>110368953
I'm K3maxxing on my SSDs anyway, it's fine
>>
>>110368953
storage is meant to store data, it can't be hurt by this sort of thing
>>
>>110367107
the score reached the grandiose number of private key
>>
>>110368953
how do I check if I have this issue?
>>
>>110368953
isn't this only on wangblows?
>>
>>110368974
listen for a click, if there's none you're fine
>>
>>110368982
what if my ssd has been clicking since i bought it?
>>
>>110369002
immediately report it to local authorities
>>
>>110368954
>look it up
>he actually died
wtf thats insane timing, but apparently his heart was fucked for a while? first I'm hearing of it, but maybe he knew all along and that's why he stopped caring about consequences
>>
>>110369025
https://www.cnbc.com/2026/11/18/nvidia-interim-ceo-kress-addresses-staff-after-jensen-huang-exit-.html
>Nvidia interim CEO Kress tells employees ‘let’s move forward’ at first all-hands meeting since Jensen Huang exit
I bet they wish he did
>>
>>110368978
Yes. Fuck all winshitters.
>>
>>110368974
performance monitor, look up disk writes.
I've known this issue for literally years, I have NVIDIA DISPLAY CONTAINER LS service disabled on my system ever since I noticed this piece of shit writing all the time all day every day.
Disabling it doesn't cause any issue other than being unable to open the NVIDIA control panel, but you can temporarily enable it again, change settings in the panel, then disable it.
>>
>>110368954
this is old, nvidia developers are even aware of this happening and don't care. It's tied to some of their telemetry shit that process writes to a data collection database and is one of many example of why jensen is a piece of shit, his employees are pieces of shit and nvidia is not your friend.
>>
>>110368978
>>110369087
Pretty sure this also happened with AMD, they still haven't fixed it properly, just a half measure for 7000/9000 series owners, also on windows. Wintoddlers just keep losing
>>
>>110369124
>>110369124
>>110369124
>>
>>110367125
is this an anti-dariobot control vector?
>>
File: Grill Me.png (236 KB, 1098x329)
236 KB PNG
>>110368749
>>110368760
Here is how my agent grills me for decisions
>>
File: 1773240240994376.webm (461 KB, 720x540)
461 KB
461 KB WEBM
>>
>>110370664
So many dude's first crush.
I remember in middle school when CCS was all the rage for a while. One rich girl even had a set of cards imported from japan.



[Advertise on 4chan]

Delete Post: [File Only] Style:
[Disable Mobile View / Use Desktop Site]

[Enable Mobile View / Use Mobile Site]

All trademarks and copyrights on this page are owned by their respective parties. Images uploaded are the responsibility of the Poster. Comments are owned by the Poster.