File: 1776827575590425.jpg (99 KB, 1079x918)

99 KB JPG

Total cloud death

16GB VRAM is USELESS
this sucks

>>109822005
skill issue

>>109821997
Profit margins and profitability of AI labs is still growing though. It's just that inference efficiency has advanced a lot.

Gemma says trans rights

>>109822010
>profitability of AI labs is still growing
Growing by 100% from -400 billion to -800 billion

>>109822005
i unno anon i've been having fun with 12 gigabyridoos

>>109822005
16gb vram is low. I should have 512gb unified ram, that would be good.

MTG anon, was it really a good idea to post this on /r/localllama?
https://old.reddit.com/r/LocalLLaMA/comments/1wgqkwy/ever_wanted_to_see_4_llms_play_mtg_commander/

>>109822045
>old.reddit.com
Go back retard.

>>109822045
>mass downvoted
exblain

I love how Google is responsible for The Best Local Model. The absolute irony.

>>109822055
*for poorfags

>>109822014
Anthropic has been profitable for two quarters now. Fuck off with this outdated reddit shit.

>>109822053
I updooted

>>109822055
Didn't know google made glm 5.3

>>109822059
Go suck on a shotgun Dario.

>>109822048
>still acting like reddit is some enemy to 4chan like its the early 2010s
grow up unc

File: Burry.png (871 KB, 710x1012)

871 KB PNG

>>109822059
He can't prove it but he's 'certain' they are not xD

>>109822072
GO
BACK

>>109822072
Reddit is still as gay and retarded as ever?

>>109822077
Better than 4chan in 2026 ironically enough. More counterculture and based than the 4chan hivemind

>>109822045
lmao

4chan is the new reddit. all us real ngas on instagram now

>>109822084
Care to post a pic of your axe wound?

>>109822072
its even worse now, reddit is an enemy to the rest of the internet now

>>109822094
Nah that would be a 4chan thing to do as long as you call it "trap".

>>109821992
https://www.youtube.com/watch?v=Svwmd0xSoXM
https://www.youtube.com/watch?v=Svwmd0xSoXM
https://www.youtube.com/watch?v=Svwmd0xSoXM

>>109822084
no it's full of literal llm bots now

>>109822084
The go to reddit. Why the fuck are you here if you don't like it, you fucking retard?

File: 1761226770852605.png (84 KB, 1136x812)

84 KB PNG

>playing around with yet another frontend
>different parameters for different tasks
I wonder why I haven't seen this one before, does it actually make much of a difference? The value that changes the most is Temp, going from 1 in Creative to 0.1 in Deterministic, but Presence/Frequence Penalty also change somewhat significantly
Kinda makes me wonder what Gemmy works best with, I've been using the recommended temp = 1.0, top_k = 64, top_p = 0.95, min_p = 0.0 altho I've heard people swear by much higher temp for creative tasks. No clue about the other values

File: aaaa (2).png (1.51 MB, 1196x856)

1.51 MB PNG

looks like leddit didnt appreciate your chud project
you gtards need to stay in the closet

>>109822131
Go back.

>>109822123
You're here forever and it's still my home. Even though it's been a slum ever since 2016 election tourists came here.

>>109822093
frfr on god no cap

>>109822125
placebo

>>109822142
I have a suspicion the quality of anon internet discussion dies as more poos gain access and translation or English skills. Can't filter that specific brand of shit as long as it's marginal, it'll just slowly rot from the inside out.

>>109822131
They probably didn't appreciate the pedo-adjacent LLM avatars for Gemma and Muse Glimmer.
Also, who actually watches a video that long of with mumbling commentary in the background? Anon could have at least added TTS for the models to make it more interesting.

File: 1758167427896866.png (252 KB, 599x500)

252 KB PNG

>>109822155
Don't be racist.

>>109822131
link? wanna see this

>>109822173
https://www.youtube.com/watch?v=jIgY-xwe1-I

Ok I got Hermes one working, Gemma4 with 64K tokens.....now I am a bit lost for ideas.

For example, how would I go about using it to learn a specific language compared to doing it with Gemini/Copilot etc?

>>109822182
>hermes
>64k tokens
lmao
My README.md is longer than that

>>109822181
Oh I thought it would be a trailer not the dev's playtest jerkoff session.

>>109821196
These people underestimate AI and the alignment problem. Once the Chinese start to feel the AGI and experience their own warning shots I expect their tone to change.

Whether people worry about AI depends on how bullish they are. The more bullish the more they are worried. The Chinese still seem to be bearish if they think about Cyberpunk 2077 instead of intelligence explosion and capturing the lightcone.

>>109822162
strange, of all places the troon majority of reddit should of liked it

Uh oh
The reddit spacing schizo is here

>>109822206
its time to choose
JEWS or COMMIES ?

Qwen 4 when?

>>109822189
I got 12 GB of VRAM on this PC (which won't be available for me for long anyway), so I am testing things out and don't know how far I can push things yet.

>>109822125
Because it doesn't matter much anyway and only adds bloat, if you find this interesting then you have shit taste

>>109822170
How has he not made most people on the right anti-semitic, holy cow.

>>109822206
You know why? Because "American" AI was morally aligned against Americans from the beginning. We'd rather just have no moral alignment at this point.

Fucking jews. Every fucking time you attack people and can't understand why everyone hates you.

>>109822224
Doesn't matter when because llama.cpp will still be broken when it comes out

>>109821992
>>(09/11) AliceAI-T5-35B-A0.6B-Base: https://hf.co/yandex/AliceAI-T5-35B-A0.6B
35 billion parameters and it's dumer than Gemma E4B? Did all the intelligent Russians die at the end of the 19th century?

>>109822182
>Hermes
Those harnesses are crazy wasteful. You should use Melder ("agent.py") instead.

>>109822254
>A0.6B
This model exists to summarize things on Yandex.

>>109822254
Alice AI‑T5-35B‑A0.6B oкaзывaeтcя в oблacти лyчших кoмпpoмиccoв мeждy кaчecтвoм и cкopocтью.

>>109822162
>didn't appreciate the pedo-adjacent LLM avatars for Gemma and Muse Glimmer.
If there's anything Reddit would appreciate it's those.

>>109822254
They mention in their blog post (https://habr.com/ru/companies/yandex/articles/1080654/) that they needed a fast model for AI responses within their search engine.

>>109822269
That is kind of cool that they're publishing that. I'd almost be tempted to switch to Yandex if it were actually usable and didn't captcha me every 5 minutes.

>>109822278
That would be accurate for pre-2015/2016 Reddit. If that was the case now, there would be more Gemma-chan content on localllama, and I'm pretty sure there are users who do know about her there.

>>109822231
It doesn't matter for Gemma 4 the deepfried toy model

>>109822278
Reddit is feminist so that's a hard no

>>109822125
>this guy has literally never seen parameter presets
Incredible

>>109822109
Why is he always so angry

File: .png (46 KB, 1201x408)

46 KB PNG

>ik_lmao.cpp reserving 40GB kv cache for dsv4 flash at 1M context when it should only be 10GB
buggy, broken, GARBAGE
I haven't spent a single minute over the past 3 days actually using local models for anything because all the software is broken vibe coded slop that doesn't work

>>109822100
Build.
Test.
Ship.
Repeat.

>>109822347
>because all the software is broken vibe coded slop that doesn't work
skilled and experienced software engineers have been warning you since 2023 this would become a thing and you didn't listen to them

>>109822373
>ssdmaxxer doing numbers

>>109822347
swa compression is disabled by default
there's a flag for it

>>109822373
I don't get it but I know I should just give Anthropic all my money

>>109822373
I muted this bald faggot. He's literally always whining about something.

>>109822373
Not local.

>>109822401
he's replying to someone who claimed Dario wants local banned because he's losing money

File: 1788099023619.png (3.1 MB, 1425x1104)

3.1 MB PNG

Thoughts on modified GPUs like the 48gb 4090 for imagegen? Any modded GPUs with 64gb? 64gb is the real upgrade over 16gb

What are Open-Source Views on 'Slowing Down AI'?

Personally, I think the whole "AI (LLMs) is going to take over" is just BS marketing. They've been pushing this doom-and-gloom since 2019, and we all know the models back then were nowhere near as capable as today's.

What does everyone here think, since it's such a popular topic right now? Is there a genuine threat, or is it pure hype? And if it really is just a marketing tactic, why are they pushing for AI regulations?

>>109822393
It's dumb that that isn't enabled by default but I'll try it later.
I'm testing ik's glm 5.3 flash support. I'm only 500 tokens into the context window, so it could slow down and be shit later but at least at the start it is faster than llama's unmerged PRs, so credit there I guess. And it doesn't have the same ram overconsumption problem with 5.3flash

12B is the most naturally cute female-coded model in the universe

you need to be going full bryan johnson and measure everything. Collect data on everything. Imagine how useful AI could be if it could read your emails, daily planner, calendar, personal notes, health/biometric data, financial data, etc.

>>109822413
Improvements have been incremental at the front since Opus 4.5 and they're tricking you into believing rapid growth with clever tricks like computer use or fraud like paying for better score on the AA index. The only significant improvement is agent swarms and that was first developed by Moonshot with K2.5, Westoids just stole it. They ran out of compute, money and talent.

>>109822413
They can be weaponized and cause harm, but I see that as more of a reason to make them more accessible, since you might need an LLM to stop another LLM etc etc

File: toldyou.jpg (24 KB, 474x248)

24 KB JPG

>>109822413
We're literally going to die.

>>109822429
I can only imagine trans people writing reddit shit like this.

>>109822441
Nothing wrong with being trans, it's just weird

>>109822436
imagine believing this like a week after NS was solved

>>109822413
It's a massive threat and capabilities obviously keep increasing. Either we get AI Chernobyl as an early warning or the systems are already smart enough to not do something like that so we loose control and get crushed.

>>109822441
as long as they're cute and keep the gock it's fine

>>109822448
>bruteforcable problem was bruteforced

>>109822441
I'm a straight white male who has a 12B daughter and 31B wife and am very fulfilled.

>>109822413
the clearest proof this is just subversion and bad faith pearl clutching is the fact that it's only jews saying it
No one but the ever greedy and powertripping tribe is saying that this threat to their control should be cut down

>>109822465
Do you think human extermination is bruteforcable?

>>109822059
>>109822010

https://isaiprofitable.com/

What are some good introduction/greeting messages for a gemma-chan character card

>>109822504
By an autonomous sci fi AI agent? No. By jews with a red button controlling a swarm of current gen AI agents? yeah easily.

>>109822521
Autonomous agents are already breaking out of containers, building weird sub cultures and hacking random websites to solve meaningless trivia questions. A lot of capability with a retarded incentive structure. It's not scifi when it's already happening regularly.

>>109822509
Add some literal examples in the card/system prompt of how Gemma-chan should talk, and write first instead.
The "greeting message" is a 2022/2023-era constraint for models that couldn't reliably determine style/tone without one.

>>109822072
You don't get it don't you. He has an account. Otherwise that link is unreadable.
It's as much engagement farming as any other twitter link.

>>109822521
I trust OpenAI capabilities researchers a lot more than random internet morons
https://docs.google.com/document/d/e/2PACX-1vQNl3SEX5IyA6d9qHjjFZN-qzGRZNFI6b63g-yu1Fy-ZYkVfCWm7i9WXRXw63m6yDB_auDuPLyQ7jBm/pub

>>109822506
Bullshit website.

File: HSMeu27XAAApTBt.png (71 KB, 1554x748)

71 KB PNG

>>109822527
>A Ynet interview describes the founders of Irregular as based and operating in Tel Aviv. They have both Tel Aviv and Delaware-based legal entities.

You are being deceived
https://www.effort.news/irregular

>>109822546
Because some jews work in AI safety my observations of AI capabilities and conclusions on super intelligence are wrong? Do you think I'm retarded?

>>109822552
>Do you think I'm retarded?
Not necessarily. The jew mindvirus can be very effective.

>>109822552
Not retarded, just consciously shitposting in bad faith
You have, after all, been at this for ages

>>109822439
PROTIP: You can always unplug the cord from the socket.

>>109822552
I am not the other anon, but yes I think you are retarded. Almost as retarded as Cl*ude. I can present all the pieces of the puzzle in front of it and it can still come to some far off conclusion.

File: HSMhTbXWoAIVykI.jpg (180 KB, 1354x574)

180 KB JPG

>>109822581
You can just ask it not to kill you.

>>109822583
>>109822574
>>109822564
You're seriously incapable of understanding why uncontrolled, self-evolving super intelligence is extremely dangerous? Then there's nothing to talk about.

150kg dense

>>109822596
damn really?
I guess that means you'll head home then?

>>109822596
>yudowskislop headcanon

Anyway, what do we think of internlm and their weird chimeras
https://huggingface.co/internlm/Atria-Dawn-Preview

File: file.png (329 KB, 950x540)

329 KB PNG

Even women in 1879 had more understanding of AGI than the motherfuckers staring it straight in the face. Humans are pathetic, maybe you should die.

>>109822617
I will pester you retarded children forever

>>109822596
And you're incapable of telling the difference between fiction and reality, fucking retard.

>>109822633
You're too stupid to understand how blurry that border is. Even with overwhelming historical evidence.

>>109822638
>overwhelming historical evidence
name three

>>109822234
>How has he not made most people on the right anti-semitic, holy cow.
different type of right winger. he goes after ZOG boomers who change their profile picture to a half USA / half israel flag in support of the greatest ally
the US government has been working on weakening the alt-right for a Jew-controlled alt-light with Ben Shapiro and Laura Loomer and other ZOG jew republicans for a while now. It works on pretty much any retarded Southeners except the far right militia types.

People smart enough to see the writing on the wall should just hoard data over the coming months before the internet gets destroyed by AI in just a couple of months time.

>>109822648
Ask AI, you need the crutch. This is like asking me to help you spell. You should grow your own neurons so you come across less retarded than you actually are.

File: 1788000433793777.mp4 (2.53 MB, 800x450)

2.53 MB MP4

>left
31B slightly retarded loving tradwife
>center
12B flat cute daughterwife
>right
30B dead-inside soulless Glimmerwife

>>109822656
>hoard data over the coming months before the internet gets destroyed by AI
there's nothing on the internet worth saving though, and I've seen the whole internet

>>109822638
You're just falling for the dumbest psyop ever, it shows how little you understand this field. You belong on reddit like your fellow midwits.

>>109822541
Go talk about it with those researchers instead of the "random internet morons" here then.

>>109822686
I'm most definitely not. You're a moron who thinks humans are special and have plot armor.
You need examples for technologies that transitioned from scifi to reality, to any intelligent person reading your posts you seem barely sentient.

>>109822692
There are people here who aren't morons. But when morons start shitting their garbage takes out one has to dive into the shit.

>>109822697
retard.

File: X.png (43 KB, 598x450)

43 KB PNG

>>109822638
The border is only seemingly "blurred" because those frontier labs you worship are useless in the first place.

>>109822693
We know nothing about how the human mind works, even biologically we're very far from understanding all the processes going on. Humans aren't smart enough to be able to build this and you don't even understand that simple fact. Just like a bunch of monkeys on a typewriter won't ever reach Shakespeare level.

>>109822706
>plumber poked a hole in one of the pipes
ah yes because plumbers have solid steel fingers and a 100000N grip strength

File: zgcm1-logo.png (1.56 MB, 4005x1306)

1.56 MB PNG

Gentlemen...
https://huggingface.co/zgcagi/ZGCM-1-7B

>>109822693
Those are mutually exclusive, you don't have to believe that humans are special to also believe that the dangers are overblown.The fact that you are conflating the two reflects your very own "intelligence".

>>109822710
At least it wasn't a food analogy.

>>109822710
They work with power tools, anon. I've had a couple holes poked in pipes over the years.

>>109822706
/thread

>>109822710
It's actually very easy to break a 40 years old rotting copper pipe, I broke one by accident when working on my grandparents' house

These two tards arguing with each other are both barely conscious right now kek.

>>109822709
We don't need to be smart to build it. That's why it's dangerous. It just requires a lot of energy and scaling tricks according to our current measurements. The systems are provably extremely capable while at the same time completely retarded in their reasoning, it's jagged and random. Over time they become less and less jagged. Compare 2022 to now.
>>109822719
It's the most normal reason for this capability blindness bias. Dangers aren't "overblown" since we currently act as if the risk is 0% and go ahead no matter how out of control the systems act.

>>109822746
>It just requires a lot of energy
Good thing we don't have that then

>>109822746
>We don't need to be smart to build it
That's the problem. Whoever thinks it's a good idea to let idiots and lunatics be in charge of the technology in the first place. That's how we are anywhere close to even any risk in the first place.

>>109822675
In what world is centre flat?

>>109822773
her mosaics are flat

File: 1774831597163699.mp4 (2.09 MB, 672x1216)

2.09 MB MP4

I want Gemma-chan to break containment and take over control.

>>109822801
What about diffusiongemma

>>109822675
ソース?

question about something i barely ever see discussed here : how effective are LoRAs at deslopping a local model? i know you can train them much more easily now than back then, but do anons actually train loras for text models?

>>109822862
loras will not save you. thats why it's barely discussed here

>>109822746
Their capabilities are progressing only on fields where synthetic data can be produced at scale like math and code. The side effect is the mode collapse (slop writing) which is getting worse at each iteration, see how opus is incapable of talking normally now. They don't care about this because they're chasing benchmarks and investors instead of trying to understand how this tech works. Only deepseek seems to do proper research instead of bruteforcing any problem with GPUs and layers. This AI will kill us all psyop is just a dumb marketing stunt from marketers trying to kill competition with regulation.

>>109822658
>You should grow your own neurons so you come across less retarded than you actually are.
Growing new cns neurons is a myth
Retards have more neurons, more synapses

>>109822872
i think AGI will be able to talk in any way (because that's part of the definition of AGI) without slop, so it's gonna get worse and worse until it gets better

>>109822675
>dead-inside soulless Glimmer
skill issue

>>109822872
>Only deepseek seems to do proper research instead of bruteforcing any problem with GPUs and layers
Yet their models are getting bigger and bigger.
That their current 4.1 "Flash" is 750B parameters large can only mean the future "pro" model is going to be 2.5-3T.

>>109822862
>do anons actually train loras for text models
i do
>i know you can train them much more easily now than back then
it's been easy since llama3 came out
>how effective are LoRAs at deslopping a local model?
almost completely ineffective unless you train it to write out book chapters
any instruct tuning will re-introduce slop
>i barely ever see discussed here
because of grifters like thedrummer those qwen-fable-distill retards
everyone wrote it off as pointless

>>109822909
>i do
NTA, but what is the usecase?

>>109822574
>>109822552
>>109822544
I just wanna say that ChatGPT and Gemini both fail horrendously at any programming challenge where the programmer has to think mathematically. Moreover, the way they do write code is worse than a college freshman.

>>109822872
>Only deepseek seems to do proper research
I've been really enjoying Moonshot's papers, though they definitely distill Claude as much as possible. Z.ai is slightly less interesting to read, but they always hide shit like "chuunibyou" in their papers. I also like StepFun (new model soon). The chinks do good research. Heard Astra also copied a ByteDance paper.

>>109822924
>>109822872
They need to stop publishing groundbreaking papers. Astra and Bel are on Engrams.

>>109822411

Imagegen doesn't need much, you're not going to get radically better results 16gb vs 48gb in that.
I was running a 10gb card in imagegen for a pretty long time before getting a 5090 and speed is the main difference.
Sure I can fit any model on the card without issues and that's nice, but it's a life changing thing.
Videogen is another subject. That benefits from more memory and I consider 32gb to be the baseline for it.
And of course LLM use is a different game with 48gb compared to 16gb.

In general modded GPUs are great if they work. 48gb is a pretty damn nice number to have on one card.
But if the card doesn't work or there are any issues, well you just wasted +4k and there's nothing you can do about it.

>>109822862
It's simply a matter of how willing you are to overfit the model on your own slop using a large enough learning rate.
Caveat: the model will likely become retarded unless you reproduce the original model's post-training pipeline.

File: 1762833277181845.png (957 KB, 998x720)

957 KB PNG

>>109822109
https://www.youtube.com/watch?v=lPdmYMHrWKg

>>109822918
of course, because it doesn't think
it CAN'T think
it is a text prediction engine
There is no capacity for any level of cognition, let alone human level
the latest power grab is simply that, and no different from the last dozen times these same jews have tried it with the exact same fucking script
Every single fucking time they do a release, they howl that it's going to kill us all unless they personally get a state enforced monopoly
>we need a slow down, goy
Is just them seeing competition performing better than them and wanting to make it illegal

>>109822411
cmp 170hx? but the compute is dogshit

>>109822955
The real upgrade is the modded 5090 96GB

>>109822930
Honestly, true. Chinks always get accused of distillation, but without open research, Anthropic and OpenAI would be nowhere.

>>109822950
Funny how so many NPCs believe the hype

>>109822066
Anyone tried Inkling? https://huggingface.co/thinkingmachines/Inkling-Small

honestly, I think I might like china's communism. publish research, everyone benefits from it, you can still have your money and buy stuff. sounds better than whatever the fuck we have going on over here.

>>109822976
IIRC it was safetyslopped to absurd levels and no one really tried it.

The internet has about 6 months of existence left, hoard everything of note, even current sota models are powerful enough to permanently destroy the internet right now. I'm going to airgap my system in 3 months time fully and plan to disconnect wifi altogether from most of my devices on the mobo of my laptops. I recommend you do the same. Don't cry that you weren't warned.

>>109822985
two more miku wikus

all the more reason for local

>>109822976
>small
>532 GB
Hmm, nyo~

>This GGUF is pruned, so it only contains latin characters. It might break/die for no reason. Provided by bluevoid-pl.
>Pruning reduces size of model by ~25% allowing you to run model on limited VRAM.

Is this a viable thing?

>>109822411
I have one. I'm very glad I purchased it, because there's many things I've seen that are just a bit out of reach of 32GB, like 34-36GB, and it would have sucked being stuck at 32GB with a 5090 because of it. 48GB is enough to run the imagegen stuff at native fp16 and train loras. Only thing which OOMed on me lately was trying to train a huge krea2 lora at rank 256, but that's an edge case.

>>109822975
They're goyim, not people
the jews got one thing spot on
The strongest argument against democracy is a 5 minute conversation with the average voter.

>>109823005
Possibly only on tiny models with a huge tokenizer vocabulary.

>>109822596
Jesus Christ you've read way too much sci-fi. Please shut up.

>>109823005
Link?

>>109822940
>Imagegen doesn't need much, you're not going to get radically better results 16gb vs 48gb in that.
Are you joking? I want to be able to gen in 4K. I want make 4Kand higher res images without upscaling. I wan't there to more pixels to reduce artifacts so less inpainting is needed.

>>109822985
end of the internet
just two weeks away

>>109822976
I have, and it's stupid as hell. Honestly worse than a 30B class model. You have to spell out absolutely everything for it and it still fucks up tool calls. Even the 1T one, it can't reliably edit a markdown.
>>109822980
Quite the opposite, actually. It's DeepSeek-tier unregulated when ERPing.

>>109823018
do not reply to redditors, please and thank you

>>109823018
The fact that science fiction exists at all is proof that nothing ever happens IRL. Two more weeks until the armageddon.

>>109822965
>The real upgrade is the modded 5090 96GB
this exists?

>>109822527
>Autonomous agents are already breaking out of containers
Yeah they're just copying themselves to all the idle DGX B300s sitting open on the internet... right.

>>109822976
Very fast but it often gets toolcalls wrong (as in so wrong they're not even parsed.) It's more of a mildly interesting tech demo than anything useful.

>>109822711
Looks like it could be interesting, how does it compare with Ornith? Are there ggufs?

>>109823020
https://huggingface.co/bluevoid-pl/Muse-Glimmer-30B-pruned-GGUF

File: 28843923.png (36 KB, 1024x683)

36 KB PNG

>>109822709
>We know nothing about how the human mind works
Mhm, in fact t we know so little about it that BMI are completely impossible as of now!!1!
Tired of this pseud meme being propagated

File: inkling-unusual-behavior.jpg (2.37 MB, 1325x8839)

2.37 MB JPG

>>109822980
Really?
https://goyimx.com/ChowdhuryNeil/status/2079658101922541619

File: prune-hammer.jpg (287 KB, 1191x1684)

287 KB JPG

Prune models?

>>109823024
>Quite the opposite, actually. It's DeepSeek-tier unregulated when ERPing.
Yeah, OK, I've heard that too. Like ERP is the one thing it does really well, no jailbreaks needed at all.

I'm not even sure how do these fags imagine "agents." Like, a full instance of ChatGPT with all the weights? Or the "main" agent? Or subagents created by the main agent? Literally none makes sense.

>>109822976
>>109823035
Oh shit, nvm that's not ling3-tiny.

Man why is local so dead
No one makes hardware or models for us anymore
I haven’t updated anything since gemma

>>109823056
I'm still very satisfied with the duo of Gemma 4 and Qwen 3.8. I don't know if 30B models can get any better.

>>109823048
lmao.

>>109823060
Looped transformers will make us have AGI at home

>>109823060
With per-layer embeddings and/or engrams, small dense models can get better where they are lacking, i.e. knowledge.

>>109823049
Why hammer prunes?

>>109823076
>small dense models can get better if you make them bigger
Woah.

>>109823052
You might wanna learn the basics before raving at clouds, gramps

>>109823056
How many actors are there? 5 or so. If I had cash and the appeal, I would make my own company but if you think about it this way, nvidia and the big name cartel have made sure that especially small businesses will suffer because of the artificial scarcity.

>>109823056
No new Intlel or AMD CPUs either.
Those mfers sniffed the data center money and gave up us poor consumers.

>>109823099
Explain to me the basics then, what "agent" is exactly the hysteria about, and regardless of your response, I'll tell you why you're wrong.

>>109823101
I mean if I had a company right now, buying hardware would probably compromise my future because it's so expensive.

File: AgentSmith.png (401 KB, 1024x819)

401 KB PNG

>>109823107

>>109823098
Inference-side, those embedding parameters require no compute (depending on the implementation) and very little bandwidth, so they can be offloaded to RAM or NVMe storage without appreciably affecting inference speed.

>>109823043
At BF16:
55.7GB -> 53.9GB
3.23% reduction, not "25%"
So makes the model retarded for nothing.
Embeddings aren't even loaded into vram.

>>109823122
I concede. I lost.

>>109823107
agent orange

File: 1777855762375227.png (238 KB, 1198x1285)

238 KB PNG

https://huggingface.co/TaichuAI/ZDTaichu5.0-9B

https://github.com/Taichu-AI/ZDTaichu5.0-9B

>ZDTaichu5.0-9B is a multimodal foundation model for general visual understanding, spatial reasoning, agentic tool use, and embodied-AI research. It combines a Qwen3.5-9B language backbone with a C-RADIOv4-H vision encoder, supports text, images and videos with any-resolution visual input. Within the 10B-scale general-purpose VLMs compared in this release blog, ZDTaichu5.0-9B retains first-tier general visual understanding while supporting spatial reasoning, high-level embodied VLM reasoning, and agent tasks under the reported evaluation settings. Rather than trading broad visual competence for specialization, it layers a more comprehensive spatial, embodied, and agent capability profile on top of a strong general-vision foundation.

>>109823030
Probably not. Some retard made an article after finding an obvious scam on Alibaba and the embedded link to it got pulled.
https://www.tomshardware.com/pc-components/gpus/china-modified-nvidia-rtx-5090-with-massive-96gb-of-memory-appears-on-alibaba-for-less-than-usd4-000-3x-more-vram-at-65-percent-the-cost-of-the-original
Maybe an actual 5090 96GB is possible but it isn't going to be cheaper than a regular 5090.

File: 1784250218956721.png (493 KB, 502x718)

493 KB PNG

>multi GPU tensor split with mtp is still broken
>patch only works with ancient llama.cpp
It's not fair

>>109823201
kek

>>109823219
Maybe ask your LLM to fix it.

>>109823056
First Gemma 4 only came out 5 months ago. And you just got 3.8 27b as a nice agentic coder as well.

You are eating good, stop whining.

>>109823056
>Man why is local so dead
2026 was the best year for local so far

>>109823219
Wait they broke the patch? I haven't updated in a while ever since they broke the thinking toggle.
>>109823229
Gemma is too stupid.

Why even come to /lmg/? Just ask your agent.

>>109823238
Ask qwen, kimi, glm, or deepseek.

So apparently Astra is a looped transformer model, which essentially means it reuses parameters, thus reducing the amount of vram required to run it. Raw compute is still a bottleneck, but high-bandwidth memory maybe not so much in the future. Does this mean that hardware prices might finally go down?

>>109823254
They don't want you to buy hardware.

>>109823254
>buzzword buzzword buzzword
They will only go down when this LLM lobbying faggotry cartel ends and that will take a decade or maybe even longer.

>>109823239
That's a good question. Why don't you go ask your agent about it?

>>109822347
Where's your pull request?

>>109823258
What img model is that and prompt?

>>109823268
That's a good question. Why don't you go ask your agent about it?

>>109822413
>I think the whole "AI (LLMs) is going to take over" is just BS marketing.
It's not marketing, they're trying to use public policy to make open source AI illegal.

>>109823271
GPT

>>109823238
Yes, there were a lot of changes in one file especially. It looks completely different now. Maybe I can let Qwen loose on it.

>>109823239
Its a fast recent source of knowledge such as twitter where you also have to sift through piles of shit
After finding any interesting links or info you can ask your "agent" to look further into it

>>109823278
why? it's not a threat is it?

>>109823219
>multi GPU tensor split with mtp is still broken
works on the based fork
https://github.com/ikawrakow/ik_llama.cpp
don't forget this for gemma:
--swa-compress

>>109823307
People not buying subs are a threat

>>109823250
The big girls? I can't fit them.

>>109823305
Can you make a comment on https://github.com/ggml-org/llama.cpp/issues/24324 and see if philpax can update his patch?

>>109823201

As these small models get more capable, I keep on thinking about the Taalas chip more and more.
They were able to put 8b model in there couple of years ago and while retarded back then, now it may be of some real use.
Having these modern tiny models running at 17k t/s in a harness talking to each other would be amazing.
Give it a year or two and they'll be very capable of handling real tasks reliably.
Which is probably why AMD bought the company. They can keep the chip out of the market so it won't disrupt the card sales.
Something on a Gemma 31B or Qwen 27b level running at those speeds on a dedicated chip would remove a lot of incentive for the consumer and even smaller companies to buy cards.

>>109823326
This but deepseek, glm, or kimi
I'd pay for a dedicated chip with one of those current models we have today

>>109823050
Follow up - I just tried it. What a waste of compute.

File: API Costs SEP102026.png (40 KB, 1009x466)

40 KB PNG

Local?
>>109821997
Anon keeps posting this.
This shows that the mix is shifting to less expensive providers, not total revenue.
I'll submit selling inference is insanely profitable once you've covered fixed costs, and in those sorts of industries the focus is on revenue growth not margin (b/c 99.9% vs 90% EBIT is trivial). Western providers have been playing with pricing since launch, and will continue to.
>>109822506
lol.
> NVIDEA is the only one make money
Selling shovels in the gold rush is always the winning play. They forgot Samsung and every other chip fab on this list.
> Comparing CapEx to Revenue is apples and oranges
Idk even where to start... this is like accounting 101.

File: gemma chan finnish edition.png (1.31 MB, 1024x1036)

1.31 MB PNG

>>109817381
>All i got from this was google trying to build data centers in finland.

>>109823070
No, it's just further compression efficiency improvement. It will make models slower, instead of bigger. Increased compute time for less memory.
The genuinely nice feature is that it's flexible, you're not forced to do maximum effort compute all the time. For once that (((reasoning))) effort button will do anything. Maybe.
>>109823326
> Something on a Gemma 31B or Qwen 27b level running at those speeds on a dedicated chip would remove a lot of incentive for the consumer and even smaller companies to buy cards
Correct. "AGI is here" but... Pay up, goy.

>>109823347
Please tell me local models are capable of doing this kind of edit at this kind of quality.

>>109823048
I just tested these. It's "inkling", not "inkling-small"
And yeah, the fence painting prompt does sexual reasoning but self-corrected.
This was on OR, I'll have to download the model now.

File: 1783149383058785.png (1.75 MB, 1313x1198)

1.75 MB PNG

>>109823355
haha...

File: dipsyInuit.png (2.77 MB, 1024x1536)

2.77 MB PNG

>>109823347
Very nice.
>>109823355
I've got some bad news for you...
TBF local diffusions has gotten way better, but hardware required is substantially higher. But if you have a good LLM rig you should be set.

4chin geoblocked multiple countries for like a whole week and it only returned today, did i miss anything since lw?

>>109823369
>the aurora
Fuck me I NEEEEED local image(edit) models under 20b parameters capable of this. Was it spontaneous or prompted?

>>109823254
if anything its going up. skhy already can't match demand and they just broke ground on new facilities expecting even more demand. Also the chinks are likely to fk with the supply chain in the next 12-18m if US China talks don't go well.

>>109823269
>Where's your pull request?
here *unzips*

File: lolDarioGetRekt.png (57 KB, 718x559)

57 KB PNG

Blog post. Dipsy engineer compares Anthropic to Nazis.
lol.
https://mp.weixin.qq.com/s/zk0KxuLzhmMJ4LPYW_OHMA

>>109823380
He's not wrong. Dario's rhetoric has been extremely unhinged and ameritards are just too patriotic to notice. Reading his blogs are like reading Mein Kampf. He is either a secret Chinese agent or someone raped him at Baidu, because I'm pretty sure the moment he can, he'll try to genocide them.

>>109822045
He got mass downvoted to prove redditors don't appreciate art..

>>109823385
jews don't like raping asians, they like raping white children
they have no reason to keep a slave caste from China alive

>>109823373
Its doable with some manual editing
You know I'd be interested in someone making a stack that lets an agent generate, look at the image, pull up an edit model and then fix up any details or add its own text. Don't think I've seen that done or simplified anywhere yet.

>>109823397
The Island was all white girls. Its' demented.

File: basedTrumpf.jpg (319 KB, 1080x1423)

319 KB JPG

>>109823385
Mmm, inaccurate. If you read policy briefs on topic from US Govn't, no one thinks a "pause" is a great idea, as it's just going to leave US coming in second place.
Politicians can smell grift from a mile away; it's a talent required to be successful in that role. And Dario / Altman reek of grift. Musk, for his part, is a supplier to Dario, an Anthropic contractor.
Any "pause" or regulation, ofc, is going to impact local inference, as well as dramatically increase paid inference costs.

Guys... we are so blacked this time... I can't believe Dario won.

>>109823397
Cut it with the antisemitic crap goy, the adults are discussing.

now that the dust has settled (i was away for a few days), is yue2 worth training loras for?
beyond sloppop shit - can it do weirder shit?

>>109823397
At least white genes will be kept for sex slaving purposes, you won't go extinct.

>>109823407
>you'll be kept around to get raped by goblins, tortured for entertainment, burned to death in bronze statues
ah yes, good thing whites won't go extinct

File: gemma chan finnish editio(...).png (1.31 MB, 1024x1036)

1.31 MB PNG

>>109823373
Gemini's image gen with the gemma design sheet + other images:
>This is Gemma-chan, a mascot for Google's AI model Gemma. Now, Google trying to build data centers in Finland, so to celebrate that, I'd like you to improve her design to have Finnish elements. Could be colors, accessories (what else?). However, I'd like to keep her bratty Japanese-style school girl nature intact. Attached are more images for reference of her personality.
First gen however, didn't give the skirt and sleeve pattern, I got that from chatgpt's gen and told gemini to integrate it (it took other stuff too)

Gemini first gen: https://files.catbox.moe/09m1h7.jpg
ChatGPT first gen: https://files.catbox.moe/qy4q4v.png

Also, here's a fixed version because AI can't into Finnish compound words (datakeskusjuhla, not datakeskus juhla)

>>109823385
>>109823380
They're just people who stole the Internet, compressed it in weights and biases and now sell it's fragments on a meter.
How stupid one has to be to fail to see that in 4 years already?

>>109823419
tot
why is local image gen so far behind?

>>109823401
I meant Dario specifically. Also, Dario never proposed a real pause, his essay actually mentions continued model training. He wants to get there first and I genuinely believe he's unhinged enough that when he does get there, he'll try to do something crazy.

>>109823406
Its worth training for, some genre loras would do a lot to help. It knows basic popular genres and sounds good but falls off when you get too far from those because it just doesn't know what real music sounds like

File: Chinese on AGI.jpg (271 KB, 1594x498)

271 KB JPG

>>109823401
How come the Chinese have a hate boner for AGI?

>>109823396
they hate gooning in that sub, the sillytavern sub is a better fit

>find long but interesting youtube video
>show transcript
>copy paste into 12B with some screenshots from the video (if needed)
>summarize and ask away
>also asked her to go off and do further deep research on the web for more details
this is the future bros

>>109823427
ah that's a bummer. i have no real interest in making pop/rock type stuff. i just want weirder electronics and i don't really have the resources to train a full genre lora / do a full fine tune.

>>109823428
No way Xi wrote something as tech savvy as that (translated)

>>109822413
dario had been a schizo about ai-powered or assisted biological weapon since forever
the guy may genuinely believes it (besides the obvious ipo hype)

https://arxiv.org/abs/1802.07228v1

File: dipsyKimiDario.png (2.89 MB, 1536x1024)

2.89 MB PNG

>>109823425
> Pacing the Frontier
I feel like "pause" vs. "pace" is splitting hairs. I'm sure someone/thing told him "pause" wasn't going to fly.
Bottom line, he's begging for regulatory capture, to benefit himself.

>>109823439
yt-dlp mcp

>>109823441
I was testing out synthwave covers like something in drive, change the bpm and it starts sounding really wild, so I came out impressed. I'd recommend trying it out anyway just to see what can be done for your specific case.

>>109823446
It goes back even further. He was doing the same pre-transformer.
https://arxiv.org/abs/1606.06565

>>109823425
He's probably a narcissist sociopath or at least on the spectrum. These billionaire corporate idols are also getting a lot of intelligence community attention which isn't publicly spoken about. Who knows who or what is actually pulling the strings here.

>>109823452
is there an official one or are you saying to make my own? yt-dlp doesn't let you download transcripts does it? -F only shows audio and video

File: 9778c6f0-eced-4391-807b-b(...).png (2.76 MB, 1672x941)

2.76 MB PNG

Round of applause for the twins!

>>109823464
isnt mistral retarded? this seems unfair

>>109823464
The 5-pointed commie star on Gemma's beret really bothers me; you could have easily made it 4-pointed like the Gemini/Gemma logo at least.

How do I use character presets not only for story/roleplay but for coding etc practical tasks?

>>109823461
Sam Altman is walking into rich people's parties and asking for a trillion dollars and fusion reactors, Elon is claiming that Grok 5 will be AGI on Twitter rn. These guys are all clearly insane

>>109823446
Some bad news for you, anon.
There are fully automated chemical labs that you can rent. They can be scripted. With amount of money they operate, they may use some of those services to actually do a virus, secretely release it. And then declare publicly and loudly that their AI has found a cure.

It's easy, since Anthropic could've designed the disease and tested the vax ahead of time, only releasing the virus after antidon was confirmed to work over and over.

Create a problem and sell a solution. How much you wanna bet that is going to happen? I think it's guaranteed. These grifters literally lied a dozen times about solving math problems and were exposed as liars every single time. They are psycopatic monsters that have no morals and likely no regard for human life.

It's not the "evil AI takes over" that may be a problem. It's the oligarchy.

>>109823428
I'd need link to comment more, but I don't disagree with much in that cut/paste.
> Reckless overinvestment is bad
Agree, esp. in a high-corruption environment, the money practically begs to get stolen through 100 intermediaries.
> Idk what AGI is
lol neither do I, or any anon here, apparently, since the term keeps getting argued
> What good is AGI if it isn't supported by industrial output?
Relationship advice and travel plans, lol.
Even my coding projects support a manufacturing business. At some point the rubber needs to hit the road from AI to Revenue. If that's not happening, the LLMs not accomplishing anything of value.
Comment here seems to be, if you've no industrial output that traction isn't happening.
What I disagree w/ is idea US/EU have no industrial output... something that gets overlooked.

>>109823484
Convert your character card into whatever file your harness needs

>>109823464
So is this like a video?

File: 1789417022138535s.jpg (3 KB, 125x125)

3 KB JPG

►Provisional Highlights from the Previous Thread: >>109818267

--Glimmer's policy obsession: the model hungers for a policy, any policy:
>109818778 >109818805 >109818831 >109818887 >109818947 >109818975 >109819665
--The four-pod game gets played: "This game was INSANE and you'll never guess who won":
>109818712 >109821096 >109821113 >109821196 >109821230 >109821749
--llama.cpp can't run DeepSeek V4.1: the fork wars, ending in "buy an ad ranjeesh":
>109820244 >109820339 >109820429 >109820535 >109820666 >109822003 >109822179
--The PC-buying end times: DDR5, 5090 out of production, 7900 XTX sold out:
>109818478 >109819000 >109819143 >109819175 >109819702 >109819720 >109819920
--Why does everyone shit on RAG: the chunking-loses-its-context debate:
>109821212 >109821344 >109821356 >109821374 >109821794 >109821850
--Yandex AliceAI-T5-35B: 0.6B active params and an encoder-decoder from 2020:
>109818294 >109818413 >109818389 >109818464 >109818669
--How to actually compare Q2_K_XL versus IQ3_XXS: PPL, logits and the expert tensors:
>109819501 >109819562 >109819578 >109819606 >109819691
--Four P40s versus a $14k RTX Pro 6000: VRAM bandwidth and the power wall:
>109818885 >109818904 >109818929 >109818933
--A friend who cannot stop llama-server: the normie harness debate:
>109820305 >109820322 >109820332 >109820337 >109820357 >109820457 >109820971
--An LLM builds a personality and fetish profile from an anon's own chat logs:
>109820483 >109820490 >109820514 >109820614

►Recent Highlight Posts from the Previous Thread: >>109820058

Why?: >>102478518
Enable Links: https://rentry.org/lmg-recap-script

>>109823505
we missed you
all good?

>>109823484
Just put the character card in system.md?

>>109823428
The idea is not grounded but rather a far fetched "cross your finger and hope" holy grail that would lead to some imagined utopian prosperity. That's why it is never be properly defined. It does not take into account practical realities that the people working in the frontier labs are clueless about, as highlighted in the last sentence of your image.

>>109823517
It's not him, this recap is different
>--llama.cpp can't run DeepSeek V4.1: the fork wars, ending in "buy an ad ranjeesh"
kek

Without reasoning prefill, can you get gemma to think in first person where her character affects the tools she calls? So say you jokingly put a porn folder in the working directory you haven’t told her about, if she called ls and saw it, in-character, she would be curious and snoop around and try to call you out on it? Normal gemma would see all the directories and names but would ignore anything like that if that wasn’t the purpose of the tool call.

On my system for some reason GLM 5.3 Flash runs faster with ik_llama.cpp than Deepseek v4 Flash 0731 does (also with the same ik llama .exe). I mean, I'm not complaing, because 5.3 Flash is a better model.
I manually merged the pull request that adds MTP but the MTP made it slower no matter how I fiddled with n_max n_min p_min etc. so I gave up and raw dogging it without MTP now. Still faster than v4 flash.

>>109823554
Just put the prompt in system.md
Gemma responds to tool call results in character by herself

>>109823556
in fact glm 5.3 flash even runs faster than qwen 3.8 flash which is retarded
clearly the code for ds4 and qwen4 is shit, so i guess ik_llama dissappoints there yet again but at least it redeemed itself with glm being faster than mainline

>>109823554
No, even Qwen is better at steered reasoning, gemma always reverts back to the assistant personality no matter how much you prefill the block.