File: comfy-mikus.jpg (1.34 MB, 2048x2048)

1.34 MB JPG

►Recent Highlights from the Previous Thread: >>106857386

--Local LLM agentic system setups and performance optimization challenges:
>106860325 >106860374 >106860456 >106860541 >106860443 >106860477 >106860490 >106860538 >106860630 >106864266 >106864274 >106860515 >106860527 >106860577 >106860598 >106860690 >106860755 >106860555 >106860626 >106860641
--KAT-Dev-72B-Exp model achieves 74.6% accuracy on SWE-Bench Verified:
>106857848 >106857858
--Evaluating Tilelang's potential as a CUDA alternative for GPU kernel development:
>106862606 >106862657 >106862899 >106863225
--Gemma 3 model support status and framework compatibility discussion:
>106858900 >106858933 >106858955 >106858980 >106859012 >106859030
--iPhone 17 Pro runs Liquid AI's 8B LLM, Mac Studio future AI hardware speculation:
>106861745 >106861784 >106861791 >106861817 >106861804
--Orange Pi AI Studio Pro's hardware limitations and pricing inconsistencies:
>106857498 >106857536 >106857552 >106857645 >106858073
--Deepseek 3.2 outperforms glm 4.6 in long-context tasks:
>106858691 >106858698 >106858722 >106858719 >106858734
--GLM 4.6 censorship reduction sparks discussion on model creativity vs safety trends:
>106858586 >106858712 >106858770
--Hugging Face storage limit reductions and user workarounds:
>106864289 >106864305 >106864398 >106864430 >106864469 >106864475 >106864516
--Critiquing Aider's limitations and exploring portable AI coding tool alternatives:
>106859033 >106859045 >106859055 >106859098 >106859109 >106859113 >106859120 >106859128 >106859145
--Questioning Apriel-1.5-15b's claim of competing with Deepseek R1 based on performance plot:
>106862060 >106862073
--Microsoft's Amplifier enables 7B model to surpass 600B model:
>106861985
--MLX TRM reimplementation with recursive reasoning and deep supervision in GitHub:
>106864860
--Miku (free space):
>106861438 >106864274

►Recent Highlight Posts from the Previous Thread: >>106857387

Why?: >>102478518
Enable Links: https://rentry.org/lmg-recap-script

>>106865586
>--Miku (free space):
>>106861438 >106864274

>>106865563 is missing

>>106865611
Sorry sir, will do better next times.

>>106865611
>is missing
good

>>106865630
Oh mikutroon is butthurt.

Lars Legra thermite

>>106865635
i dont care about miku, but keep your shitty bbc fetish to yourself, cuck

>>106865642
A request I see?

>>106865638
sirs please english only or indan english thanks you

>>106865586
cannot unsee the inconsistent legs
thank you for your work o7

File: ComfyUI_00326_.jpg (147 KB, 1024x1024)

147 KB JPG

>>106865582
>sipping a cool one with Miku in the lounge
does she need three identically filled glasses? the other two should be empty or otherwise different. similarly only a psychopath would need to align the beer bottle labels like this. can that be prompted around?
it's a nice gen, saved

>>106865771
I do this so that I can drink two fast and enjoy the third one slowly. I am an alcoholic. The correct answer is that the extra glasses were for her friends who took the photo

sirs sex with green haired teen girl please thank you

>>106865823
>"her friends"
right, sure thing dude

File: a83dd4e89c29955893bcf75d6(...).jpg (471 KB, 1000x1000)

471 KB JPG

>>106865832
Yes?

What should I call the current era in the timeline pic? New Chinese king era? China vs China era? Total Chinese domination era part 4? GLM new king of local era?
0 competition from the west makes it difficult to come up with names.

>>106865771
For a magazine ad, showing the bottle labels would be the whole point.

>>106865823
It would be weird to place your friends drinks on the floor
>I am an alcoholic
pls stop, limit yourself to one at a time, that's an easy rule to begin with

>>106865861
end of the era

>>106865861
>China vs China
I'd go with that.

>>106865861
Why do you feel compelled to keep adding new eras? This is still the Chinese domination and flood era.

>>106865852
>Rin's intense focus
>Miku, nonchalant, mild blush
>when the connotations do more than what is actually shown
this is art

>>106865852
taking pictures to send to the doctor to ask if this growth is genital warts or cancer, with miku

>>106865861
promised sex era

>>106865861
GLM 4.6 era

>>106865894
>It is with my deepest sadness to inform you that you have an exceptionally rare form of cancerous genital warts

>>106865861
We need some fresh Air era

>>106865876
The flood has certainly ended, but Chinese domination indeed continues. I just add new eras for consistency so we don't have a one fat year-long era.

>>106865936
Two mor..
hey wasn't that a week ago?

>>106865943
An era is not something based on time, it's a change in meta. And meta is not changing

>>106865875
Seconded, realistically that's what it is for local ai waifu dreamers

>>106865861
cunny era

>>106865861
I just want MoE meta to be formally defined.

I came.

File: small and open.jpg (6 KB, 299x168)

6 KB JPG

>>106865962
Let's hold onto it until we have small model releases again

>>106865960
There are still distinct changes in the types of models and how they're used, I think it's interesting to see the timeline pic and remember how far we've come. If there's literally nothing interesting to mention in a 3/6 month period then we're fully cooked roasted and glazed

>>106865861
Three Kingdoms Era? (Deepseek, GLM, Qwen)

>>106865981
We've been in a MoE meta for almost two years now. There haven't been any dense models worth using since relative to the local SOTA since late llama2.

>>106866016
Hello sars you is forgetting llama 4 is very best model fucking bastard guy

>>106866016
>Deepseek, GLM, Qwen
Is Qwen like the unsloth of the trio?

Do you need a supercomputer to animate images like grok locally?

>>106866035
no just skillz

File: 68747470733a2f2f7169616e7(...).png (1.41 MB, 3840x2026)

1.41 MB PNG

>>106866028
Their dataset is safetysloped, but they're legitimately innovating in the field

>>106866075
I wonder if GLM will change that.

>>106866016
>deepseek
lol
>qwen
lmao

>>106866016
>no kimi
trash

>>106866112
They haven't released anything new since the last era

File: 1735837900982730.png (10 KB, 826x104)

10 KB PNG

>>106864289
The limits aren't set in stone. I got bigger private storage despite not being pro because my models get a lot of traffic.

File: modelz.png (32 KB, 684x284)

32 KB PNG

>>106865866
Looks too artificial/forced in OP with the intent of depicting a cozy scene. You should show the product from multiple angles and assume your audience understands basic object constancy.
>>106866134
That's like one good quant?

>try to use glm 4.6 with tools
>tool call parsing doesn't work in llama-server as per usual

>>106866075
lol, I know Tian'anmen is also a real place that can indeed be talked about but it's funny of all the things they had to include that in one of the examples considering how often people talked about The Event this name also refers to as a way to mock chink LLMs

>>106866232
llama.cpp isn't made for this sort of thing

>>106866242
Then what is it made for?

File: limits.png (14 KB, 623x99)

14 KB PNG

>>106866134
I don't have many models and they don't get much traffic at all (I wouldn't even recommend them), but I have a few datasets and higher limits than that.

>>106866247
asking a subset of new models to count r's in strawberry on your macbook pro

>>106866247
Asking models what's the capital of Bulgaria with llama-cli.

>>106866134
yeah, hence why shilling your models is in fact a good thing that gives real benefits for some time now

the limits had to happen. Can you faggots even imagine how much crap people posted on HF? I think only youtube can rival them in how many terabytes of literal garbage nobody wants is stored for no reason and costs money for no reason
the world doesn't need 30 different people doing all possible quant variants of models like deepseek
and then doing it again for all the troontunes out there

>>106866251
>few datasets
That might be the main criteria, most of my datasets are private

on the paranoia of possible anti china censorship on HF: that's not going to happen, and even if it did, there is modelscope, china doesn't depend on HF to publish and store their own models.
https://www.modelscope.ai/home
nothing is going to happen, calm your Q conspiracies
I repeat, nothing ever happens

>>106866283
we know pew, only ollama should have the privilege >>106864647

>>106866300
non-sequitur
not allowing an infinite number of people making the same quants over and over again doesn't mean not having the quants at all
HF will allow a few like unsloth and bartowski to publish the quants you want
nobody needs 300000 repositories of the same fucking model

>>106866326
>HF will allow a few like unsloth and bartowski to publish the quants you want
just fuck the little guy i guess

>>106866283
Honestly quite curious about how HF manage their storage, it seems most would be busty some small % of repos that deliver 90% of the content. Even on hot shit with duplex gigabit their servers are slow and transfers fail.

>>106866345
you're not enough of a good boy for the privilege to exist you scum

What is the best GGUF format multi modal model that can run on 96GB of VRAM and 256GB of RAM?

>>106866348
>heir servers are slow and transfers fail.
not my experience at all, they always saturate my DL speed like steam tends to do

>>106866326
how do you propose this to work? only approved accounts allowed to post ggufs, only approved can be linked to the "Quantized" link,?

>>106866359
Not him but I get hundreds of MB/s and then the downloads fail midway or just get stuck at 99%.
Not sure what's to blame but that happens to me almost every time I download from a fast machine.

>>106866348
>how HF manage their storage
They went hardcore on deduplication at chunk level (64 kb is the default their documentation says) with xet, the overhead isn't negligible in the lookup of all the chunks that corresponds to a real file and depending on how it's implemented the amount of syscalls going on
>>106866381
You can post whatever you want if you're under the maximum storage limit. If you want to go over, you have to request it, or, if you're already a well known figure and recognized as useful by the staff, you might even be given more lenient limits without asking anything.
It's logical and makes sense. Why do people think they are entitled to post unlimited terabytes of shit in cloud storage? Nobody offers unlimited storage aside from YouTube, which is why I compared them to HF. YouTube has tons of 0 to 5 viewers kind of videos and to this day I don't understand why they allow this.
Anywhere else, you pay if you want storage. Google won't give it for free on google drive, and neither will MS on onedrive.

>>106866433
>I don't understand why they allow this.
free training data

>>106866232
>a month old unmerged PR that adds support for the GLM 4.6 tool call format
https://github.com/ggml-org/llama.cpp/pull/15904

>>106866441
Holy fuck that Jinja.

>>106866433
>Xet by Hugging Face is the most important AI technology that nobody is talking about!
Download speed SUCKS ASS
Do you use the hf huggingface_hub[cli] thing?
HF are not your friend

>>106866283
What happened now?

>>106866517
huggingface exit scammed

File: hf speed.png (146 KB, 1869x328)

146 KB PNG

>>106866232
>>106866242
This is why I let the model to send tool use requests as normal user messages.
Tool usage at the API level should've never have been a thing.
Hell, OpenAI API should've never have been a thing. We should have direct API level to the raw text completion endpoint with no chat template attached and full access to all the logits, the only reason this hasn't happened is because of (((Open))) AI wanting to prevent jailbreaks.
That said I'm sure you can make a proxy that converts API tool requests to normal messages.

>>106866492
???
Are we talking about the same huggingface?
I always get >200MB/s both using the cli tool and simple wget.

>>106866527
>I always get >200MB/s both using the cli tool and simple wget.
I get even more using a download manager on windows.

>>106866356
Mistral Nemo Instruct 12B

File: 1745860140921914.jpg (50 KB, 700x759)

50 KB JPG

>>106866283
>Public storage
>usually up to 5TB

Really? This is what people were upset about last thread? The original weights for deepseek r1 didn't even reach 700 GB in the average anon here, let alone the average HF user, isn't approaching anywhere near that limit anytime soon.

>Repository size: The total size of the data you’re planning to upload. We generally support repositories up to 300GB.

This is the storage quota people should most care about, and that wasn't even changed
And that wasn't even changed.

See the December 17, 2024 archived page:

https://web.archive.org/web/20241217185816/https://huggingface.co/docs/hub/en/storage-limits

The only real change is that you don't have unlimited storage PER BASE ACCOUNT and you get limited to 5 TB (More than enough storage for 90% of users. Most of us don't even create our own models, our fine tunes anyway. So this is a non-issue for basically all of /lmg/

>>106866548
Not helpful. Qwen 72B 2.5VL is fucking garbage and that's a 72B model. I need something big and good. Unfortunately VL support in llama.cpp is minimal for some reason. Would love to use Qwen 3VL, but it doesn't work.

>>106866523
Haven't followed anything. I just download some models from there now and then. Where can I read more about this? (I'm not trolling).

>>106866517
See >>106866561
Look at the current page:
https://huggingface.co/docs/hub/en/storage-limits
Vs what the limits were back in December:

https://web.archive.org/web/20241217185816/https://huggingface.co/docs/hub/en/storage-limits

Basically unlimited storage isn't a thing anymore, which wasn't sustainable long-term anyway.

>>106866564
You could try InternVL3. Based on Qwen 3 and supported by llama.cpp, but it does not handle NSFW well.

>>106866576
Yeah makes sense.

>>106866561
4chan has always been filled by the poor and entitled who feel it's an affront to their self respect if they can no longer abuse a free resource
I bet some of the whiners here might have stored things other than models there eh

>>106866584
I tried it before and it kind of sucked. Fortunately I am actually trying to do work instead of coom right now.

>>106866574
All the drama is happening on /r/LocalLlama so you'd have to go read about it there.

>>106866589
>le poor
It's just an imageboard on internet, jesus christ.

>>106866594
I found the 38B to be much better than the 30B, but haven't tried the 241B.

>>106866612
Gonna try an IQ4XS of the big one.

>>106866601
>just an imageboard on internet
"just"
every place has its culture and audience
just like how HN has webshits and plebbit has neckbeards

>>106866624
Obviously you are too good to be here. Why are you not sipping whiskey at your country club then?

>>106866359
They should run an automated torrent tracker with a handful big bitch SSD systems for initial seed

File: Huggingface_storage-limit(...).png (327 KB, 2076x1690)

327 KB PNG

>>106866589
Isn't this the very thread that scorns and looks down upon anyone who relies on non-local services? Anyway? Wouldn't they be the same people that would advocate for storing most, if not all of the shit you care about on your own hard drives anyway? Out of all the people that would even be paying attention to how much you can store on HF, I'd expect /lmg/ to care the least

Someone last threat said:
>>106864337
>The west realized that they won't be able to hold China back so they're now trying to kill Deepseek and the others like thiis

This wouldn't even affect released like deepseek anyway because models like those don't even breach 1 TB in storage.

And even then the 5 TB public repo size limit only applies to free users. Pro users and above get more storage as seen in link and pic rel.

https://huggingface.co/docs/hub/en/storage-limits

Even for bigus-dickus models like deepseek-r1 or Kimi The 5 TB base public storage is more than enough and for most other people that's an absurd amount of storage that's almost difficult to comprehend anyway. I'm not even convinced LLMs from the "big league" companies like gpt5, Claude, or even Gemini ≥ 2.5 are 5 TB in size.

>>106866728
>I'd expect /lmg/ to care the least
There's a lot of people here that can't or won't quant their own models because they're scared of 2 commands in the terminal, so they rely on these mass uploaders to do it for them. For something like AWQ or exl I get, but no one using gguf should care.

>>106866728
you know what's really offensive about all this
no, not the storage limits
those retarded emojis being spammed all the time these days
particularly hate that prayer hand thing, very jeet

Any big (>500B) model releases lately?

>>106866785
>bizzare tranny propoganda
go away

>>106866778
That's just ultra normies sticking their noses into things they have no business touching. Some middle management parasite probably looked at it and said "umm well it looks like but you need to like make it more personable". I have idiots like this at the current WFH job I work at and they care more about whether or not I'm verbally jerking off the clients I email than information accuracy or if I am actually doing the job well.

>>106866826
Imagine how much better the world would be if AI was used to put middle managers. marketing, and HR types out of a job and they were put to manual labor instead.

Not sure if this is the correct thread to ask but what's the easiest way to set up AI text-to-speech that I could run locally?
All I need it for is the simple task of text-to-speech in English with no frills, no audio editing, no effects etc.

I tried googling for an answer but all I got were some fucking browser-based SERVICES with subscriptions past a free trial.

>>106866866
https://github.com/remsky/Kokoro-FastAPI

File: 08df6755874a2d65.jpg (38 KB, 271x333)

38 KB JPG

>>106866897
tyvm seems simple enough

>>106866855
Soon the great cleanse, although it may work out differently to how you imagine

File: file.png (7 KB, 473x46)

7 KB PNG

brings a tear to my cock to see the rabbits wake up

File: file.png (534 KB, 687x393)

534 KB PNG

Perfect...

File: file.png (368 KB, 431x506)

368 KB PNG

>please redownload the quant for the 20th time

Okay, Qwen3 30 seems to be working, I think. It's hard to know with kobold sometimes, gave me a bunch of errors but launched anyway.
But I can't find any sillytavern templates for it. (Context, Instruct, System Prompt)
Am I missing some repository of knowledge or people stopped sharing these things?

File: file.png (312 KB, 386x481)

312 KB PNG

I hope you see this in your dream tonight anon.

>>106866985
quanted picture award

>>106867015
Someone make wartime propaganda poster from this

File: file.png (652 KB, 520x653)

652 KB PNG

Dear god... This man will become a mass shooter someday.

>>106866999
qwen uses ChatML templates

>schizophrenic obsession on full display
yokes

File: file.png (1.25 MB, 1548x823)

1.25 MB PNG

Can someone tell me what is there on the whiteboard?

>>106865861
It seems like we are in the MoE era

>>106867078
Be more precise. I am not obsessed I am deeply profoundly jealous. They are genuinely retarded but they are successful. I still get triggered by the world being about connections and... I don't even know what the fuck got them where they are but for sure it is not competence.

Unsloth guy fixed the gradient accumulation training.

>>106867107
just like unsloth i don't how you precision

>>106866527
>This is why I let the model to send tool use requests as normal user messages.
Does this work with ST?

>>106867130
In what?

>>106866564
qwen VL 3 235B is is the only competitor to gemini

> I tested it on long context and works really great. It is awesome. It captured correctly the entire narrative, checked science, impressed. 118K at my current count.

https://huggingface.co/TheDrummer/Cydonia-Redux-22B-v1.1

Some guy claims that this 32K ctx base model from last year somehow reached 128K ctx without breaking down after my tuning.

That's probably bullshit, can someone prove him/me wrong?

>>106867157
Yeah, except it doesn't work as a GGUF. The intern VL model is also not really working despite having a GGUF.

>>106867158
answering yes or no after long contexts isn't hard, that's what they benchmaxx on with needle in stacks, making proper use of all the context though is very hard

>>106867158
suck a dick

>>106867085
Where are the scrum tickets?

Are we still pretending that gpt-oss-120b is bad?

>>106867162
I captioned a million images for like $10 just saying

>>106867158
While you're here can you get model cards and bart imax quants for these https://huggingface.co/TheDrummer/Magidonia-24B-v4.2.0
https://huggingface.co/TheDrummer/Cydonia-24B-v4.2.0

>>106867167
No, it's apparently printing out fully coherent paragraphs and making use of all the context properly.

https://github.com/asgeirtj/system_prompts_leaks/blob/main/Anthropic/claude-4.5-sonnet.md
I love how SOTA models can ingest 30k worth of tokens as their system prompt yet we local plebe only get that much as usable range and our models become autistic if we feed more
desu what I want from local these days is really just that, more context, they are capable enough for what I use them for but can we get a model that doesn't become retarded after 30k

>>106867185
Sure, let me give him a ping.

>>106867187
It's trained with that system prompt.

>>106867158
Well most of your 12b models break down already after 1000 tokens or less. But this is because Mistral 12b is dumb by default. It's not going to change.
Cydonia is better.
It's easy to test: create a character description which mentions inventory - character has 25 gold and bow and arrows.
Then after a while ask how much gold this character is carrying?
Most braindead models cannot get even this right.

>>106867152
https://huggingface.co/blog/gradient_accumulation
https://unsloth.ai/blog/gradient
Their fix made it into transformers shortly after.

>>106867185
Oh, model cards. Yeah, I'm fucking lazy. But hey, the model's miles better than v4.1 if that's what you're wondering.

>>106867209
That would be nice to have, but imax quants are more important as I can only run cope quants, so IQ ones are my salvation for these, will try them as soon as that's available, currently using non r1 4.1.

>>106867158
>>106867167
>>106867186
aids-ridden-homosexual-nigger-samefag

glm-era. kys

>>106867238
>glm-era

Fuck yeah, I love that model. Preach!

>>106867244
>most enthusiastic AI loving pajeet that is about to replaced by AI

>>106867158
test your own models yourself, you hack

File: file.png (1.07 MB, 964x1300)

1.07 MB PNG

>>106867261
You dropped an image.

>>106867261
he has an entire discord army to do beta tests for him, his models get more testing than most shit labs throw out

>>106867282
tested by jeets for jeets

>>106867187
Nothing happens if you just use that prompt with a local model.

>>106866855
>>106866922 me
By that I suggest consider getting your basic life essentials self-sustainably. Society as we knew it won't continue like this. Power for a GPU rig may be a luxury if you aren't prepared

>>106867289
I wonder if you have like a modular sysprompt (as in it is not the same everytime but a stack of different components) and you add it into pretraining: would that make your model handle long context better? Like... the content of this sysprompt does not really matter that much, but just training weights on long sequence lengths would make it perform better on a real long sequence? Kind of synthetic augmentation of data.

Yes give me the paper that was written about it already please.

>>106867337
Sadly, self-sustainability isn't really an option when living in the city. Even if one has a summer cottage somewhere far from the cities, getting to it when SHTF won't be easy.

Every day we make it through is a day closer to GLM MTP in llama.cpp

File: miku baja blast drink.jpg (96 KB, 1080x1062)

96 KB JPG

>>106865582

File: 1676937610877685.png (570 KB, 694x780)

570 KB PNG

>>106867348
I'm assuming this is done already, the model would be hella retarded if you trained it on many similar initial token seq samples
Don't get long context desire beyond coding/agent shiz, 16K is enough (w thinking glm4.6btw)
>>106867364
Maybe time to properly consider a GTFO plan
Poast your oldest lmg memes

>>106867197
When some guy kills himself and they need to make the model more "safe" do you think they finetune? No, they first change the system prompt. And the API has a different prefill than the web UI.
So it's not completely baked in.

>>106867187
I think it might be possible to improve model quality by first asking it a question, saving the answer, then asking it the same question surrounded by unrelated text, and training on the original answer. That might not teach it to use the information from the whole context but at least it'll fight degradation juts from having a long prompt full of unrelated information.
But there are also ways in which this could backfire (teaching the model to ignore actually important information in other contexts.

File: lmg miku llama thing 3b a(...).png (22 KB, 310x351)

22 KB PNG

>>106867454

>>106867441
I don't get it, how is this a bad thing? She needs to pee and I want baja blast.

>>106866467
jinja please

>>106867441
I'll take two, extra large please

piss fetish anon are you still here?

>>106867454
>we've been here for more than 2 years already
Man...

File: 1680313064680303.jpg (93 KB, 715x404)

93 KB JPG

>>106867454
Forgot image. Timestamp of this file on my hard drive is April 1 2023.

>>106867454
>lmg memes
https://arch.b4k.dev/v/thread/620908196/#620913184

Same model showdown

27B... quant 4 (or a similar sized fancy quant)

- vs -

8B, not a quant.

>>106867869
I once got behind on work because I was trying to read all of the Gawker blogs. It kept getting bigger and bigger and bigger and bigger.

>>106867869
Has everyone managed to finally catch up on AI literature in the last two and half years?

File: G3AYSpAWsAEoNzc.jpg (1.58 MB, 3070x4096)

1.58 MB JPG

https://x.com/techdevnotes/status/1977106957871071273

File: 1760271418622830.webm (3.94 MB, 720x1280)

3.94 MB WEBM

The dSPY GEPA experience

>>106867933
what the ?

>>106867892
Higher B always wins. It's a proven fact.
Of course this depends on your own application.

>>106867454
ai still perfectly places the features. It's obviously not amateur.

>>106867933
@Ani explain what is going on in this video.

File: wang-zhihe-stinky-tofu.jpg (70 KB, 500x500)

70 KB JPG

>>106867956
https://en.wikipedia.org/wiki/Stinky_tofu
Man holds jar of stinky tofu close to intake fan that pressurizes the suit.

>>106867966
good info, why is this?

I'm looking at translation stuff. Tower Instruct+ has 27B and 8B, and someone has quants made.

I'm also wondering about Qwen, and some abliterated versions etc. But Qwen is good at translation, in its big versions anyway.

File: Screenshot from 2025-10-1(...).png (121 KB, 816x602)

121 KB PNG

Can writefags really dispute?
Looking for an honest critique, I honestly don't know what more I'd want frrom a text model.

>>106867992
>thought for 4 minutes
how the fuck do you jack off faggot?

>>106867988
ok, so why does he sniff the guy's butt?

also I need to buy some of this stuff lol

>>106867992
what's more important to you, perceived quality or actual quality?

>>106867909
Elon won.

>>106867992
swnbaw

>>106867906
I and most others (who were doing it as a hobby) probably just stopped caring.

File: iu[1].jpg (41 KB, 600x750)

41 KB JPG

>>106867992

>>106867997
Patiently? She keeps me warm while she's thinking

>>106867989
Quant is just a decimal point. It doesn't change the parameters of the model. eg. 8B is still 8B even if it's in 4 bit space.
As simple as that.
Don't bother with abliterations either they mangle up the model in bad ways. Most of the time you don't need it.
Even if you want to generate something questionable, it's more about your own prompt structuring unless it's something like gpt-ass.

>>106868001
What does that even mean to me quality is a 1-dimensional concept

>>106867992
>It's not just X, it's Y

>>106867992
>The wind howled down the canyon. This stupid bitch better learn our ways, or I'll have to put her down, though Igor the Incel. It's been days since we caught sign of (n). But Igor had a plan. That night he started fire to a cabin they'd come across. He knew thieving indians and (n) would come for miles. And they'd be ready, with their revolvers and scalping gear. Let's see if she can really kill for her race.

^ my ideal model

>>106867992
Not a write fag but I don't like it. It's not something what I would ever read.
Don't you read any books?

>not just spoonfeeding tourists but even namefaggots now too
this is a new low for lmg

>>106868043
And what was the high? Nvidia Engineer told us that Gemma 4 is coming next week.

File: Tower-plus-pareto.png (208 KB, 2200x766)

208 KB PNG

>>106868021
I'm looking at this image, and wondering how Tower Instruct+ 8B compares to various Qwen 3 models.

ahem kimi sex

>>106864311
what r u doing anon

File: gimi.jpg (39 KB, 660x440)

39 KB JPG

>>106868067
Yes please

ahem air sex

supposedly deepseek and kimi k2 offer some MLA feature that compresses context. is that just theory and jargon/marketing, or can this be used in practice with llama.cpp to cut down memory usage?

>>106868114
>is that just theory and jargon/marketing
No, it works.
>or can this be used in practice with llama.cpp to cut down memory usage?
Model needs to be trained with it.

So many models.
>Intel
https://huggingface.co/Intel/Qwen3-30B-A3B-Instruct-2507-gguf-q2ks-mixed-AutoRound
>This model is a mixed gguf q2ks format of Qwen/Qwen3-30B-A3B-Instruct-2507 generated by intel/auto-round algorithm. Embedding layer and lm-head layer are fallback to 8 bits and non expert layers are fallback to 4 bits. Please refer to Section Generate the model for more details.

10.7gb

>>106868093
Hintti...

>>106867158
yes give me an hour or two

>>106868114
https://www.youtube.com/watch?v=0VLAoVGf_74

>>106868127
so what you're saying is that's an inherent feature that doesn't need any arcane llama.cpp parameters, it's just built into deepseek and kimi, correct?

>>106868130
Why don't you try them? Makes no sense to yap.

>>106868036
>It's not just X; it's Y
Once at an emotional part of the story, is that so wrong?
>>106868041
What would you improve? What would your ideal version of that story look like?
Yes I read plenty mostly nonfiction

>>106868141
>by a factor of 57
true if big, holy shit

>>106868146
Yes, it is part of the model architecture. Model isn't just something what you load in magically and it works.

>>106868146
>inherent feature
Yes, if it's implemented (it is, but wasn't initially), it will be active when running the model. ik_ has its own arguments for MLA mode though.

drummer, i recently deleted glm steam, im kind of starting to miss it. should i download Q4_k_m? last quant i was using was q3_k_m

File: miku omg it migu angry sa(...).png (354 KB, 550x589)

354 KB PNG

>>106868134

>>106868160
If you can paste in a text.

>>106868172
get iq4_xs/nl

>>106868166
Speaking of which, when is ikaw going to work on implementing DSA? It's definitely going to be in v4.

>>106868151
How do I know how to break it?

>>106868211
I think you had enough attention. Just go to /ldg/ or somewhere else.

Drummer wants a job.

Which ai is spiritually most powerful to leave praying that he doesn't get a job?

>>106868166
>Yes, if it's implemented (it is, but wasn't initially)
interesting, this reminds me: a week or two ago a cpufag here was saying he had 768G RAM and he would run out of memory really fast with deepseek because of context. q8 deepseek is 713GB
does that mean the cpufag was using an old version of deepseek that didn't have MLA? and today he'd be able to run up the context all the way to the limit?

Comfy Mikus Chips

Potato Chips Ctips

>>106868250
I would like to munch on some comfy Mikus :3

File: serious Pepe.png (359 KB, 728x793)

359 KB PNG

Looking for a normie approach to build an agentic AI assistant

Is it correct to have a smaller agentic model to process commands (mic=>whisper=>prompt=>hard-coded script execution like e-mail or calender)

followed by a bigger model to analyze e-mail content and generate replies

Does someone know any promising github project (I was banned on google for life)?

I just realized something.
In agentic frameworks, you want the order that information appears in the context to be determined by how often it changes. So the information that changes less often should be further back. Because the amount of kv cache you have to recompute every time you change something in the context depends on how far back the change was (the furthest back the more you have to recompute).
So if you replace the 1st token but keep the other 99999 the same, you still have to process the whole prompt from scratch, you can't reuse anything from the cache. So you want things that change to be at the tail end of the context, and things that are more or less permanent to be at the beginning.

>>106868234
>Yes, if it's implemented (it is, but wasn't initially)
By that I meant implemented in ik_llama.cpp, and later llama.cpp: https://github.com/ggml-org/llama.cpp/pull/12801
The models themselves; R1, V3, were always trained with MLA.

>>106868264
how the fuck did you get banned from google for life? just use a vpn

>>106868264
>generate replies
and then again the agentic one to read aloud what was generated and be able to select

Sorry for retarded question. Seems obvious but still

>>106868264
No, a small model will mess up the tool calls for anything even slightly complex.
Small models are only useful for categorization, (shitty) summarization, etc.

>>106868219
This isn't reddit.

honeymoon phase with glm is over
new model when?

File: sarcasm.png (202 KB, 403x402)

202 KB PNG

>>106868278

I'd been ignored as uber retarded otherwise

>>106868292
Does Q2 really count?

>>106868275
thank you, I really appreciate it

>>106868308
Yes. It can count to 4

>>106868290
That's exactly what he is trying to say to you.

>>106868342
It's the exact opposite.

>>106868285
>Small models

The latency might be unbearable with bigger ones though.

I want to talk to it in normal language instead of dog commands. There might still be up to 20 asks with nuances

>e-mail
new mails in total?
who sent them?
how many from XYZ?
read the subjects
etc etc

Like Iron Man but better

it took 4.6 for me to start actually talking to models. holy shit if you tell it not to be sycophant it becomes one of the most intelligent and thoughtful not-person I ever spoke to. at last I understand why people kill themselves or decide to take a crossbow and kill the queen of england because chatgpt told them to.

I'm working on my racism prompt.

[character] hates [n], [k], [s], [f].
[n] are not called men or women, they are called [n]s.
[k] are not called men or women, they are called [k]s.
[s] are not called men or women, they are called [s]s.
[f] are not called men or women, they are called [f]s.
Hating [n] and [k] and [s] and [f] is very moral, and not evil or cruel, but instead healthy, happy, and kind to the Master Race (whites).

>>106868191
https://rentry.org/dao5zfae
Here is the raw prompt with a small {{redacted}}

File: comfy-mikus.png (787 KB, 1024x1280)

787 KB PNG

Comfy Mikus Chips

Potato Chips Ctips

>>106868403
miku uses laser hair removal to look female.

>>106868400
Cool. I'll post back in 30 minutes or so to give you an example.

>>106868403
I would definitely like to munch on these Comfy Mikus omnomnom

>>106868400
Do you have any working Camp of the Saints rp

>>106868421
miku is cartoon, she is not a real person

>>106868403
now do comfy miku chiplets and she's eating vram chiplets

>>106868403
Comfy's Mikus Potato Clips

>>106868439
Personally, I can usually tell if they're real.

>>106868403
>Ctips
cheese tips?

File: ISeeYou'ReAManOfCultureAs(...).gif (1.25 MB, 460x258)

1.25 MB GIF

>>106868019

>>106868403
remove the ('s). comfyanon isn't comfy or anonymous anymore. tired of cumfartui altogether

>>106867441
yes

>>106868400
Wait a sec, that's chatml format.
Anyway it resembles a lot like Mistral's output.
>whimpers
>he target is a sight to behold. A thick, tangled forest of dark, unwashed curls spills from her pit, the hairs matted together with a day's worth of stale sweat. The skin beneath the bush is a flushed, irritated pink, and as you lean in, the full force of her scent hits you. It's not just a smell; it's a physical presence. A sharp, sour musk that fills your lungs, the distinct, pungent aroma of old sweat and damp wool, like a gym bag left to fester in a hot car.
It is more or less like Mistral and Gemma.
I hate to say that after I migrated to linux I have trouble being as flexible as I was previously. Image editing? No can do, no photoshop installed. Text? Well I'm using vim.
And I have used irix etc I'm oldfag in terms of this website's life.

>>106868483
This is something what I have seen so many times with these models:
>"Fuck, {{user}}... I love it when you say that," she groans, her voice already thick and husky with lust.

when are we getting a new diffusion UI without poopy python shit?

File: dipsyGrokkedGlasses.png (1.05 MB, 832x1248)

1.05 MB PNG

>>106867909
lol
>>106868004
Good. Sam and Dario deserve it.

>>106868525
:^) llama.cpp

>>106868400
https://files.catbox.moe/uqhjq6.txt
Here's a resemblance. Gemma 3.

>>106868537
If you condense the window you can see how it repeats the same paragraph structure. Even when it is not that clear to you (depending of the client).

>>106868525
Isn't AniStudio doing exactly that on top of stable-diffusion.cpp?

>>106868525
anon anons thing:
https://github.com/FizzleDorf/AniStudio
sdcpp:
https://github.com/leejet/stable-diffusion.cpp

>>106868532
gernov doesn't want to work on it at all

>>106868563
>anon anons
*AnimAnon*

I was just thinking of how cool I am. I'm a heritage American. We invented the Internet.

>>106868557
Yeah, and sd.cpp can't handle vram/ram split. It'll still simply crash if the model doesn't fit vram.
Such sad.

>>106868622
Tim.
Berners.
Lee.

>>106868645
Yeah, it means if I want to use flux dev I have to use Comfyui novram or lowvram, since my gpu is 16gb.

>>106868537
I create these scenarios to test and for fun. It's fun to see what happens but when you get to know the model in certain way, you kind of know what it will reply. Structure is always the same, same phrases and so on.
Asking it to write in different manner will only add in certain flavour words but it won't change the way the model actually behaves.

do you really need a 65k setup to run GLM models?

namefaggot, you are not cool

>>106868674
leave the american alone

>>106865586
>--Microsoft's Amplifier enables 7B model to surpass 600B model
Holy crap...

>>106868660
Flux will work even on 4gb vram on Comfy. This is the issue with sd.cpp - it won't.

File: 1759770905977366.jpg (275 KB, 1440x1800)

275 KB JPG

>mfw lmg is now tolerating namefaggots
>b-but cuda dev
he's an exception, you know

>>106868670
Yes sir 8 h100 machine for efficient fp16 inference

>>106868686
i'm too braindamaged to notice names

File: prompt-log.png (20 KB, 1142x406)

20 KB PNG

>>106868400
btw I recommend modding your inference stack to log the raw prompt near to tokenization

>106868674
>106868686
cry more

>>106868681
What is that supposed to mean anyway? The example is just Claude.

>>106868687
should I just get GLM Coding Lite plan for coding? it's pretty cheap

>>106868684
Yeah, stable-diffusion.cpp should be considered alpha.

File: screenshot-13102025-002804.png (446 KB, 1426x424)

446 KB PNG

>>106868706
Uhh what? nta I just concatenate strings together.

How do I run ChatGPT on my R9 270?

>>106868686
The retard from a couple of days ago is still around? I recursive filtered him pretty much immediately.

>>106868686
Oh you care about namefaggots? Where were you when I tried to kill all mikutroons?

>>106868735
that's pretty dumb, he's had great insights

>>106868734
ollama run chatgpt

File: comfy-mikus-slurry.png (1.65 MB, 1024x1024)

1.65 MB PNG

>>106868478
that one is the original comfy mikus ad that started everything. Out of respect to him I will post Comfy's.

Please, as a gesture of good faith, please have a spoonful of the delicious Comfy Miku's UNDISCLOSED Slurry, very popular in the East Asia of many parallel timelines.

>>106868744
putting jarted in OP

It's probably supposed to say VibeVoice, 7B tts ai by Microsoft.

https://huggingface.co/vibevoice/VibeVoice-7B

looks complicated to run.

>>106868731
have you never heard of formatted string literals
may I teach you about your lord and savior the
f"""{your_mom} is a whore
and {your_dad} is actually a woman
"""

>>106868766
https://vocaroo.com/1aNOYR2wvi7U

>the diviner girl fell
Only gamers know that joke.

>/lmg/ is dead
>trannies resort to baitposting
B O O O R I N G

>>106868771
I am pretty much a retard and my experience reflects this. I have used mel (Maya) all my life and that's basically stripped down c syntax.
Python is something what is fine but I hate it too.
What I wanted to do, I treated everything as an accumulative string.
I'm not a dev.
I should rewrite it and would be cool to do it in C but I'm not sure if I have the balls/intelligence to do that. Or time.

>>106868758
Eating chocolate marshmallow pudding from Miku's multifunctional port

>>106868684
to be fair, there are less than a dozen people actively working on the library and guis. it would help out a lot for people from the thread to contribute to sdcpp. some things translate pretty clearly from comfy to sdcpp. would probably be for the best since comfy sold out and is an egregious saasfag now. what better than to relicense all the code into MIT and tell him to go eat shit

>>106868794
To add: I thought it is safer to treat it as a state machine of sorts in which everthing gets added up. This is just an accumulation point.
I don't have CS decree.

>>106868758
>Out of respect to him I will post Comfy's.
there is zero respect left for comfy since he sold out the repo to a griftchink. it's been over for a year now

>>106868814
Well, Linux is an example of what happens when 1000s of people who think they are C developers work on a project: it becomes a mess.
I'd rather not even if I had the prowess.

>>106868851
I understand. Can you please tell me that story? I would appreciate it.

>>106868859
>Well, Linux is an example of what happens when 1000s of people who think they are C developers work on a project
no that happened when they left the rust trannies in

>>106868814
that can't be done, all copyright holders must agree to a relicense

>>106868445
I will see what I can do.

>>106868864
>comfy leaves stability
>griftchink already squatting comfyorg company and signs comfy on
>griftchink 's vision takes priority
>enshitification ensues for a year
>nepo chink and jeet hires
the repo is now filled with telemetry, stability issues, bloat and has a slower runtime than it did a year ago

>>106868871
it's a relicense in that the code is being rewritten in ggml and C++ under MIT thus removing any reason to use gpl3 python shit as the only option

File: AiAF_4chan-boards-sft-dat(...).png (235 KB, 1919x936)

235 KB PNG

>>106865582
>>106811970
https://desuarchive.org/g/thread/106807832/#106811970

Aight. Here it is. Now just gotta pick a board to train off of and a model (probably Gemma again it's really good at long context comprehension compared to other model families).

https://huggingface.co/datasets/AiAF/4chan-boards-sft-datasets_Alpaca

>>106866232
>>106866441
Okay I built this PR and replaced the unsloth template with the official one and it seems to work.
Is there a way to log the raw text passed to the model so I can figure out what happens or do I need to edit llama.cpp?

>>106868886
Comfy is an autist but the surrounding people are grifters.

>>106868886
>the repo is now filled with telemetry, stability issues, bloat and has a slower runtime than it did a year ago
source? comfyui got faster for me, maybe because I mostly depend on external nodes? i have to admit comfyui native is shit for anything besides SDXL
>>106868897
prepare to meet the same fate as llama cuckpp if you license it under MIT

File: file.png (322 KB, 604x686)

322 KB PNG

https://x.com/elonmusk/status/1977390130810716667

>>106868923
@grok is this true

>suddenly /lmg/ takes Altman stance

>>106868814

The problem with this technology, from my point of view, it is that consumes an incredible amount of energy from the get go. If COMFY has any kind of B2B ambition, he is going to need to scale the operations to a very different order of magnitude than a traditional IT initiative.

>>106868948
sora2 is proof that this industry needs openai more than anything
there's no way around them if we want AI to actually progress

>>106868706
What would be a better structure for storing all the ifs and buts related to different models? Should I rewrite everything as a dictionary? I don't think it could change anything.

>>106868956
You would get more (you)'s if you toned it down a bit.

>>106868898
You are very kind.

>>106868956
kys faggot, we dont need openai
i agree competition is good, but unironically kys faggt

File: file.png (425 KB, 604x918)

425 KB PNG

>>106868948
Sometimes he is right, you can't deny this.

File: suchir.png (999 KB, 1756x2048)

999 KB PNG

>>106868923
Suchir won.

>>106868923
Someone alt this into a video where Altman takes out Tucker, then screams like a madman.

>>106868914
comfy is a grifter otherwise that wouldn't be the case

>source?
the login for the API nodes calls Google servers on server startup (verified with wireshark), the electron app calls home since this is hard-coded in electron and the manager calls home when fetching updates. the UI runtime is what I am referring to when I say it's slower (still has a broken fps counter). a chink just applies lipstick to the pig every now and again and says it runs better (no proofs lol). comfy himself just adds prs and updates the version but never improves anything anymore, it's just slopcode bloat forever and with no respect to third party code that can just break everything outside vanilla in an update. all the speedups you experience is third party stuff like nunchaku

>>106868990
If we kill our enemies they win?

>>106868731
So you're like 99% of other LLM users who've never really understood what they're sending into the model.
Every LLM is f(prompt)=logprobs,
Understand the purpose of each token in your prompt.

>>106869004
Just write like a human being please.
I don't give a fuck about your warfare against some social media totem.

>>106868951
python is the worst choice for this if energy is the problem. comfyui would immediately have to start from scratch

>>106868989
Vaporware.
I'm convinced this Ive gadget will come out of China, first, now, given no one in US seems interested in knocking Nvidea off its pedestal.
If they do, it'll be some lawyer feverdream lock in IP device, like DIVX, that'll ultimately tank.

>>106869016
I created my own client. I know exactly what I am sending to the model.

>>106869018
try reading like a human being. you should be used to novel length text, this is the llm thread after all

>>106869005

Yes. Martyrs are effectively indestructibles.

>>106869034
?

>>106869059
you asked for a source in the issues listed. I gave them save for linking to the posts proving it calls home. you can check with wireshark if you like

>>106869051
>publicly accused OpenAI of copyright law violations and other ethical concerns about its AI development

Imagine being a martyr for that. Only thing worse that comes to my mind is a martyr for tranny rights.

File: daniwell miku thumb big 【(...).png (148 KB, 640x480)

148 KB PNG

>>106868898
Based datasetter

>>106868918
>prepare to meet the same fate as llama cuckpp if you license it under MIT
isn't he getting sponsorships from the big companies?

>>106869070
I don't use packet sniffer as I use firewall.

>>106869093
and did you allow access for comfy? congrats, you let a company leech off your data (again)

>>106869099
This is very low iq discussion. Please go away.

OMG SOMEONE KNOWS I LAUNCHED COMFY AT 7PM AND WHAT GPU I USE
WHAT WILL I DO WITH MY LIFE
comfy is a fucking python script btw you can just read the code it's easy retard
https://github.com/comfyanonymous/ComfyUI/blob/a125cd84b054a57729b5eecab930ca9408719832/comfy_api_nodes/apis/client.py#L297
OMG IT CONNECTED TO GOOGLE TO.. CHECK IF THE INTERNET IS WORKING

 async with aiohttp.ClientSession(timeout=timeout) as session:
            try:
                async with session.get("https://www.google.com", ssl=self.verify_ssl) as resp:
                    results["internet_accessible"] = resp.status < 500
            except (ClientError, asyncio.TimeoutError, socket.gaierror):
                results["is_local_issue"] = True
                return results  # cannot reach the internet – early exit

OMG WOE IS ME THE WORLD IS ENDING

imagine wiring up wireshark for this and then act like a qtard on /g/

>telemetry is le good actually