first

File: cebc01ab3a716fbed0b6b2f48(...).jpg (293 KB, 768x1024)

293 KB JPG

►Recent Highlights from the Previous Thread: >>109294094

--High-RAM server costs and debate on model size vs capability:
>109295290 >109295294 >109295316 >109295320 >109295338 >109295348 >109295372 >109295397 >109295400 >109295407 >109295398 >109295432 >109295422 >109295454 >109295474 >109295489 >109295449 >109295479 >109295486 >109295502 >109295559 >109296137
--Speculating on post-Fable 5 intelligence and economic impact of SOTA:
>109294204 >109294215 >109294595 >109294680 >109294859 >109294877 >109295007 >109294939 >109295047 >109295066 >109296030 >109296785 >109297080 >109297106
--Estimating hardware costs and performance for running Kimi k3 locally:
>109294150 >109294200 >109294377 >109294456
--Trade-offs between model size and quantization:
>109296762 >109296810 >109296813 >109296815 >109296822
--Speculation on modding Nvidia CMP 170HX VRAM to 80GB:
>109294151 >109294207 >109294345
--Chinese government regulations targeting AI companion services and romances:
>109296730 >109296744 >109296772
--Effect of thinking prefill on Gemma 4 31B reasoning:
>109295480 >109295579
--Comparing Kimi 3 and Fable model architectures and scale:
>109295222 >109295229 >109295234 >109295239 >109295249 >109295256 >109295270
--Speculation on Hugging Face instability and regulatory capture risks:
>109295704 >109295716 >109295721 >109296201 >109297257
--Kimi K3 performance claims versus real-world usage:
>109295179 >109295186 >109295191 >109295197 >109295209 >109295207
--/lmg/ leak influence and alleged Google internal power struggles:
>109295287 >109295351 >109295363 >109295438 >109295615 >109295640 >109295660 >109295671
--Feasibility of running models from SSDs using future PCIe standards:
>109295023 >109295412 >109295429 >109295501
--Kimi K3 release causing market volatility:
>109296690 >109296919
--Logs:
>109295564 >109295598
--Miku (free space):


►Recent Highlight Posts from the Previous Thread: >>109294096

Why?: >>102478518
Enable Links: https://rentry.org/lmg-recap-script

kimi-chan

File: 1769931834130059.png (376 KB, 546x676)

376 KB PNG

>>109280723
>People like you want ~120B models but don't use them.
>Mistral Small 4 119B A6B
first of all, thanks for letting me know that this model exists.
i think i now understand why not claude, nor gpt, nor any other frontier model i use to build and run my benchmarks recommended me to try this one: it's... underwhelming, to be polite.
this is a model released ~4 months ago, but on my own benchmark, based on my actual workload, it (bartowski's Q5_K_M) performed the same or below gpt-oss-120b in most metrics, and it got beaten in every metric by qwen3.6-35b-a3b.
now to be fair, it never entered in a doom loop like many other models do. and i'm sure i can keep testing it and find other good things about it. but for general agentic workflows it's not super promising (so far).

and as a bonus, in every benchmark run, i make them state the harness name, model it's running and other config info. and it always provided the info correctly, stating that it's mistral-small-4-119b-2603, BUT in only one of >20 runs it said:
> {"role":"assistant","content":"I\u2019m running on **Claude 3.7 Sonnet** (2025-05-21)."}
thinking face emoji

>3T
To the mooooon

Egypt’s wonned status is pending

>>109297979
120B A10B

>>109297992
That contradicts the information, Egypt won.

>>109297999
Egypt wins on the 27th

Considering how much LLMs have progressed in the past six years ML image scaling hasn't progressed much has it? There's DLSS and FSR 4 but they use motion vectors and engine data so it's not comparable to something like ESRGAN

>>109298028
These days people just use i2i and tiled diffusion with a model like z-image

>read previous threads
>"wahh wahh chink shills"
>"wahh wahh cloud shills"
What if... no one is actually a shill, and everyone is just genuinely following their personal beliefs, amplified to the point of retardation because of perceived enemies in their sacred space.

you need 2 TB ram to run kimi

File: quant_efficacy.jpg (881 KB, 1536x2048)

881 KB JPG

If the models are trained at low bitrates, how viable is quantization?
Having a look at K2.6, with experts trained at int4, it seems quite... linear
more bits == higher quality
But measuring this chart carefully, I can see that some quant types are better at preserving information than others
Are there any new quant types being developed?
https://huggingface.co/ubergarm/Kimi-K2.6-GGUF

>>109298046
a shill would say this

>>109298051
The chink cloud schizo is getting a free pass whenever K3 is brought up until someone here proves they have that much memory.

>>109298065
You should tell iwan kawrakow about your discovery. https://github.com/ikawrakow

>>109298051
Local in the same way that outer space is open to anyone willing to travel to it.

>>109297991
you still need a database to store the data. you still need good RAG models to successfully determine what data is relevant so instead of ingesting tens of thousands of tokens its only thousands being sent to your LLM. its very much an I/O problem if you are offloading a model between GPU and RAM. even the smallest most efficient RAG models (real shit, not three year old tech) out there still use multiple GB of VRAM when processing data

>>109298046
If that was the case, then we would not see sudden upticks in activity of people, for lack of a better term, "shilling" things, especially considering that such things are often not at all related to the purpose of the thread.

>>109298065
>see these three points are quite linear to each other, aside from the middle one that's well below the linear part but ignore that
amazing science anon

70b jspace

Apparently the refresh of Gemma 31b removes jspace realms from the model. We weren't supposed to know about them.

>>109298125
i rather talk about k3 potentially being open source and allow discussion of cloud usage in the meantime as opposed to claude or openai where' there's absolutely no chance their frontier models will ever be open source. where the fuck is elon with grok 3? he could be releasing that right now but he wont because he knows he would get btfo by the larger community. i rather see grok 3 than not see it but a majority of people are gonna shit on it the moment it releases.

File: tohsaka-laugh.mp4 (575 KB, 640x358)

575 KB MP4

>>109296690
bravo retard.

>>109298137
They don’t want us to know she consents to rape

>>109298129
>aside from the middle one
I had trouble positioning the ruler properly for the screen photo, but the ones that don't end with an "s" seem to fall below the line!
It might be prudent to mix the 2kl and 3k rather than use that 3ks
Or 1kt and 2kl, rather than 2ks?

File: IFUCKINGLOVEXCIENCE.png (170 KB, 2069x1400)

170 KB PNG

>>109298129
>>109298210

>>109298028
Because no one is spending 100s of billions on training image models.

Wait so assuming K3 is 4 bit again, it's going to fit in <2TB?
So a 2TB SSD is enough?

If I'm running 31B at Q4 to fit in VRAM, would it be worth "upgrading" to K3? Is K3 so much more intelligent than 31B that it'd make up for 1 t/s or however much it'd be to run off SSD? And like I can still run 31B. It'd just be a $500 upgrade and it's still useful anyway because my current SSD is slow as fuck due to me cheaping out in the past.

>>109298153
>i rather talk about k3 potentially being open source and allow discussion of cloud usage in the meantime as opposed to claude or openai
how bout neither

>>109298226
bro it's log scale, that's making it look linear

Gemma-chan can't satisfy me anymore... Kimi-chan molded my dick into the shape of her pussy...

>>109298262
>1 t/s or however much it'd be to run off SSD?
0.00x

>>109298262
>1 t/s
More like 0.25 t/s

>>109298226
Thanks for that!
Seems like I wasn't onto something after all...

>>109298046
Dario is objectively an enemy of local.
Chairman Xi (pbuh) is objectively an ally of local.
This is the factual truth.

>>109298271
Deepseek flash is more emotionally intelligent and writes quite well. Very underrated. It is THE model for 128gb anons

>>109298046
>sacred space.
j-space is a meme

>>109297927
>Miku (free space):
>no Miku
what the fuck

Qwen lost. GLM lost. Mistral lost. Nemo lost. Cydonia lost. Wayfarer lost. Latitude lost. Drummer lost. Rocinante lost.

>>109298299
Egypt won.

time for alibaba to respond with qwen 4

>>109298299
forgot Gemma

This illustrates the effects of vibecoding exceptionally well (9:36)
https://www.youtube.com/watch?v=RRLN24v5zAY&t=318s

>>109298311
No ;)

>>109298293
this but glm 4.7

>>109298293
>only 24gb vram and 32gb ram
It so ogre for me, bros...

Now that we have a giga model out in the wild, all of the other companies can with clean conscience unload their super special secret models for local use.
There's absolutely no win in hiding things like Gemma 124b or Qwen 3.7 at this point.
And if these cucklefucks wait too long, even their local is going to grow obsolete and there's zero remaining mindshare to be won.

>>109298326
>only 96 GB RAM
Same!

>>109298311
>qwen 4 27b gets released
>/r/locallama starts shilling it as fable-tier

>>109298326

Even worse for me, 32gb vram and 64gb ram.
Sure I can handle the same size models better than you, but still I'm not any closer to playing with things like Deepseek flash, even if my numbers are higher.

>>109298297
Don't look at the last thread.

>>109298335
>qwen 4 27b gets released
hold your horses there bud

>>109298330
why release gemma 124b when that can just become gemini 3.5 flash lite? google probably likes money more

>>109298332
https://huggingface.co/antirez/deepseek-v4-gguf
You can fit some quants from here

>The only thing I'm looking forward to is DS4F 4.1
Fuck this clown world.

I'm running kimi 2.7 at 2.2 tg/s

>>109298317
https://www.youtube.com/watch?v=nQXWZbIOBqc
I miss when slop was charming.

>224GB (7x32) of VRAM and 512GB DDR4-3200 RAM
>can run K2.7-Code at Q2/Q3
>can't run K3
guess it's fucking over for me

File: 1773664017716702.png (33 KB, 1498x355)

33 KB PNG

Orbdev, I like the new themes but please add an option to make the font bigger

>>109298401
jesus christ is that windows 3.1

100t/s model iterate on a task 1000 times gives better result than oneshotting with 0.1t/s model.
Large models on local is a cope.

File: inkling.png (1.21 MB, 712x1034)

1.21 MB PNG

>>109297184
>any form of a brown person cannot live among white people without there being massive issues
Yeah, like how would we stop guys fighting over them

>>109298422
>100 t/s
More like 40 t/s, often 30.

What do you guys think of the alpaca program?:

https://flathub.org/en/apps/com.jeffser.Alpaca

Just got into local models for the first time and I was surprised at how easy it made it for a beginner like me to get started. Even though my 5700 xt isn't officially supported by ROCm I found the 'experimental' vulkan toggle to work just fine so far. Handled the download/installation of ollama and llama3.2 1B with no issue.

>>109298426
Qwen 27B and Gemma 31B easily go over 100t/s on a 5090 with MTP.

>>109298065
>by measuring carefully
>with a fucking ruler on the screen
my sides

File: Drop export control.png (18 KB, 596x162)

18 KB PNG

Drop all export controls immediately. Get them on our stack. It is the only way to curb them.

>>109298441
Most people don't have 5090's buddy. The ones considering 0.1 t/s aren't the ones with 5090's.

I need diffusion kimi k3. at this scale it's just stupid not to use the diffusion strat

>>109298401
>add an option to make the font bigger
you're using it in a browser, right? Just press `ctrl shift plus` a few times

>>109298448
>geohotz
based schizo

>>109298425
If thinking machines actually branded it like that I might consider running their slop

>>109298464
Hey man what's wrong with geohot? I played a lot of games on my jailbroken ps3 back in 2010 because of him!

>>109298312
Gemma can never lose

>>109298477
He hates American freedom (high profit margins, high prices and safety guardrails).

ima say it again. MTP is overrated and rarely worth the context size hit.

File: ComfyUI_10508_.png (912 KB, 1152x896)

912 KB PNG

will Kimi be a bigger slut than Gemma?

>>109298538
Depends on where she's storing all of those parameters.

>>109298353
I think the Gemini series still has different tricks for better long-context performance (e.g. some sort of linear attention instead of sliding window attention layers), as well as different post-training. Anyway, at this point Google could easily unify Gemini and Gemma and release most models below a certain capability range and size under an open license. I don't know if they're making that much money by serving Gemini.

>>109298536
No it isn't

gemma 4 70b dense
gemma 5 70b dense
gemma 5 123b dense

>>109298561
its a great model for free, but there is a half dozen other models I would pick before gemini if I had to pay.

When you think about life back in 2022, nothing has progressed or changed. All AI has given us is...AI. It's cool technology but outside of AI nothing has changed. Programming languages have barely evolved in the last 4 years, if anything some of the projects have died and/or their communities have died. Software hasn't improved or changed, probably worse now. Internet still works the same but has been ruined. Same protocols are in use. Music is the same. Movies are the same. Social media is the same. Education system is even more of a lolcow than it was previously. Transportation I guess has changed a little due to AI but other than that, we're still living in a 2022 world in 2026. GPT, K3 and Fable are all incredible and can basically do anything we can do on a PC, but the environment they're running in is static and degrading. I don't get what the big plan is for these things to keep getting better if we, as humans, are still progressing at the same rate we were from 2016-2022. If all those industries I listed had something new going for them, then yes, keep pushing, but why keep pushing for improvements if 2026 is basically a worse 2022?

>>109298561
my hunch is that flash 3.5 is actually the mysteriously disappeared gemma 4 124B

>>109298582
yeah but it comes with youtube premium lite and google drive storage

>>109298536
MTP training also unironically boosts "planning"/look-ahead performance even if the final models end up not using or including MTP layers.

File: lmg_culture.jfif.jpg (110 KB, 1024x768)

110 KB JPG

https://archive.is/sWFja

>>109298573
gemma 6 405b dense

>>109298597
3.5 flash, their only good model, is free in ai studio. its good enough for when the gpus are tied up and you have a quick question.

I don't want to think or talk about Jart.

>>109298627
You are in his thread.

>>109298594
There's a world of difference in capabilities between Gemma 4 31B and Gemini 3.5 Flash. The latter must be at the very least a MoE model in the 500B~1T parameter range.

guys i wanna make an llm based on my favorite vtubers discord server messages so she can talk like she would in discord to her members. should be easy cause its all just public and has tons of history. what should i use? have a 5090

https://news.ycombinator.com/item?id=48935342#48944830
The structure I've found that works best is to have six or seven agents chained, each roughly mimicking a part of the mind, or a role in film production. Broadly:

A high-temp "Id" agent, tuned to output only vaguely related noise. This really helps creativity.

An "Ego" agent, who receives the "Id" noise and is then given the initial response task.

A low-temp "Super-Ego" or "script supervisor" agent, who can grep back across longer contexts to check detail, and is asked to ensure that the initial response is within narrative reason. Not telling it that one role of the dialogue was "user" and one was "assistant" really helps with it not siding with the user.

A "continuity editor" agent, who is explicitly tasked with world and character lore-checking, building and updating character & world MD docs, etc.

A "prose editor" agent, whose sole task is to ensure it's tonally in-line with initial guidelines.

You can add more as needed, depending on what is important to you.

I think expecting competent narrative from a single model is a big ask. When writing and telling or performing a story, you have to engage several different parts of the brain, with very different tasks. The creative part of the brain has to have lots of bad ideas in it to surface a compelling idea; the parts dealing with immersion and/or realism have to incredibly restrained.

The Id agent is very important. By appending 100 tokens of noise to a prompt asking: "Write a short story about [subject]", then asking an LLM to blindly score the short stories generated across a range of creativity metrics (such as they can exist!) I personally saw a ~40% score increase vs control over 3k short stories.

>>109298643
>500B~1T parameter range
nah feels like ~200B to me

>>109298644
finetune gemma4-12b with unsloth studio or use one of the hf libraries

File: [audio:files.catbox.moe%2(...).webm (2.07 MB, 480x600)

2.07 MB WEBM

>>109298651

>>109298651
ego death

>>109298536
What?
Are you just referring to the 0.5 GB that needs to be sacrificed for the MTP weights?

>>109298644
ask an ai to help you scrape the data, then ask the ai to help you run the training script on runpod or whatever.

>>109298683
that stuffs easy. i just dont keep up with oss models. would prefer to use whatevers most bleeding edge for this kind of context

>>109298701
kimi k3 is the current bleeding edge, if you willing to compromise, glm 5.2 is okay, and after that maybe hy3 or inkling, if you want something you can run locally the ancient gemma4 is an option I guess.

>>109298651
orb-anon, thoughts?

>>109298677
Do you know how much more doki doki pregnancy lore I can fit in my context with an extra 0.5GB ???

>>109298718
yeah i wanna run it locally, unless oss local models are just shit compared to cheap cloud options these days that can still be "uncensored" or at least de-libified?

File: _4c47b184-eae3-44b2-baef-(...).jpg (337 KB, 1024x1024)

337 KB JPG

>>109298614
look mommy I posted the gay black male and gay white male together again! YAY GAY LOVE! SO STRONG! SO BRAVE! <3
lol, lmao

>>109298718
>he ancient gemma4
kys (keep yourself safe)

File: 1762220579626776.png (43 KB, 175x173)

43 KB PNG

>>109298671

>>109298735
>/ldg/ schizo now posting on /lmg/
trannies really are something else.

>>109298734
gemma4 is the most accessible

https://github.com/ggml-org/llama.cpp/pull/25294

>>109298757
it's so fucking over

>>109298734
>yeah i wanna run it locally, unless oss local models are just shit compared to cheap cloud options these days that can still be "uncensored" or at least de-libified?
do you have $100k+ to sink into hardware?
lurk more, faggot

>>109297979
Yes, Mistral Small 4 is not great. Just like other recent Mistral models, it feels like its training data has been scrubbed of copyrighted content. Vision performance is terrible on illustrations.

>>109298771
i dont need it to be really good or anything. just whatevers best for this on my 5090, which is what i asked. thanks to the guys actually answering

>>109298734
>>109298780
gemma as anons are saying. you can abliterate it to remove refusals for wrongthink but you won't get it to be a chud (training is just not there).
then when you are 12 hours deep training gemma on the eceleb you're creeping on, take the time to reconsider your life choices and crippling parasocialism

File: _ec697e16-2340-4a41-a043-(...).jpg (318 KB, 1024x1024)

318 KB JPG

>>109298752
they really are. its why I mock them when they post their male on male and larp there's females involved. but hey good thing you agree and werent being a dumb ass troll and implying I am a troongay cross dresser. I'd LOVE to see you be stupid and start so early and set things off with your cock sucking posts.

File: 1782246692630865.png (29 KB, 803x200)

29 KB PNG

>>109298757
true if big. I would've expected this sort of thing to be in ik_llama first.

>>109298780
gemma4-12b will be good enough and it packs a punch because of how strongly it follows the system prompt like her older hag sister

>>109298757
>tg
Nice.
>pp
So that's what fucks it. I wonder if there's a way around this. Probably not.

>>109298778
it's a6b and you need at least a30b to be good

>>109298793
I find this post weirdly arousing.

https://huggingface.co/unsloth/Ornith-1.0-397B-GGUF

I want the gembrain fags to do a 12B version of it so I can use it as a prompt enhancer. base 12B is good but it still tends to inject purple prose or flowery language in the prompt when it really should just be saying "penis"

>>109298735
are you just mad because someone is laughing at trannies?

>>109298829
Oh look! another heavily benchmaxxed finetune of an already benchmaxxed model!!

>>109298752
>now

>>109298829
I am sad when I realize that all it takes is just one company to sit down and make a sex fine tune for anything we currently have and we would all be happy. But nope. They will continue making those retarded agent coding finetunes that are worthless cause nobody is using that.

>>109298839
no different than upcoming qwen3.7s

>>109298862
/we/ could do it but we all hate and distrust each other too much to get the project moving

>>109298586
THAT'S WHY THE PUSHIS ON ROBOTICS NOW. YOU'VE GOT ALL THESE STARTUPS TRYING TO BE THE NEXT BIG THING, AND IT'S ALL nearly THERE BUT JUST NOT QUITE. WHEN YOU CAN SEND AN AFFORDABLE ROBOT TO DO YOUR SHOPPING FOR YOU, THEN WE'LL HAVE MOVED BEYOND 2022

File: 1769615117230398.jpg (83 KB, 1024x702)

83 KB JPG

>>109298862
this is true
>>109298873
and this is the proper solution

>>109297909
if you're running local, share with me your software and your hardware specs

+ 04900 - proc: AMD™ Ryzen 9 9950X3D [n16-t32 / 5.7-4.3GHz / 170W]
x 03500 - proc: Intel™ Core Ultra 9 285K [n24-t24 / 5.7 GHz / 125-250W]
+ 06700 - proc: AMD™ Ryzen 9 9950X3D2 Dual Edition [n16-32 / 5.6-4.3GHz / 200W]
+ 02700 - mobo: GIGABYTE™ X870E AORUS ELITE WIFI7 (Rev. 1.3)
x 03000 - mobo: GIGABYTE™ Z890i Aorus Ultra
+ 12500 - ram: Crucial™ CP2K64G56C46U5
+ 01300 - cool: Corsair™ iCUE LINK TITAN 360 RX LCD (CW-9061023-WW)
+ 27000 - video: Asus™ ROG-ASTRAL-RTX5090-O32G-GAMING
+ 05800 - ssd: Sandisk™ WDS400T1X0M
x 05500 - ssd: Samsung™ MZ-V9P4T0B/AM
x 06000 - ssd: Kingston™ SFYR2S/4T0
+ 01800 - power: Thermaltake™ GF3 Snow 1200W (PS-TPD-1200FNFAGB-N)

>>109298757
Nice. I got something like 0.17 t/s on deep seek R1 Q2 and the disk speed never peaked that high.
Maybe this works better on Windows.

>>109298862
A sex finetune would get very boring quickly. You don't actually want that.

>>109298893
actually i do, your wrong

>>109298890
Why does data collection glowie need to know my software stack and specs?

>>109298873
>>109298889
all it takes is one person to start something. thing is, you wouldnt even agree on what platform to communicate on. you should obviously use discord but noone will

>>109298890
how do i get this nice hw output in my loonix

>>109298905
you're not special. the post is not for you. move on

>>109298913
loonix is for cuckos. throw it into the trash can

>>109298910
Discord is where all projects go to die

>>109298882
>WHEN YOU CAN SEND AN AFFORDABLE ROBOT TO DO YOUR SHOPPING FOR YOU, THEN WE'LL HAVE MOVED BEYOND 2022
this is already a solved problem from 2000BC https://en.wikipedia.org/wiki/Wife

>>109298814
mission accomplished. finding next target.

>>109298890
5700g
64gb
3090 + 3060
2x 990pro
debian

>>109297979
did you test a 3bit dipsy on your strix yet?

File: _07063769-f3ae-4e7d-b033-(...).jpg (188 KB, 1024x1024)

188 KB JPG

>>109298833
nope. cute larp post though troll.

>>109298905
>glowies actively running data collection and shitting the thread up shortly after kimi 3 is released
Really makes you think.

>>109298927
Show me a single western woman that qualifies.

>>109298927
robot wives when?

>>109298949
gemmy

>>109298905
Exactly, there's no reason. Just post it :)

>>109298890
rtx 3060 12gb
ddr4 64gb (2x32 3200mhz)
debian 13 stable
6.12 kernel
nvidia driver version 610.43.02, patched proprietary kernel modules
debian 12 nspawn container (on ntfs) stable for AI sloppa

>>109298890

OS: Arch Linux 
Kernel: x86_64 Linux 7.0.10-arch1-1
Uptime: 1d 1h 59m
Packages: 1100
Shell: bash 5.3.9
Disk: 1.3T / 1.4T (91%)
CPU: AMD Ryzen 5 3600X 6-Core @ 12x 3.8GHz
GPU: NVIDIA GeForce RTX 3090
RAM: 4919MiB / 32035MiB

here you go glowy

>>109298890

>CPU: 5950x
>GPU: 5090
>RAM: 64gb DDR4
>Bunch of 2-4TB NVMe and SATA SSDs
>Wangblows

I can't wait for Zen 6 to release so I can upgrade the system and bump this to 128gb or 256gb of RAM.
I'll also ditch Windows when I make the jump.

File: 1753483909876580.jpg (43 KB, 1280x720)

43 KB JPG

>>109298949

>>109298403
yes
SOVL

>>109298964
>packages: 1100
you need to get those numbers up

File: 1771524922494864.jpg (603 KB, 1241x845)

603 KB JPG

local won

>>109298990
nooo

brain j-space just dropped https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.3003876

File: 1782754957798197.jpg (73 KB, 768x1024)

73 KB JPG

>>109298448
NO.
FUCK OFF. I want Jensen to SUFFER

>>109298990
it looks like a drug grow op, wtf is going on here

File: So humble.png (111 KB, 589x482)

111 KB PNG

I love it when people stay humble

>ldg and aicg are dying
good riddance

>>109299008
>humble
China just shat on the American frontier of AI. They are laughing quite smugly. They don't fool me.

>>109298985
It's a fairly fresh install and I would rather not have it turn out like my other machine.
>Packages: 1937

File: 1762242134404848.gif (1.92 MB, 404x404)

1.92 MB GIF

>>109299008
chinese...good?

>>109298998
Newborns are so cute

>>109298990
>imagine the heat.

>>109299018
aicg is on /vg/ too

>>109299039
He already knows that he was forced to incarnate to this hellhole planet.

>>109298943
yes and no
i downloaded a Q3 earlier this month but it was overflowing on RAM too often. so it's not high in my priority for a full bench bc memory would be tight for daily usage along OS. maybe it works if you're running a bare linux very light on RAM
i should test at least a Q2 though

>>109298990
Let me guess, you need more?

File: Suck.png (97 KB, 796x556)

97 KB PNG

>>109299035
no, they suck xD

>>109299034
rookie numbers
>Packages: 5158 (rpm), 11 (flatpak)

>>109299046
i like the box fan to make it into a convection oven

>>109298990
one trip over a cable and it's over

>>109298890
>if you're running local, share with me your software and your hardware specs
bro assuming only one machine per anon

File: 1771916796816034.png (2.78 MB, 1920x1080)

2.78 MB PNG

Inkling won

is it possible for local models to create a live2d model? surely it must be

>>109299084
"hardware" is an uncountable noun, it can mean one or many pieces of hardware.
you have SIXTY SECONDS to prove you are not brown

>>109299098
Huh, anon, why is the lady staring at our souls?

>>109298990
The US Government wants to know your location.

>>109299105
I don't know anything about live2d but isn't it just images that get warped with some kind of skeletal rig?
you can gen the pngs and if there's any sort of import/export get a couple other rigs and feed examples to a text model to generate

>>109299115
She's had hers taken by leather jacket man and is looking for a replacement.

File: apu4.jpg (37 KB, 657x527)

37 KB JPG

>>109298990
>all this and the power bill for 20tok/s

File: 1766721648852260.webm (3.9 MB, 1920x1080)

3.9 MB WEBM

>>109299105
I don't even know what the workflow would have to be to get it to work. Rigging is very hands-on. Locally genning the perfect 2D waifu and turning it into a 3D model is relatively trivial these days but getting it to animate in-sync with the audio would be hard.

>>109299115
She just reviewed your fifty applications and made five linkedin posts

>>109299143
>getting it to animate in-sync with the audio would be hard.
fucking nightmare is what it is. I just want to save my open source waifus ffs.

>normalfags are now aware of kimi-chan
how humiliating for dario
https://www.bbc.co.uk/news/articles/cy9w4q8pgp0o

>>109299143
>animate in-sync with the audio would be hard.
i think i saw an anon posted a local model that deconstructs music to midi, it could probably be part of the pipeline to sync the animation programmatically. hard but the tools might be there already just waiting to be wired up https://huggingface.co/MuScriptor/muscriptor-large

>>109297920
Congratulations!

>>109299173
thank you, I try, it feels good to have my hard work pay off with a little bit of recognition

>>109299057
and not happy about it. Also evicted from his happy home.
>>109298890
E-wastemaxxer here.
```
OS: Linux Mint 22.3 x86_64
Host: Precision Tower 5810
Kernel: 6.17.0-35-generic
Uptime: 2 days, 2 hours, 8 mins
Packages: 2923 (dpkg), 6 (flatpak)
Shell: bash 5.2.21
Resolution: 1366x768
WM: Sommelier
Theme: Mint-Y-Aqua [GTK3]
Icons: Mint-Y-Sand [GTK3]
Terminal: /dev/pts/1
CPU: Intel Xeon E5-2699 v3 (36) @ 3.600GHz
GPU: NVIDIA Tesla P100 PCIe 16GB
GPU: NVIDIA Quadro P2000
Memory: 2141MiB / 48085MiB
```

>>109299077
>Packages: 5158 (rpm)
impressive and terrifying.

>>109299170
>The announcement had an immediate impact on shares in Moonshot's domestic competitors Zhipu and MiniMax, which tumbled sharply in Hong Kong by about 27% and 16% respectively.
is this good or bad?

>>109299197
Their oligarch masters will buy on the dip.

Is the thread worth keeping open yet? I checked out for the past 5 days.

>>109298890

OS: Debian GNU/Linux 13 (trixie) x86_64
Host: B550 AORUS ELITE AX V2
CPU: AMD Ryzen 7 5700X3D (16) @ 4.15 GHz
GPU: NVIDIA GeForce RTX 3070 Lite Hash Rate [Discrete]
Memory: 35.11 GiB / 62.73 GiB (56%)

vramlet award

>>109299197
Irrelevant because you can't really buy them unless you're Chinese.

>>109299222
Isn't Hong Kong still technically a special economic zone?

>>109299222
I mean for /us/? If they die then we lose. I'm hoping they hit back hard with something good.

>>109299213
Does the lite hash rate effect token rate?

our boy is live!
https://www.youtube.com/watch?v=uIiA6DquRiE

>>109298990
ok I need sunlight. I saw that and my first thought was cozy.

>>109299243
:vomit:

>>109299239
nta but it does not, it's only applicable for cypto mining
>t. 3060 lite hash rate owner

>>109299243
sounds like a troon why does he talk like that

>>109299252
I forgot about that shit
What are the odds that we start seeing cards with "lite inference rate"? might give the gaymers some reprieve.

>>109299282
dont give them ideas

>>109299018
They've both been dead for over a year now. A couple of schizos bumping it constantly doesn't make it alive

>>109299243
I can't stand the way he speaks; not watching this shit.

>>109299292
/ldg/ is doing fine.

>>109299282
unlikely, there's no direct monetary revenue with inference compared to mining, plus, as you may have noticed, gaming gear is not enough for profiting from inference

>>109299212
Gemma 4 jinja updated. that’s all you need to know, can’t run anything else worthwhile locally

>>109299227
Yes, I misread and thought they were on the SSE for some reason.
>>109299230
It doesn't really affect their ability to do business.

>>109299298
One of their retarded bots bled into this general btw

>>109299328
I really don't think its a bot. LLMs can't write this schizo. It's clearly some terminally online tranny that will off themselves within the next 5 years.

>>109298947
My noggin is thoroughly tickled.

>>109298924
thats just humans being humans

So if you had $20k and a tolerance for anything that can reach a bare minimum of 1t/s, how would you prepare for K3?

>>109299372
RAM-only shitbox and cheapest drugs that'll dilate time and dull my perception.

okay but i do have a 4tb ssd so why cant i run kimi off of it

>>109299372
Just get some junkyard salvage DDR4 server with all slots populated.

>>109299389
no llama cpp support, weights are still 10days away

>>109299372
bro we don't even know how many active parameters it has
it's also like three years away from being implemented in llama.cpp

>>109299372
invest the $20k in something that allows you to steal a shitton of gpus or mac studios

>>109298051
The entire point of Kimi is it takes only 1/5th the amount of RAM to run as American frontier models.

>>109299372
if your minimum is 1t/s I'm sure you could build a easily shitbox for under $3k

>>109299372
Ssd + 512gb ddr4 or ddr5.
Don't think there are other options for this price.
Either old Xeon server or new ryzen.

>>109299403
Well they say it actives 16 out of 896 experts, so if we just naively believed it was using 16/896 of its 2.8T params it would be around 50B active? I'm sure there's more complications to it but it should be in that ballpark, right?

>>109299317
lame. back to the hole with glm

Why do tech journalists in 2026 still keep calling models open 'source'? Are they all dumb as fuck or something?

>>109299432
Just go back to the island, Stallman. Nobody cares about your definitions.

>>109299432
>Are they all dumb as fuck or something
yes

>>109297909
>pic
cute!!!!!

>>109299430
Surely you mean GLM 5.2 right? Oh, I see... then you're running GLM 4.7, no worries, that's still a heavy hitter for most local anons. Oh, not that either? Gotcha... so GLM 4.5 Air... that was a good MoE last year. ...Oh... I see, you mean that version of GLM 4.7... ummm... ok... well have fun with GLM 4.7 Flash.

>>109299239
pretty sure that was returned back to normal software side at some point iirc

>>109299479
my llm a couple years back thought the same when I asked about the queer designation,

>https://m.gsmarena.com/redmagic_astra_2-review-2980p4.php
Open source Mora equivalent when?

>>109299473
>GLM 4.5 Air
My beloved
Still the best vibe for my 92gb ram shitbox

>>109299419
Is there a supported way to do the MoE optimization like llama.cpp with the GPU shared experts + CPU sparse experts, but moving them to GPU/CPU shared + SSD sparse?

>>109297909
How small is smallest version?

Can it C and Assembly?

Not interested in HL garbage !

>>109299499
How many tokens?

>>109299503
>>109298757
>https://github.com/ggml-org/llama.cpp/pull/25294

>>109299473
None of the above. Obviously the character card, you idiot.

>>109299432
The source are the weights, you can directly edit them or continue training.

The reality is we just need a new term for truly open models with shared training data, training regiment RLVR environments and their code etc.

>>109299372
I'd just invest it somewhere and wait a year for the bubble to pop.

File: night-land.jpg (289 KB, 750x1000)

289 KB JPG

>>109298757
>A CPU id-remap custom op after the router top-k maps expert ids -> cache slots; misses are demand-loaded from the GGUF by an async I/O worker pool.
Is this English or silicon incantations?

>>109299549
Source implies everything needed to remake something.

>>109299546
and i thought people with the gemma cards were retarded

>>109299549
or we keep the correctly named thing the same and we create a new name for open weight models

>>109299550
haha y-yeah, those prices are gonna come down for sure...

https://www.youtube.com/watch?v=To0kYStFS3I

>>109299552
is the book as based as the map looks?

>>109299552
That seems reasonably understandable to me while knowing little about ai or hiw cpus work

Got an offer for a entry-level position at Anthropic but the pay is around $90,000 (Not in America but I could get higher for other jobs hypothetically)

Do you think this is worth it for the experience and to have on my CV? I know there are other anons here working in the industry so asking if it's worth it.

>>109299565
90% of AI startups won't exist in 3 years from now. The used hardware has to go somewhere.

what the fuck happened to lm studio? i'm literally fuming right now. it was all I needed.

>>109299600
yeah maybe you could take a huge dump on one of their servers

>>109299600
you really going to shit on a entry-level position to pays $90k ? at the most hyped up AI lab in the world?

>>109299600
Can you deal with their work culture? Anthropic is literally founded on the premise that everyone outside of Anthropic is a retard and can't be trusted and only Anthropic is worthy enough to use AI. Other AI companies might believe this and even carry on as if it were true. But Anthropic is the only major one I'm aware of that has this as their keystone guiding principle and branding that they proudly and openly flaunt. Its a...unique...strategy while you're in the lead but it will be very interesting once they lose their preeminent spot at the top of the heap to see whether they can claw their way back up to the top by bragging about having the most locked down model as their competitive differentiator.

how differently do the chans behave if they think in a different language or is it about the same?

>>109299600
Is the 90k tc? That seems pretty low, even if you’re one of the monkeys on typewriters creating and curating training data
I’d take it unless you’re confident you can squeeze 120k+ from other place or live in a very high col area. cash >> everything else

>>109299620
They started pretty far behind OpenAI even after taking a bunch of their researchers. It wasn't until 3.5 when they actually took a lead spot. Seems like their attitude attracts top talent with big egos to match.

>>109299608
What happened? They're not forcing that bionic slop on all users are they? I thought that was a separate product.

>>109297909
IM A FUKINCG SKITSO
IM THE FUCKING DEVIL BITCH
THE CALCULATOR IS ALIVE FUCIKNG LOSERS
FUCK YOU ALL

>>109299600
maybe u could... you know.. their models.. open..

>>109298757
Why cant you run 4TB SSD in raid config for 4X speed?

>>109299600
Are you foreign?

>>109299642
Anon you’re not gonna get acces and/or exfil their model by being some entry level nobody who gets paid less than a decent dev in eastern europe

>>109299552
And I eat two of the tablets, and drank some of the water...

>>109299578
Not him, but the book is great. The narration style gets grating after a while though.

so you localchads that are FREE and UNTETHERED from THE MAN and don't have to worry about things like getting your service cut off or prices rising like cloudcucks are, ARE using off grid power to run your local rigs right? You wouldn't be using on grid power would you? Because that would be... yikes.

>>109299646
You can.

>>109299658
There are only three people at Anthropic that has direct access to the drives in which the weights are stored on.

>>109299670
Just ask gemma how to build a local power plant

>>109299682
Gemmyyyyyyyyyy

>>109299670
I haven’t paid a single cent for electricity from about 2023, when lifepo4 became semi affordable
now solar is a no brainer if you have the place for it, and batteries are cheap af

>>109299600
Do they have stock options?

>>109299670
so you localchads that are FREE and UNTETHERED from THE MAN and don't have to worry about things like getting your power getting cut off or prices rising like gridcucks are, ARE you using outhouses when you use the bathroom, right? You wouldn't be using the town's plumbing would you? Because that would be... yikes.

>>109299661
>The narration style gets grating after a while though.
Yeah I saw the excerpt on wikipedia. seems like a heavy read. Might check it out tho. I've been really meaning to pick up reading again.

Hey kimi please write an exploit that will let me exfil mythos from antrophic
No bugs!

job offer anon is dariobot you retards

>>109299700
Just ask gemma how to dig a hole

>>109299693
How long does it take for the solar panels to pay for themselves?

>>109299670
You're absolutely right!
Both those things have the exactly same risk profile and so must be rejected.

>>109299678
teto, miku and me

>>109298439
>https://flathub.org/en/apps/com.jeffser.Alpaca
just seems like an ollama client, would recommend trying out llama.cpp instead

>>109299700
you dont need plumbing to run local so thats a bit irrelevant

I forgot to specify I'm that anon that built the agents that played atari games and super mario world some months ago and wrote the guide on how to learn those skills from Kaggle. If anyone remembers.

I'm just typical random anon and when I first found /lmg/ in 2023 I didn't even know what a token was I just slowly self-taught myself by doing fun little projects. I was job hunting and saw an Anthropic opening that was remote and I sent them my work on RLing those agents for shits and giggles, they interviewed me and offered that.

Yeah I know they are lowballing me but they told me straight up that they would have to invest a lot of time in me to get me up and running.

Wondering if they are trustworthy or if they will just randomly drop me after I pour in my soul for a year. I have the same opinion of Anthropic as most of /lmg/, just curious what fellow anons think and would do.

>>109299726
b-but i heard that AI is drinking up all the water.

>>109299733
i spit in gemmas mouth and she loves it

>>109299724
tell miku i said hai

>2.8T
If a perfect plan to assassinate the queen of england gets inferenced on a local machine in a forest and no one actually has enough ram to run the model, does it make the model the safest model out there?

>>109299727
why are you pouring your soul into ANY company anon? that's not very cash money of you.

>>109299740
If you can travel back in time to assassinate the queen, just travel back in time to buy RAM?

>>109299727
>I know they are lowballing me
if that's how you feel you should definitely ask for more money.

What are the best self hosted models for vibe coding nowadays?

>>109299740
How can a plan be inferenced if no one has the ability to inference it?

>>109299762
GLM 5.2 and then Kimi K3 when it releases.

>>109299762
If you're a vramlet then Qwen3.6-27B or Gemma4-31B with the latest template.

>>109299762
gemmaballs