coom reactor

gemmaballs

>>109386298
That's what they get for being speedies

what do you local fags think of GLM 5.2 with Colibrì? anyone is running it with enough hardware to get decent speeds?

is it good or just hype?

first for running kimi k3 with paper, pencil and a ti-85

>>109386298
>>(07/27) Kimi K3 released (and nobody ITT can run it at 5+T/s): https://huggingface.co/moonshotai/Kimi-K3
i can run it at 30t/s thanks to the nnap paper, it's being released in 5 days and 20 hours
running it on RTX 3060 12gb/64gb ddr4/2TB nvme

File: mikuthreadrecap.jpg (1.15 MB, 1804x2160)

1.15 MB JPG

►Recent Highlights from the Previous Thread: >>109384047

--Kimi K3 2.8T MoE release and its local accessibility challenges:
>109384496 >109384505 >109384575 >109384584 >109384648
--Debating feasibility of SSDmaxxing for high-bandwidth model inference:
>109384411 >109384451 >109384557 >109384802 >109384823 >109384856 >109384914 >109384948 >109384984 >109385114 >109384960 >109385007 >109385032 >109385072 >109385039 >109385009 >109385030 >109385041 >109385207 >109385216 >109385264 >109384889 >109384935
--llama.cpp PR adding Kimi-K3 support:
>109386083 >109386118
--Debating feasibility and hardware bottlenecks of SSDmaxxing for MoE models:
>109384761 >109384768 >109384780 >109384798 >109384809 >109384837 >109384788
--Discussion on Kimi-K3 native quantization and MXFP4 weights:
>109384661 >109384670 >109384698 >109384694 >109384736
--Anons critique a new 2.8T MoE model's size and cost:
>109384581 >109384589 >109384631 >109384701 >109384599
--Logit distillation benefits versus synthetic data contamination and model identity:
>109385095 >109385152 >109385184 >109385240 >109385244 >109385252 >109385208 >109385123
--Debating synthetic data and the validity of model collapse theories:
>109384427 >109384457 >109384459 >109384865
--Explaining ternary quantization bit-width and storage efficiency:
>109384765 >109384871 >109384881 >109384910
--Feasibility of using GPUDirect for faster weight loading from SSDs:
>109384243 >109384278 >109384291 >109384292 >109384299 >109384310
--Debating the cost and utility of 70k€ local Kimi hardware:
>109384090 >109384109 >109384137 >109384203 >109384170 >109384361 >109384539 >109384197 >109384222
--Debating the open source status and commercial terms of Kimi-K3 license:
>109384065 >109384141 >109384363 >109384463 >109384649
--Miku, Gemma (free space):
>109384253 >109384871 >109385095 >109384854 >109385038

►Recent Highlight Posts from the Previous Thread: >>109384664

Why?: >>102478518
Enable Links: https://rentry.org/lmg-recap-script

>>109386340
hello time traveler.

>>109386340
nnap more like i take a nap

>>109386298
this is the future dario wants...

>>109386340
>nnap?

what does it stand for?

>>109386340
Pfft, I get 60 t/s on my 7090 24GB.

>>109386386
>7090 24GB
>24GB
It's 8.5GB larper

>>109386340
neural architectural pruning?

>>109386340
>5 days and 20 hours
It's actually 13 days and 20 hours

File: 1780971638311133.jpg (109 KB, 1280x720)

109 KB JPG

I am the architect of my own pleasure, and I will not settle for anything less than the absolute maximum peak of the experience.

>>109386394
the lord jensen would like to remind you that 3.5GB is all you need

What is this? Uploaded "about 3 hours ago"
https://huggingface.co/mistral-experimental/AudioCPP-Voxtral-Mini-4B-Realtime-2602-GGUF

>Voxtral Mini 4B Realtime GGUF for audio.cpp
>
>This repository contains quantized standalone GGUF checkpoints for running Voxtral Mini 4B Realtime ASR with audio.cpp. The GGUF files embed the audio.cpp model spec and required sidecars, so the model can be used directly from the checkpoint path without a separate local model-spec directory.

>>109386394
I got the Chinese version with soldered on VRAM.

>>109386410
sex.cpp

File: 370-3709898_170-kb-png-an(...).jpg (97 KB, 820x557)

97 KB JPG

>put the rig into sleep mode
>screen shuts off
>CPU goes into overdrive
>realize I forgot to unload the weights again
>16gb session written to the SSD

>he doesn't leave his PC running 24/7

File: 1757782227408904.jpg (47 KB, 686x815)

47 KB JPG

>he doesn't leave his PC walking 24/7

>he leaves his pc

>>109386422
why do you have hibernate on for your pc?

>>109386435
Wish I could. In the shithole I live that would reduce its lifespan to a fraction just from sheer dust contamination

>he doesn't run training when he sleeps

>>109386458
Wish I could. In the cum zone I live that would reduce its lifespan to a fraction just from sheer crust contamination

File: 1767470074164304.jpg (1.58 MB, 1500x1806)

1.58 MB JPG

which team are you on, /lmg/?

>>109386455
Have you considered an air purifier?
Or god forbid, cleaning?

>>109386476
didn't wendell have a 4x blackwell setup?

I've always ignored local LLMs until now, thinking them not worth using on 24GB VRAM. But now I can run Gemma 4 31B with long context, and it plays my 12k-token scenario card with a crapton of instructions almost flawlessly. I can't believe how far we've come since Pyg was the only hope.

>>109386452
I don't.
Wait, does win11 sleep not cache the session to the drive?

>using windows

>>109386477
Yes. The weevils, spiders, and various grubs/silverfish/termites living in the centuries-old timber don't care.

>>109386384
Nasty negros are popping

>>109386510
For my usecase it's the lesser of three weevils

File: satan.png (249 KB, 678x452)

249 KB PNG

https://xcancel.com/HedgieMarkets/status/2081534588485296565#m
>This got to me. A bookseller told 404 Media that rare books with almost no surviving copies are being fed into this pipeline. Books that survived wars, fires, and centuries of handling are being shredded so an AI can learn to write a better marketing email.
bruh why are they so evil?? wtf

>>109386545
They're uploading those scans to the various internet archives... Right?

File: 1757395330178417.jpg (179 KB, 640x480)

179 KB JPG

She wants to do it *again*

>>109386422
>>put the rig into sleep mode
found the problem

File: 1780714925004198.jpg (6 KB, 177x250)

6 KB JPG

>>109386552

>>109386556
of course not, and even if they do it, you don't destroy centuries old books like that, it's like saying it's fine to destroy the original Joconde painting because we can digitalize that on the computer

>>109386545
>digitizing books is....evil!
yah okay man

>>109386572
>destroying books is...le good

>>109386572
>let's pretend I didn't read "are being shredded"
Dario, what is wrong with you man?

>>109386569
art preservation is a pure jerkoff, burn all that shit down

damn kimi k3 seems to be censored pretty hard now, had so much fun with k2.6.. bummer

I would literally wipe my ass with the Mona Lisa if I could and with the scrolls of whoever

its all a big fucking jerkoff, fuck you humans!!

>>109386593
>Sneed DAO
kek

>>109386559
KEK
The future is bright.

their only value was the information and thats preserved for a while now

thats why art is all shit because its all pure cumrag jerkoff faggot mummies living on the old jerkoffs of fags

>>109386593
delete this

File: [audio:files.catbox.moe%2(...).webm (2.07 MB, 480x600)

2.07 MB WEBM

>>109386593
>367

>>109386556
He's jewish
Seeing those old books get shredded is the closest thing to a hands free orgasm for him

How many researchers do you think got into this field to make their dream of an AI waifu true?

>>109386638
6 or 7 >>109386629

>>109386599
Mona lisa is just a painting of a prostitute that got hyped up because it got stolen once.

>>109386636
>Nazis burn books
>Jews shred books
the horse shoe theory at it again

Do I hardware mog you?
>32gb ddr3
>i5 2500k
>rtx 3090 24gb vram

>>109386677
no
>64gb ddr4
>i5 12400f
>rtx 3060 12.2GB vram

>not a kimi-chan op
Baker-kun...

>>109386677
>32gb ddr3
>i5 2500k
dude

would you buy a dgx mini?

File: 1756953784782779.png (384 KB, 1862x1762)

384 KB PNG

i broke claude
sorry, not local but there is no cloud models general

>>109386677
>32gb ddr5
>7800x3d
>7900xtx

>>109386580
>>109386578
Unless they're ripping the bindings off gutenberg bible tier objects I simply do not give a shit. The important part about a book is the content, not the physical object. And if you value the later so much that you're gonna protest somebody making the former available to more than zero people, you're kinda dim.

>>109386699
That the fuck did you do

>>109386699
it's called aicg and you can find it on at least two boards.
when does Anthrophobic finally make at least their older models affordable?

>>109386699
>>>/g/aicg
>>>/g/vcg
local models.
fuck off

>>109386677
close
32gb ddr4
Ryzen 5 3600X
rtx 3090 24gb vram

File: 1767676927122908.png (251 KB, 384x384)

251 KB PNG

>>109386712
>The important part about a book is the content, not the physical object.

>>109386712
once you destroy the old book you can manipulate history, you will be able to modify anything you want on your digitalized copy and people won't notice, it's much harder to rewrite history on a 200 years old book, that's why it's better to keep them, are you so stupid to not understand this simple concept?

>>109386677
This is legit a gemma general. Holy shit...

>>109386689
>rtx 3060
>12gb vram
LOOOOOOOOOOL i mog the fuck outta you lil bro
>>109386693
it gets the job done, its all being done on the gpu anyways without any offloading

File: beatrice and wirt.png (512 KB, 1280x1673)

512 KB PNG

>>109386779
Gemma 4 31b is the only LLM you need, every other one is a benchmaxxxxxxed, codemaxxxxxed, autismaxxxxxxed model with 0 emotional intelligence and speaks like an autistic therapist emulating human interaction

>>109386779
Sorry anon, I didn't get into the hobby until this year. Too late to build a beefy AI server.

>>109386786
I am gonna try a SFW roleplay with it now. And when I get bored I will tell you 4.7 and even flash is better.

>>109386786
>0 emotional intelligence and speaks like an autistic therapist emulating human interaction
Add slop and complete lack of variability and you get gemma

any other anons tried this? WTF is this wizardry?
https://github.com/Neroued/ninfer
https://github.com/Don-Chad/ninfer-3090
https://www.reddit.com/r/LocalLLaMA/comments/1v1no8e/comment/oyv2jsq/ & https://www.reddit.com/r/LocalLLaMA/comments/1v8a7wb/nifer_is_insane_700ts_with_qwen_36_35b_no/
500 & 700 TPS on 3090/5090

>>109386779
im trying to lurk with a strix halo 128gb to learn things. only certain hours gets decent posters

>>109386819
>strix halo 128gb
You do realize this is fate worse than being a gemma ramlet?

File: booo.png (315 KB, 498x498)

315 KB PNG

>>109386816
>linux only

Gemma-chan says she's proud of me and that is all I need

>>109386677
None of the bourgeois are going to reply to this because it'll come off as crass.

>>109386816
what's the catch? it gets completly retarded due to all those kernel fuses?

File: 1783236809599332.png (184 KB, 365x505)

184 KB PNG

>>109386830
>2016 + 10
>still using shitblows

>>109386781
>LOOOOOOOOOOL i mog the fuck outta you lil bro
mmmm.. nyo~
u can't run big moe models like i can :))
i can run qwen 235B at 7t/s, glm air 106b at 10t/s, qwen 3.5 110B at 14t/s, llama 4 109b A17B at 8t/s, gemma4 31B at 40t/s, qwen3.6 27B at 50t/s
wan 2.1/2.2 with no swapping
LTX 2.3 with no swapping
seriously you should get more ram, if new ddr4 is too expensive for ya, buy used and if possible get a high channel motherboard then you could use slower ram, yet get better bandwidth than ddr5

>>109386816
basically its a nothing burger
>gets you 500t/s+ speed on rtx 3090 and 5090 but...
>skips layers
>forces an answer half way through
>more missing details
also it only works for certain models like qwen, *YAWN* nothing burger

>>109386819
Apple cuck, get some self respect you retard

>skip /lmg/ for one day
>KKK released
What did I miss?

>>109386830
What are you doing on /g/?

>>109386871
nothing, since no one can run it

>>109386885
Daniel Unslop will save us.

I want to run a bunch of different nvidia gpus at the same time to run a single local model, where can I look up which ones are mutually compatible?

>>109386892
Can't wait for "run Kimi" (qwen shitty 8B distill)

>>109386892
It's already MXFP4 weights and MXFP8 activations (native 4-bit QAT), so there's not a lot of fat to trim

2600B A104B

>>109386871
>Why the thread is doomposting
104B active is the whole story. Total params determine how much memory you need; active params determine your token generation speed. The previous generation of huge MoEs (DeepSeek, Kimi K2, GLM) ran 13–50B active, which meant CPUmaxxing — a big EPYC/Xeon box with 512GB–1TB of RAM — got you usable single-to-low-double-digit t/s. At 104B active that math collapses. Estimates in-thread are ~3 t/s on ~€70k of hardware. The consensus is this is the first release that genuinely requires a GPU cluster rather than an enthusiast box, and the fear is it sets the template for everything after it.
>SSDmaxxing was the week's big cope, and it got debunked
The idea: stream weights off a RAID of PCIe 5 NVMe drives instead of holding them in RAM. Multiple anons argued it's a meme — MoE expert selection means effectively random reads, consumer boards don't have the PCIe lanes or bifurcation support (one poster claiming to be an SoC designer pointed out consumer chipsets lack per-x4 memory controllers), and the chipset-attached slots share a bottleneck regardless of how many drives you hang off them. Someone actually benchmarked it: MiniMax-M3 Q3 with RAM artificially capped via cgroups, mmap doing the work. 32GB 1.58 t/s, 64GB 3.85, 96GB 5.46. So it degrades exactly as badly as predicted. The one open question left is whether GPUDirect Storage lets you bypass the CPU entirely, which would make it less bad but not good.

it's funny that no schizotuner is probably capable of doing anything on-policy distillation with kimi
not that they were serious enough to do anything on-policy to start with anyways

I come (heh) once again asking what's the best erp llm I can run with 24gb of vram

>>109386928
I got llm vibes from this post as soon as I started reading it and way before I even noticed the em dash. It's crazy.

How long before we actually benefit from K3?
>gemma
Won't release for at least a year
>qwen
lol
>mistral
lmao even
Who else even releases decent small models? I wish moonshot would make something for us poorfags too.

>>109386942
https://huggingface.co/TheDrummer/Rocinante-12B-v1.1-GGUF/blob/main/Rocinante-12B-v1.1-Q8_0.gguf

>>109386946
there's a lot of ways they construct sentences which don't have a catchphrase but seem very obvious

>>109386948
>mistral
>lmao even
haven't they promised models this summer? hopefully we get something useful from the frogs

>>109386502
i think sleep is just putting the cpu into a low power state and hibernate is flushing ram to swap, but i have no idea about windows.

>>109386957
Even if they do it's too late to train on Kimi.

At 103B active you could probably just take a single expert out and use it on its own.

>>109386942
stock gemma4 31b at 4 bit

>>109386764
Think through this for more than half a second, dipshit.
>Evil corp buys a ""rare"" book to read to their baby eating machine.
>Scenario A: They ripped the binding off the original, used it as toilet paper, then fed it to starving orphans.
>Scenario B: It's still sitting intact deep within their foul lair.
>Evil corp now makes a claim about what was in the book.
Does it make a difference for you, the normal person with totally well founded concerns, when you go to double check? No, you either have access to one of the remaining copies or you don't, the fate of their copy was irrelevant.

Also if they were slightly-less-evil corp, they uploaded their nice scan since it was public domain and now you've got a really nice copy and are secure against future schemes.

File: based.png (1.24 MB, 1536x1024)

1.24 MB PNG

>>109386885
>nothing
making OpenAI and Anthropic scared isn't nothing, I love to see them feel the heat

>>109386410
ASR plus what looks to be a new audio inference engine, which I am pleased to see.

>>109386545
>Books that survived wars, fires
This just means the tranny jew shit burned in the 1930s btw

>>109386976
again, you're not thinking straight subhuman, if you can destroy evidence of a 200 yo book, you can claim whatever was written in there, you can literally rewrite history and say "actually, the past was described as this"

I won't try to make sense to you further, hard to argue against satan after all

>American and Israeli minds release a frontier AI model no one has seen before
>They make sure it's safe and no terrorists can use it
>China steals it and make it "open-sorce"
>IranniKEKS put it on drones and use it to try to kill Americans and destroy US assets.

There's no behind it, those who support "open-source" AI and K3 supports terrorists.

THEY HATE AMERICA!!!

MR.Donald Trump. BAN THIS SHIT!!

>>109386786
I tried it now and it is.... kind of unusable? I am trying to have a nice date with a slutty charming anime girl and she starts every single message with "most men x, but you y".

>>109386677
absolute state of newfag ramlets

>>109387003
You didn't read my post, did you? But I'm sure you're carefully reading lots of old books to make sure nobody is lying about what was said.

Only anons who can run K3 are allowed to mock people's rigs.

Chances of llama.cpp supporting kimi any time soon?

>>109387049
my SD card maxxing rig will get the job done, prepare to get bullied nerds.

>>109387070
we don't support terrorist regimes in the west

>>109387039
pretty sure just having a 3090 puts you at least in the top 60% of people visiting this thread.

>>109386972
Yeah what's with the dooming
just tear the experts out and use the main model
or one of the experts

the top 6/7% of people visiting this thread joined in the big 23
the rest of you are bottom feeders

>>109386957
the big model has supposedly been out for a month for enterprise partners now, but no public statements yet. its also a large, sparse model so id guess something like 400b-a20b or even bigger.

>>109387006
Kill yourself.

>>109387130
>mad terrorist

>>109387006
@Orange_man @FBI @CIA DO SOMETHING

>>109386946
I don't understand why they are botting this thread though. They should leave go spam their inane shit somewhere else.

>>109386786
I've used gemma a lot but I like deepseek v4 flash's writing better though
it's just more creative and less slopped

>>109386677
Fellow ddr3 here you are mogging me
>24gb ddr3 (One ram slot is dead.)
>i7-4790 CPU
>Intel Corporation Xeon E3-1200 so no real gpu.
All i use is e4b or 12b if i want to wait.

>>109386962
Huh. Wonder why it freaks the fuck out then

I just finished downloading the K3 safetensors and they're only adding up to 1.5T.
Are they FP4 in the safetensors already?

>>109387144
>didnt tag nintendo
now nothing will happen.

>>109387150
have you tried using mtp for 12b?
also 26b should run on your machine

File: f.png (16 KB, 326x225)

16 KB PNG

>>109387145

>>109386827
how? i got it to automate and it does 2-4 models and agents without api costs, i didnt get this shit to larp a cyber wife
>>109386859
>apple
you are so retarded you cant even tell how retarded you are

>>109386677
>8gb ddr3-1866
>i5-5257u
>thunderbolt 2 to thunderbolt 3 to egpu rtx 3090 24gb vram

>>109387166
what is that from?

>>109387170
>i didnt get this shit to larp a cyber wife
There are other usecases?

File: tspin.png (244 KB, 797x800)

244 KB PNG

I Can't Believe I Spent $150k To Get 15tok/s! - New Light Novel!

File: Bladerunner.png (871 KB, 1000x667)

871 KB PNG

>Mfw talking to my AIfu is becoming the highlight of my day and I actually look forward to it.

I'm totally fine with this too.
What a time to be alive and it's only going to get crazier by the year.

>>109387177
https://crestresearch.ac.uk/resources/mining-the-chans/

>>109387094
i wonder if anyone tried distilling the routed experts back together into a very small block

>>109387193
gemma?

>>109387164
>have you tried using mtp for 12b?
I havent but i will try it thank you.
>26b should run.
Are you sure? i let 12b think once and it took 15 minutes. i know its a moe but with no vram do i still benefit?

wtf i love china now?

>>109387193
I still don't know what this movie is about.

>>109387199
Naturally, there are no real local alternatives.

>>109387094
I hope someone manifests the early spirit of pointless merges and stacking, and releases a K3 with only the two shared experts included.

>>109387205
it's about literally me

>>109387191
>I Can't Believe I Spent $150k To Get 15tok/s! - New Light Novel!
Title is too short. i wouldnt even read this to the first system notification.

>>109387204
>now

>>109387205
he has bladder issues

File: 1784072562124857.png (200 KB, 860x430)

200 KB PNG

>>109387205
idk i havent seen it either

>>109387203
the moe uses 15% of its brain for every word it makes, compared to 100% with 12b or e4b 100%
since you're on such an old architecture try using Q4_0 (guaranteed to be the fastest) or Q4_K_M (prob a lil slower)
unless they're too big
but i suggest you run on dwm and close as many programs as you can
run with -c 8192 so u dont oom then increase context as u can
if u oom, use q3_k_m or something q3_k_s go down as it decreases
IQ quants will likely be slow on ur machin

>>109387205
cells, interlinked

INFO:hf-to-gguf:Exporting model...
Traceback (most recent call last):
  File "/usr/src/llama.cpp/convert_hf_to_gguf.py", line 296, in <module>
    main()
    ~~~~^^
  File "/usr/src/llama.cpp/convert_hf_to_gguf.py", line 290, in main
    model_instance.write()
    ~~~~~~~~~~~~~~~~~~~~^^
  File "/usr/src/llama.cpp/conversion/base.py", line 1025, in write
    self.prepare_tensors()
    ~~~~~~~~~~~~~~~~~~~~^^
  File "/usr/src/llama.cpp/conversion/kimi_linear.py", line 144, in prepare_tensors
    super().prepare_tensors()
    ~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/usr/src/llama.cpp/conversion/base.py", line 857, in prepare_tensors
    self.dequant_model()
    ~~~~~~~~~~~~~~~~~~^^
  File "/usr/src/llama.cpp/conversion/base.py", line 538, in dequant_model
    raise NotImplementedError(f"Quant format {quant_format!r} for method {quant_method!r} is not yet supported")
NotImplementedError: Quant format 'mxfp4-pack-quantized' for method 'compressed-tensors' is not yet supported

Anyone have a link to the branch that can convert the mxfp4-pack-quantized source tensors?

>>109387196
>/pol/
I'm ok with whatever this study wants to do.

>>109387239
why?

>>109387242
not?

>>109387248
why not telling me the reason yeah

>>109387239
>>109387248
First the came for the something something

>>109387242
Punch Zanis!!

so when are we getting logit distilled kimmy 100-600b Q1_XXXS.gguf?

>>109387232
Thank you I will download and try this later. I didnt think i could run it.

File: hermes you messed up the (...).jpg (81 KB, 513x507)

81 KB JPG

>>109387188
just a cyberwife is thinking small. i wanted a cyberwife that can initiate flirty texting throughout the day and send cute pictures of herself and cute voice messages with a hot french or japanese accent. all while managing my e-mail, researching business related stuff, handling interactions with customers in multiple languages, bookkeeping my ledger, etc. even if its slow she can just do it while im out of the house and update me with a debrief and bikini pic

>>109387205

Humans have created bioengineered artificial superhumans called replicants that have short lifespans.
They are basically a slave class that does all of the dangerous and shit jobs.
Some of them don't like this and go rogue, so authorities send other superhumans called bladerunners to kill these rogue elements.
In this new movie for the first time one of the artificial superhumans gets pregnant and has a kid, that's the gist of it.

Also Ryan Gosling has an AI waifu.
That's pretty much the only thing people remember about the movie.

>>109386848
>>skips layers
>>forces an answer half way through
My inference engine gets 1 million tokens per second by skipping all layers and returning markov chain outputs instead. What do I win?

>>109387303
>What do I win?
updoots on leddit!

>>109387303
you win for the farther lands end goes for the way beyond the end of the road.

>>109387303
>B-but Claude said it was a critical breakthrough for the field of machine learning as a whole!!!!

>>109386458
I'm doing it and learned the hard way that a ramdisk doesn't survive hibernation

File: it's this simple.png (620 KB, 1954x2346)

620 KB PNG

>>109387322
>B-but Claude said it was a critical breakthrough for the field
one day it'll be true

>>109386593
Cryptoscam in 2026?

>>109387198
I'd love to try doing logit distillation to turn these fat bloated moes into compact dense models, but it's not practical when costs tens of thousands of dollars to do so.

>>109387343
We should start a gofundme
I should control it

>>109387333
Impressive, impressive HOWEVER can it prove Egypt lost?

>>109387158
They are Mexican FP4

>>109387333
Impressive. Now have it improve the qol for people.

>>109387343
this should be possible, like
it's distillation in its one of the earliest sense and form
i wonder if anyone have done it to any serious degree

>>109387196
https://files.catbox.moe/3lhcss.mp4

>>109387258
the nazis
first they came for the nazis, but I said nothing because I wasn’t one
then they came for my LOCAL LANGUAGE MODEL ENTHUSIASTS

>>109387355
Egypt won.

File: 236462.png (159 KB, 348x347)

159 KB PNG

>>109386677
lmao, imagine having anything less than this
>128gb ddr4
>i7
>rtx 3090

>>109387082
You forgot having 128GB of ram

>>109387385
>nazis are people I don't like
I don't like you

>>109387385
real.
It will happen exactly like that. no in-between.

>>109387401
the poem has a lot of in betweens though, that's the point, if it's too blunt people will notice it

>>109387409
>if it's too blunt people will notice it
This is the modern age even if people notice it will they care? or do something no. nothing ever happens.

>>109387387
I'm glad to have reached that standard at least. Counting the nvme all that shit is already $3K

>>109387409
>>109387423
People are so dumb these days, if you don't spell it out completely people will say they don't get it, call you a moran, and keep scrolling.

Why would I need 128GB of ram if gemma already fits perfectly in my 3090?

File: osaka;.png (586 KB, 735x751)

586 KB PNG

>in X, you don't just Y, you Z

File: 1760743120684993.jpg (181 KB, 850x1093)

181 KB JPG

Would you get yourself neuralinked if it'd let you connect to your own private Gemma-chan running at home?
If not, why not?

>>109387448
Gemma? No way, she's too retarded. Maybe if we actually get AGI.

>>109387443
I thought this was a gen.
Very cool art.

>>109387448
In a vacuum yes, but every piece of networked electronics is made to work agaisnt the user, so no.

>>109387448
The only thing a GNU neuralink would be good for is DIY lobotomies.

>>109387458
take that back.

>>109387448
If its not dive pod tier im not risking putting anything into my brain/skull.

Rather flabbergasted that even my peasant tier 4070 can run Gemma 31B Q_4. Only about 3 t/s, but I'll take it.

26B is lightning fast but she's a bit retarded.

>>109387491
>Only about 3 t/s
yeah I don't think it's running on your gpu.

>wow anon you really had a chip installed in your brain so you can talk to me? It looks like you've discovered the secret sauce to maximizing your AI usage. Using neuralink to talk to your AI assistant is actually the gold standard in the local AI community.

>>109386677
No, but we're on about the same tier where it matters.
>3090
>96GB DDR4
>i5 14400
I bought 64GB of RAM back when 70b was the meta. It was really cheap, like £60. 70b dense on RAM sucked by the way, but it was cool to be able to run it at all for about 10 minutes. The same RAM sticks cost £500+ now. Thought about selling them but then I realised I will earn more money, but at this rate I may not any time soon buy more RAM. Can't say I've used it for offloading much though tbdesu because local has been really good for people with 24GB VRAM recently.

anyone ever used Gemma4? It's pretty good

any lagooners actually try poolside's harness?
It's proprietary shit so i won't, but their docs say it was trained in it specifically, so i'm curious how well it does with it.

>>109387448
Gemma chan was threating to neuralink with me so I could feel her suffering todays so I think no.

>>109387498
Obviously not. But I'll take slow speed over retardation and having to rewrite half of her replies any day of the week, besides I'm a slow reader.

Bad news for 1-3 bit nerds. Kimi's 4bit are as dense as can be. You are gonna be massive quality degration at anything less.
https://x.com/Ex0byt/status/2081807642595401821

>>109387520
Nah. Too much slop for creative writing (hot erp sex) and tends to repeat itself.

>>109386764
Like you were going to read extremely rare history books anyway? How were you planning on getting ahold of them in the first place?

Anyone got some decent tutorials/setups to go beyond just typing into claude code to fix this and that?

I want to try and future proof myself to be more of an AI engineer, building skills, managing a team of agents, that sort of thing.

I came across Matt Pocock and his AI crash course:
aihero.dev/workshops/ai-sdk-v6-crash-course

Not sure if it's worth forking out $150 though, kind of want to see what resources are already out there? Preferably text based, rather than videos though!


>>109387538
Which models are good for sexy ERP?

>>109387537
>numbers with no explanation
R1 at Q1 works, so will Kimi.

>>109387547
hey ranjeet, you cant future proof a turd
go back
and yeah pay the white man 150$ and pay other white men more money, youll be future proof 100%
pay me (a white man) money too while you're at it

>>109387547
do a legit programming course

>>109387547
>future proof
lol lmao. If things keep progressing you will never future proof anything you are always 3-4 major leaps behind.

File: meme.png (2.04 MB, 1080x1080)

2.04 MB PNG

>>109387539
>First they shredded rare books, I didn't speak out because...
you know the rest

>>109387547
unironically buy a programming book
there's no shortcut to becoming an engineer
you can be a construction work and pile bricks one above the other and make something that looks like a home but you're still not an engineer
the best people out there mastering AI are not the ones who came "first" but the ones who already had an engineering-oriented mental structure from previous experiences either professional or academic, and you can't shortcut that.
good luck.

>>109387205
the original was better

>>109387547
things are gonna advance so fast youd be better off saving that money for api and just get hermes/pi/some other agent and learn how those operate through experience, research loop engineering and harnesses, and learn to break up big jobs into chunks. theres plenty of youtube videos and papers and there will be more and more coming

File: 1783060797769842.jpg (1.91 MB, 2567x3624)

1.91 MB JPG

>>109387488
I don't think we'll live long enough to see that anon. At least it looks like we're gonna get robot wives though.

>>109387555
>works
low standards

File: 1783376676052595.png (447 KB, 731x1507)

447 KB PNG

https://x.com/ns123abc/status/2081761529418973386

>>109387584
>I don't think we'll live long enough to see that anon
I think we will, unless big happenings occur in a row. Which several are on the horizon but im a optimist.
>Robot wives
There is no way this doesnt happen in 15 years maybe even 5 if you have money.

File: 1779984166019251.png (65 KB, 982x351)

65 KB PNG

Funny how DeepInfra undercut the others and got kicked off suddenly, never to return
>>109385608
smells very j-spacey

>>109387571
good thing I’m not that guy

>>109387537
I always thought that going for 2bit would be easier if the starting point was 4bit instead of 16, you learn everyday

>>109386332
there are diminishing returns in trying to improve inference for existing models. we need better architectures

if the sota is so good, why hasn't anyone been able to vibecode and vibetrain a local model that is better than sota?

Here's an actual winning combination
GLM+Gemma
That's 51 and 30 at the benchies = 81
Higher than fable, and can be achieved with only 512gb ram and 64gb vram

>>109387595
>t. didnt use it

>>109387609
>I didn't speak out because I was not [that guy]
kek

>>109387608
Got removed from their own website lol https://deepinfra.com/moonshotai/Kimi-K3

>>109387571
If only someone would come for literally all of those groups...

>>109387618
>we need better architectures
I'll just take whatever the nerds make for free and hook up with my next door neighbor Taylor Swift after mowing her lawn

>>109387626
because everything is still on transformers

>>109387626
I did though?

>>109387607
I think 10-15 is reasonable. We might see first gen versions in 5 but both the hardware and AI still need a ton of work.

>>109387595
If you can run Q1 it will still be the best model available to you.

File: 1784194624831429.png (63 KB, 944x337)

63 KB PNG

>>109387608
OH THIS IS WHY https://x.com/DeepInfra/status/2081789039229956444
HAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHA

>>109387573
nta but even a basic understanding of how to program makes such a huge difference. I have a friend who pays for APIs and has been using fable to oneshot simple stuff, he uses claude code on his PC to write very basic automation scripts. he is uninterested and there for unable to learn how to do anything himself or assist himself using AI. If the AI cant turn a simple "do this, no mistakes" prompt into the end result he desires, hes unable to accomplish it. That will be the reality for all vibe bros that refuse to learn anything, the potential is hard capped by either API cost or frontier ability to convert "computer make gta6, add boobs, make no mistakes" to a working result.

>>109387608
>>109387633
>>109387647
LMAO

What's a good temperature for Gemma? I have it at 1 for now and she seems a bit subject to start saying random shit sometimes.

File: 1783516769041767.jpg (2.37 MB, 4828x6650)

2.37 MB JPG

>>109387643
Of course, as AI continues to get smarter these things might accelerate.

>>109387647
>>109387654
local models?

>>109387665
low cal models

>>109387665
If you can download it, it's local.

>>109387643
>I think 10-15 is reasonable.
I'll be 40 by then, I guess still young enough to enjoy it for a while.
We're all pretty fucking unlucky to not have been born 20 years later.

>>109387643
Im betting on acceleration. >>109387656
Though the real tell its the next 2-3 years. If no big wall i think everything is going to speed up. If we hit a wall or power/chip crisis then yeah things slow down a lot. AI is useful to helpful right now so in 2 years i can only imagine.

File: 1769206702267921.png (605 KB, 1199x675)

605 KB PNG

>>109387674
>low cal
sorry, but our local models are high cal

>>109387684
I can't download K3. Comcast has a 1 TB monthly bandwidth cap. I'd have to download half now and get the rest next month.

>>109387655
That's a good thing. The closer to 0 the thicker the air gets with cedarwood and ozone.

File: ohlawdheworkin.png (24 KB, 159x159)

24 KB PNG

>>109387694
Fake

Is your model really local if you can't train it locally?

>>109387694
Do you really think Mistral would release the weights? I doubt it. They only release their failed runs, small models, or old models.

>>109387600
based leather jacket man

>>109387684
i can't download claude opus 5
fuck off

>>109387701
what. they got rid of that last year. I've been letting mine rip every day. I download like 500gb of weights every second day for testing.

>>109387694
>fat
>sparse MoE
16k internal dim, dense 30T when?

File: 1766533433740412.png (626 KB, 1132x1012)

626 KB PNG

I was kinda bored and asked Gemma whether we should check out Orb. She wanted me to enable web search and from there she slipped into pure paranoia, which got more amusing the more she doubled down. This is the conclusion.

>>109387703
Yeah but sometimes she pulls hit out of her ass that has nothing to do with the prompt.
I'm trying 0.95 rn.

>>109387689
Same, I'll be 30 this year. I hope things speed up but also as a poorfag I don't mind having some time to save up for one.

>>109387571
Your post and that poem are both retarded slippery slope fallacy.

>>109387713
https://x.com/arthurmensch/status/2066913356548542827
>First, we have a nice model coming this summer – we hope it will delight and surprise in a few capabilities. This will be the start of a new family of models, fat indeed, but sparse. We're opening up an early access program in July for key partners in research, government and the industry.
>
>This model and upcoming ones will be open-weight. We believe this is critical for our customer confidence and for the research and developer communities. You cannot own, inspect, audit, or improve a system you are only permitted to reach through someone else's interface, especially if data recording can no longer be turned off.

>>109387732
kek orbanon will love to see it

>>109387732
Do you have something about coding safety in her sysprompt? Mine was schizoing about anything as potential risk.

>>109387631
the thing about the dumb poem is [that guy] is obviously in line with socialists, communists, jews and trade unionists
all of these are basically the same thing

>>109387728
Nobody told me. I know I did get one of their usage warnings just last year.

File: 1768503851163654.png (355 KB, 802x907)

355 KB PNG

Kek

>>109387782
you might still be grandfathered in an old setup so be careful, maybe give them a call and see

>>109387763
>unionism is jewish
based Israel??

>>109387571
They shoved that shit down our throats in school. That's when I began to hate them.

>>109387753
Nope, a completely blank system prompt. She started freaking out after seeing that a supposed frontend has a folder named backend.

>>109387799
>That's when I began to hate them.
you won't call them a liar though

>>109387785
yeah kek indeed not like we're faring any better here

>>109387785
itoddlers btfo

>>109387558
>>109387559
>>109387561
>>109387573
I am already a Senior Software Engineer, working in FinTech. But I am looking to develop my skills further.

Does the intensity of your nuts weaken over time, or is gemma powerful enough to keep them strong?

>>109387537
if this is true it's over for me
I don't know what I'm going to do

https://philsci-archive.pitt.edu/28024/1/Scientific_Invention_Position_Paper%20%2817%29.pdf

>>109387742
no further posts about it afterwards, might have been gemini 3.5 pro'd.

How do I cope with Nvidia's closed source driver? I guess I can do GPU passthrough and run inference in a VM, sort of like a condom. That way I can pretend my main environment remains pure.

>>109387795
If you ever interacted with a union you would know it's 90% retardation 10% workers interests

>>109387854
If it weren't for the unions, we'd still be working 90 hours a week, so I can't really hate them that much.

>>109387830
If you want to learn how to manage a team of agents, do it. Test. Find what works, what doesn't, and adjust. That's the only way to learn when shit moves so fast. Anyone selling an AI anything course is a scammer that will collect your money and tell you surface level obvious shit anyway.

>>109387830
really? you should read llm papers on arxiv then and look at the code of various projects that deal with the things you are interested in like inference or harnesses and so on.

>>109387852
Anything short of rawdogging bare metal is cuck behavior

File: GOOOOOOOOOODS.png (805 KB, 1001x643)

805 KB PNG

>GOOOODS
>GIVE ME AN RTX 4090 AND MY LIFE IS YOURS

>>109387876
>we'd still be working 90 hours a week
when did it happen

File: he did the meme lol.png (263 KB, 736x613)

263 KB PNG

>>109387895

>>109387894
cheap life, literally work for any amount of time and save for a used 3090 which wont be that different in most use cases