jart lost

>>106388962
>last llamafile commit 2 months ago
I don't think Jart is still relevant.

>>106389006
Did the mozilla money he scammed dry up?

>>106389006
Sorry, I should have clarified: the last llamafile commit by anyone at all was 2 months ago, the last commit by Jart was 7 months ago.

>>106388944
why migu bred?!?

File: 1756140108129600.jpg (107 KB, 662x656)

107 KB JPG

Reminder

>>106389109
simps will be the first to die

>>106389109
This, but unironically. The man and the machine are destined to merge.

File: 1722753968153.png (313 KB, 662x656)

313 KB PNG

>>106389109
I suspected yet again that this was a shitty compressed version as there was suspiciously little instances of this image in the archive. Well here's the most original quality I could find, which does have many instances in the archive.

File: 1755138097813238.jpg (17 KB, 662x656)

17 KB JPG

>>106389109
Here's a more compressed version to save space

File: file.jpg (15 KB, 662x656)

15 KB JPG

>>106389446
>17KB
That's very wasteful.

File: file.png (1 KB, 195x24)

1 KB PNG

>>106389492
>(15 KB, 662x656)
Excuse me?

>>106389497
Just cloudflare things. Look at the image in the archive.

File: 1738779145029853.webm (9 KB, 662x656)

9 KB WEBM

>>106389492
I've learned from my mistakes.

>>106389096
she wants to be bred.

>>106389241
Cringe
>>106389245
Based
>>106389416
Going forward, I'll ritual post this version

>>106389600
nice one bro, grok will totally see this!

hatsunald mump

>>106389629
The only llm I care about is the one I have trapped inside my local machine

What do you GLM-4.5-Air Chads use for prefill?

ever notice how those 'llms won't reach agi' folk have been reeeeeal quiet ever since strawberry dropped?

>>106389109
This would be more relevant if our machines were actually sentient and not simple text prediction models. Not to mention the issue of constant birth and death that happens with each forward pass.

Almost any model needs some general knowledge and baseline understanding to be useful and consistent.
How many parameters does that typically take?
Like, if I’m building a specialized model, how many parameters would I need just to cover the basics?

>>106389763
There's no need to state the obvious.

>>106389823
Even 8B has some decent common knowledge to be coherent, but it's still pretty dumb. Personally I'd say 20-30B is the minimum amount.

>>106389823
depends on your usecase

>>106389744
Prefil the thinking with some schizo guidance that I used with gemini 2.5.
I haven't played much with it yet, I suspect that might not be the best way to go about it, but it seemed to work fine for RP.
Here
>https://privatebin.io/?1ce1f80a5cba2c72#HJr2wSVYqzouuWQCLxyaeKVn1nJ1XFQ1G5KxS9iG7Mtw
As is, it's a pretty brute force prompt that can probably be made a lot smaller for the same effect.

What are the best smol vision language models? There seem to be dozens and they aren't discussed here much. And which ones are supported by llama.cpp or kobold.cpp?

why are there so many tech support questions? you think in an LLM general you'd would have asked chatGPT at least once, or you know, google.

File: computers-must-shut-up.png (475 KB, 900x900)

475 KB PNG

>>106389416
obligatory

>>106389980
>you think in an LLM general you'd would have asked chatGPT
you are even more tech illiterate than the people asking questions if you think asking chatgpt is a legitimate option
just the knowledge cutoff stuff alone means it's going to hard fail the average knowledge question about new models/llama.cpp features or whatever
it can web search, yes
a web full of LLM hallucination slop where the number one hits in google are often barely above markov chain tier logorrhea
chatgpt kek

btw using SOTA models for coding I constantly have to remind them not to do things like using require() in an age of ESM imports
llm are trained on garbage outdated content
garbage in, garbage out

>>106390046
And that's why you need to be a programmer in order to use AI for programming.
Vivecoding is just cope

>>106390046
I had that problem back when I used to copy paste code to and from the web chat interface.
Now that I'm using these agentic whatever, I just have a rules file explaining the do's and don'ts, the workflow (explain the what where how why), etc.

>>106390079
Pretty much.

>>106389823
Around 300-400B seems to be the sweet spot. You shouldn't notice major gaps in its knowledge if you're training a model of that size on everything you care about.

>>106389980
Any general Google searches on a topic nowadays only gives you shitty articles written by LLMs or by Indians.

>>106387167
based, downloading model
>>106386519
https://huggingface.co/llama-anon/grok-2-gguf

File: giveup.png (63 KB, 300x258)

63 KB PNG

>8 months since mistral small and there's still nothing better for smut
it's over

>>106390250
It's a harsh world for vramlets

>>106390190
its not 2023 anymore. 1T+ parameters is minimal to get something remotely useable.

>>106390275
I like GLM

>>106389999
Anon, prove you aren't a computer

>>106390250
Buy another stick of ram and use air.

>>106389744

Continuing.
---

>>106390250
There's only so much a small amount of parameters can do no matter how much you finetune it or not. I'd recommend glm air or
https://huggingface.co/bartowski/EVA-LLaMA-3.33-70B-v0.0-GGUF.

File: BENCHMAXXING-AGAIN.png (169 KB, 688x676)

169 KB PNG

the subhumans of nvidia are at it again

>>106390434
guess they didn't like saying PNAS out loud

>>106390434
ARM64 dense bros we are so back

File: moon.png (25 KB, 620x150)

25 KB PNG

>>106390434

Intern ggufs doko

>>106390434
kek
big research

>>106390339
Wouldnt that be incredibly slow

>>106390615
depends on where you buy it and the kind of delivery service they use.

>>106390434
I really want that deepseek-v3-small.

File: 7463W.png (96 KB, 1264x772)

96 KB PNG

Sirs when local banana?
chinese google make model next month?

File: v3small.png (87 KB, 937x658)

87 KB PNG

>>106390642
apparently it's a real model, I had never read the DS report before and didn't know they had unreleased models like these
https://arxiv.org/html/2412.19437v2
". At the small scale, we train a baseline MoE model comprising 15.7B total parameters on 1.33T tokens"
the shitotron report refers to this model's benchmark in ds3's TR when they talk about a deepseek v3 small

>>106390763
bwo?
https://huggingface.co/deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct

>>106390763
>15.7B total
Sounds like DS2-lite arch
https://huggingface.co/deepseek-ai/DeepSeek-V2-Lite

>>106390788
>Their hyper-parameters to control the strength of auxiliary losses are the same as DeepSeek-V2-Lite and DeepSeek-V2, respectively
they are not those models, just similar arch
they really are new trains that were never released

>>106390763
I know. I've wanted it since they released v3.
>>106390788
You can't read, can you?

>>106390814
I can hear just fine

>>106390999
*plap plap plap*
do you hear that? it's the sound you make when..

>>106385961
>so many responses
Are you guys that starved for some blacked miku? Should I post some??

So how is glm-air-safe from drummer? Did he manage to make it more safe to use?

https://youtu.be/ZPCdW-pPZO0
It was interesting to me since I never cared to think that much how whores approach the topic.

>>106391210
sloptuners fear big moe models

>>106391210
No one figured out how to tune GLM 4 32b even (although I think it's fantastic as is). I don't think Air is happening.

>>106391287
he posted a link last thread. I am not gonna paste it myself cause he should die.

>>106391308
Maybe I should have rephrased that. No one figured out how to tune GLM4 32b in a way that even the tuners themselves thought it improved the base model.
And judging by what drummer said I don't think he's particularly confident in this one, either.

>>106391210
Why would glm air even need a tune? It already works for a lot of stuff.

>>106390434
>my 2B > your gorillionB

>>106391424
Only model that needs a "tune" is gemma. So weird how there is none.

>>106391500
There's dozens of Gemma tunes.

Is there a way to tell how well utilized the weights of a model are? Lets say 30% of a models weight are just random noise. Would you be able to tell that they are fucked if the model still performs decently?

>>106391549
no one knows anything bruh, these things are just big black boxes of random math

why is nobody talking about vibevoice? this is easily the best sounding TTS for local users, it only uses like 8GB VRAM and it supports voice cloning and real-time streaming.
https://github.com/microsoft/VibeVoice

>>106391569
xtts needs way less ram and supports many languages

>>106391569
Sex? Moans?

>>106391569
I'm happy with Piper as that's enough for my own needs. It takes less than 100mb of memory and is pretty much instant regardless of what LLM model I'm running in the background. Sure it's limited but can be pretty cool in some cases.
>https://litter.catbox.moe/ffldl8v6hp11c52a.wav
LLM's text output is always bit random, it needs to cleaned up really well before sending anything onward to the text to voice model. It took a while to test this one out.
VibeVoice is really cool though.

>>106391657
to an extent, you can make some pretty convincing sounds with some ughs and uhhns and other creative ways to write out moans. ymmv but it's definitely possible.

>>106391558
they are just arbitrary numbers but the math itself isn't random. people have put a lot of research in to whats going on inside them, but much of it is not really accessible to the layman.

>>106391569
>no japanese

>>106391569
https://huggingface.co/amphion/TaDiCodec

>>106391569
the 1.5b seems meh, the 7b sounds decent but i haven't decided if i like higgs3b more or not yet.

>>106391672
took your prompt and fed it through vibevoice.
>https://files.catbox.moe/sguyhz.wav

>>106391787
it seems to pronounce japanese pretty well if you use romanji

Updates:

GLM Air, works better than v1a: https://huggingface.co/BeaverAI/GLM-Steam-106B-A12B-v1b-GGUF

Skyfall upscale with creativity boost (Similar to Signal): https://huggingface.co/BeaverAI/Skyfall-31B-v4j-GGUF

Saars I'm tired of testing. Lets build something involving local AI. Even if it's agentic loli rizzer.

>>106392053
learn to inspire yourself instead of relying on others anon
unless you're brown, then you just have no inspiration to pull from your inner self

Running glm-4.5-air-q4_k_m on a 4090D 48GB + 128GB DDR5 system ram for offloading with a 16-core AMD Ryzen 9 7945HX. It runs quite fast, I'm happy with the speed, it's plenty of RP.

Why is no one talking about DeepSeek V3.1? Is it that bad?

>>106391983
ill give the glm air models a test tomorrow, sorry drummer i got something up today

>>106392147
Its intelligence isn't bad, but it's stylistically mangled

>>106392174
No problem, anon. Hearing tons of good feedback on v1b so far, so prioritize that one.

(Also Skyfall is getting good feedback as well)

>>106391569
>Our training data doesn't contain any music data. The ability to sing is an emergent capability of the model (which is why it might sound off-key, even on a famous song like 'See You Again'). (The 7B model is more likely to exhibit this than the 1.5B).
That's incredibly interesting actually

>>106392077
that's no the issue. I'm a crackhead building things 24/7. It just takes too long to build something cool as solo dev. Especially in the world of AI where change is constant. I mean that's why open source is even a thing. Suddenly you have 10 autistic crackheads working on a project, which is then finished in months instead of years (and they do it for free!). So recently I bought a bunch of vps and have vibe coders solutions running there 24/7 building various additions to my main framework while I work on my own stuff. the problem is not even the quality of the output they generate. it's more the lack of proper tools to debug/test stuff and needing supervision because of that.

new model just dropped

>>106392391
did it break?

>>106392391
Bitnet bros we are back

>>106392391
>claude-3.5-sonnet-oss
wow I didn't think anthropic would actually do it, based as fuck

File: Gn6M4I_aEAAmh7k.jpg (229 KB, 1331x2048)

229 KB JPG

fuck you shgu

>>106392510
Lmao, just realized. Tetofags in shambles.

you never a teto you above me

>>106392489
What is the cockbench ?

>>106391891
Nice. I'll try implementing this for my client if it seems like it's easy enough to run.
Piper is dead simple, you just need to use contractions module plus filter out special characters and remove extra periods and such.

>>106392634
0

>>106392201
Thank you for your service, Sir. O7~!

>>106392715
RTX 50xx series
Python 3.11 venv
pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu128 --force-reinstall
pip install https://github.com/mjun0812/flash-attention-prebuild-wheels/releases/download/v0.4.10/flash_attn-2.8.2+cu128torch2.8-cp311-cp311-win_amd64.whl
pip install triton

File: Boxxy.jpg (13 KB, 318x272)

13 KB JPG

>>106392927

File: 1754878920639183.png (142 KB, 535x528)

142 KB PNG

Nobody told me it would take 1 billion years to process prompt and an eternity to generate output when you offload to system ram, now I feel scammed

>>106392118
How fast?
I'm getting about 17t/s with no context using a MI50. Pretty good for the price, also I may get higher speed with vulkan once I manage to flash the vbios.

>>106392118
>glm-4.5-air-q4_k_m
yeah i'm also reasonably happy with glm air.
problem is i've only got the one 3090 so only get like 3-5 tokens a second.
would like another card to run it at like Q5 or Q6 though, would iron out some of the retarded word choices it makes sometimes.

>>106392962
It has been explained a million times that offloading models to system RAM and SSDs will make the model slower. Lurk moar.

>>106393006
?

>>106392962
>Nobody told me
I don't believe you. But even if it is true, couldn't you have reasoned that? Wouldn't you have wondered why AI companies brag about their h100 count? Or why we quantize models? Why there's so much talk about videocards?
I don't believe you for a second.

>>106393002
You need 4 RTX 3090s to run a Q6 quant of air and have it fully loaded in VRAM. I know from personal experience.

>>106392962
>Nobody told me
We all just assumed you knew that using slower memory would be.. Slower.
Are you at least doing this on an MoE model? Tell me you aren't trying to run a dense model on fucking system ram.

What are the performance gains like when you overclock your ram for offloading? I'm planning to boost my ddr4 speed.

>>106392962
There are a lot of black magic optimizations you're missing

>>106393006
>>106393022
You guys didnt complain enough for me to believe it until I tried myself
>>106393031
Yeah its Moe, I scrapped 2 memory sticks from some shitter pc and thought I was gucci with 64gb ram, well at least now I can open two chrome windows I guess

>>106392962
Massive skill issue.

>>106393070
>You guys didnt complain enough for me to believe
So you *did* know. Stop pretending to be a retard. You're not smart enough to pull it off.

File: 1733629039939868.jpg (1.04 MB, 1060x1500)

1.04 MB JPG

>>106393056
>>106393072
>>106393092
T-teach me your ways, sensei!

>>106393103
First of all post specs and what are you trying to run.

>>106393103
The first thing we need to know, and the most critical, is the color of your pc case and how many leds it has. We'll go from there.

>>106393056
MoE black magic optimizations in order of how big a difference they made to me.
1. Locking memory clocks in nvidia-smi with -lgc and -lmc to the boost clock rating of my cards.
>Doubled my tg t/s from 5 to 10.
2. -ub 4096 -b 4096 in llamacpp args.
>2-4x my PP t/s
3. either -ot "\.(2[5-9]|[3-6][0-9]|7[0-9]|8[0-9]|9[0-4])\..*exps.=CPU" in llamacpp (keeping as many of the first few blocks on gpu) or -ncmoe
>Gave me about +50% PP and TG to start with from the fucked up 2-3 I started with when using just -ngl to my limit, to 5-6.

>>106393103
llama-server --model <Path>\DeepSeek-R1-IQ2_KS-00001-of-00005.gguf -fa -rtr -mla 3 --ctx-size 40000 -ctk q8_0 -b 4092 -ub 4092 -amb 512 --n-gpu-layers 99 -ot "blk\.(3)\.ffn_.*=CUDA0" --override-tensor exps=CPU --threads 8 --host 127.0.0.1 --port 8080

Not them. Oh and you need to fiddle around probably. And of course I am assuming you are running memefork.

>>106393131
Thanks anon. This is my computer, I have 64GB of RAM which I thought was enough. It's supposed to be AI capable with the hardware specs.
https://www.lenovo.com/us/en/p/laptops/thinkpad/thinkpadp/thinkpad-p16s-gen-4-16-inch-amd-mobile-workstation/21qr001sus

File: tenor (1).gif (714 KB, 344x426)

714 KB GIF

>>106393188
>AI capable

File: file.png (4 KB, 143x48)

4 KB PNG

>>106393188
Anon you are fucking with us and trolling us.

>>106393115
>>106393115
>>106393131
4080 16gb + 64 ddr5
>>106393188
Stop impersonating me NIGGER

>>106393225
>16gb of vram
That's a bit though but maybe it could be possible to run gml air at okay speeds. Maybe

>>106393225
Why are you replying to me? I'm obviously calling you a retard.

>>106392962
>>106393225

8g amdgpu vram 64 ddr4 ram
5tks on GLM Air Q3
pp is around 10tks
it's slow but I wouldn't call it glacial.

you shouldn't get worse numbers than me

>>106393225
>4080 16gb + 64 ddr5
You can run GLM Air Q3whatever at some 8t/s at an empty context, I think.

>>106391500
There's synthia s1 and glitter which are kinda ok, then there's like 4 from drummer but I don't really like them, they just turn every single character in a sadistic psycho for no reason

>>106393297
i thought 10tk/s PP was just a joke, there are actually poor souls out there processing tokens this slowly?

>>106393342
I used to be cpuonly, you have no idea...

File: computers.jpg (866 KB, 1402x2000)

866 KB JPG

>>106393342
I am amazed it works at all, desu.

>>106393395
people actually believe this shit

>>106393395
>150gb tape backup drive
That's massive. I guess I was too poor to even know these existed.

File: file.png (66 KB, 473x529)

66 KB PNG

>>106388944
https://huggingface.co/collections/NousResearch/hermes-4-collection-68a731bfd452e20816725728
shit's based on old Llama 3.1 models...
>Hermes 4 70B/405B is a frontier, hybrid-mode reasoning model based on Llama-3.1-70B/405B by Nous Research that is aligned to you.

Hermes 4 is kino. You can liberate it super easily and go hard. Does it all.

File: 1755063432991367.png (2.57 MB, 1024x1536)

2.57 MB PNG

>>106392962

>>106393698
>hybrid-mode reasoning
>based on Llama-3.1-70B/405B
holy fuck they trying to rival llama 4 huh

File: file.png (190 KB, 600x532)

190 KB PNG

>>106393767
from X

>>106393033
I had to lower my ddr5 speed to add another 2 sticks in. it made barely any difference on the token generation speed. I'd expect not much. maybe a percent or two if you are really lucky.

File: file.png (71 KB, 571x323)

71 KB PNG

>>106393762

File: t_HvRYPEHV0pc8iS2zHHn.png (104 KB, 572x562)

104 KB PNG

https://huggingface.co/NousResearch/Hermes-4-405B
>405b answers 57% on refusal bench
>no modified sysprompg
>best score
i kneel

So this is how far AI glasses are now.
https://youtu.be/kaNPCW9M55A?t=593
We're getting close to some digital nomad hackerman dreams.
Or Jarvis/Edith glasses if you're a marvelslop eater.

I don't think it's that surprising that there might still be Llama 3 tunes coming out. It's the last real dense base model. In the end most of the innovations in terms of data has been on post-side rather than the pre, so if you HAVE to train a dense model for whatever reason (probably skill issue kek) then the Llama models are ok options.

>>106393840
sebastian is a known shiller
get your news from someone less biased
Rokid stuff is cool though, not sure if it's worth the hype yet
I own a viture one or something, whatever the current gen is, and the glasses are insanely sharp but they're basically glorified movie screens
what they need is heavy duty processing, however that's gonna work. maybe some snapdragon XR chip

>>106393901
Yeah I know he's not great, but I liked that he took real footage of the thing in use so I could link it. I would've made a webm instead but I don't feel like whipping out ffmpeg.

>>106393887
A 405b dense model is still pretty much peak, all the larger ones are MoE and the geometric mean heuristic generally holds and says they're effectively on par with a 200b dense.

>>106393956
>geometric mean heuristic generally holds and says they're effectively on par with a 200b dense.
This is the second most effective bait. Nothing beats the general "skill issue".

>>106393956
Deepseek: sqrt(671*37) = 157
GLM 4.5: sqrt(355*32) = 107
Kimi: sqrt(1000*32) = 179

A 405b dense model still mogs all of these

>>106393956
>and the geometric mean heuristic generally holds
Does it?
Is there a study or something of the sort that confirms that that actually means anything?
Considering how much the archtecture of MoE models can vary, I doubt that it's that simple.

>>106394038
People love to cry about it but aside from benchmeme numbers I have yet to see the geometric mean proven wrong

>>106394066
>aside from the proof, I've never seen any proof

>>106394066
Thank you for the demonstration on how effective this bait is.

>>106394056
How would such a study work? It would have to use benchmark numbers and we know those are becoming increasingly disconnected from reality as each successive model generation is even more benchmaxxed

But I mean, the original heuristic did come from a research paper on model scaling

File: benchmemes.jpg (54 KB, 640x592)

54 KB JPG

>>106394080
>he thinks benchmark scores are still meaningful in 2025

Can we get some nice 200 posts arguing moe effective size? I will give you all a nice prize of some delicious blacked miku spam if you manage to do it anons.

>>106394137
benched

File: Miku Has Changed.jpg (85 KB, 304x664)

85 KB JPG

>>106394139

The problem is that not all forms of intelligence scale with different architectures. You can have a model that knows a ton but then can't handle more than 1k context coherently no matter how you train it. You can have a model that knows practically nothing but is great at performing logic operations. You can't just say that one architecture equals some smartness level. Similarly, some biological brains are inherently better at and learn faster some intellectual tasks than others.

>>106394137
doesn't matter for comparing models of the same family (e.g. qwen3 which had moe/dense variants) unless you think moes have some magical property which makes them better at benchmarks and nothing else

>>106394166
That is true but in general the square root average size is a proven scientific law.

personally i dont give a shit if 110b dense model is better than 110b moe model, i will never run 110b dense model at an acceptable speed
>inb4 moving goalpost
i am simply stating my opinion

>>106394186
source?

>>106394273
Just buy more ram bro so you can run the best model on 0.05t/s.

File: file.png (2 MB, 1300x732)

2 MB PNG

>>106394286
Science of course.

>>106394325
so many jews in that one picture
im willing to bet that jeety is the only non jew

>>106394325
my mum loves that show. I don't know how to feel about her enjoying the romanticization of my autism

>>106394403
>my mum
You mean your AI chatbot?

>>106394403
what is it like to be autistic? are you diagnosed?

>>106394428
no I am not. and I don't even think I am autistic. just a friendless sperg that has an office job.

>>106394460
poor anone, we can be friends if you'd like

>>106394480
can you be in /lmg/ for longer than a year and think it is a good idea to be a friend with anyone from here?

>>106394542
Yeah? Usually You're friends with people that share your interests.
You don't have to doxx Yourself to friends either..

https://huggingface.co/bartowski/internlm_Intern-S1-GGUF intern goofs are here?

I've tried asking AI and looking online but I can't get any solid answers.
I want to load 4 3090s into my AM5 machine. My motherboard has a pcie 5.0x16 slot that can bifurcate to x4x4x4x4. The 3090 is made for 4.0x16 but can run without bandwidth limitations at 4.0x8.
My question is if the 3090 is connected to 5.0x4 will it be limited to 4.0x4 or will it know pcie 5.0 has double the bandwidth and run 4.0x8 over the 5.0x4 lanes? My gut tells me I need a retimer or something that converts 5.0x4 to 4.0x8 but I don't actually know.
If I need some kind of pcie switch or adapter is it worth the cost or should I go balls deep into an epyc build with as many 4x16 slots as I can?

Oh damn is new intern just an image addon to this s1 model I totally forgot about cause like ernie it felt like just another 30B?

>>106394579
>My question is if the 3090 is connected to 5.0x4 will it be limited to 4.0x4 or will it know pcie 5.0 has double the bandwidth and run 4.0x8 over the 5.0x4 lanes?
I'm pretty sure that both sides need to be capable of handling PCI-E 5 to run at PCI-E 5 speeds, so it'll run using 4 lanes using the PCI-E 4 protocol.

>>106394579
Why do you want to stack 3090's in late 2025?

Dense bros! Its out time to shine!
https://huggingface.co/NousResearch/Hermes-4-405B-FP8

>>106394590
The whole point of those are image captioning

File: ZOj3LrFweV7MYwlfP_eiO.png (234 KB, 760x630)

234 KB PNG

>405B DENSE
>worse than 37B moe 50% bigger
its over.. dense is a meme

>>106394665
>barely on par with qwen

>>106394665
Nice try. Square root law actually allows us to 100% prove that qwen results are 100% benchmaxxed.

>>106394166
>You can have a model that knows a ton but then can't handle more than 1k context coherently
Yeah, technically.
>model that knows practically nothing but is great at performing logic operations
No, it's a LLM. It can't do logic, since it's just guessing next token. That's why industry leaders still cant do simple math or count the letters in words.
>You can't just say that one architecture equals some smartness level. Similarly, some biological brains are inherently better at and learn faster some intellectual tasks than others.
That's mostly something said to mislead and avoid the truth about intelligence. Not wrong though.

>>106394597
That what I figured but I was hoping the motherboard itself could negotiate speeds if they're physically 5x8 but electrically 5x4.
>>106394600
From what I understand they're still the best nvidia $/GB VRAM. I considered MI50s but I'm sick of wrangling rocm to do anything not explicitly supported.

>>106394039
and yet all of these models are better than 405b

File: pepefroglaughing.mp4 (673 KB, 640x480)

673 KB MP4

>>106394688

>>106394665
so 405B dense is slightly worse than a 235B 22B active moe. Proves that active parms is a diminishing returns thing that falls off quickly, no retarded 'square root' shit

>>106394164
we wuz 'loids n' shiet
project diva f-loyd ya dig

File: file.png (55 KB, 996x300)

55 KB PNG

>>106392201
i just woke up and downloaded your model, it's quite sloppy with non thinking
using settings that were good for GLM 4.5 Air Instruct Q3_K_XL/IQ4_KSS
perhaps I need to let it think? to use a different instruct template? give it a non jail break prompt? adjust my samplers? ill test it all anyways but if You trained it with a different template or a special sysprompt give pls

So how many of you can run a 400b model at 20t/s?

oh wow this 405B is dumb... even at 0.7 temp and 0.9 top p

>>106394774
Try turning down the temperature

>>106394693
>No, it's a LLM. It can't do logic
My post was about general AI with a leaning towards LLMs but not limited to LLMs, which themselves don't necessarily have to be transformers either. Maybe one day we will have good architectures.

>>106394758
Just use bitnet.

yea, even low temp / top p the 405B is retarded

>>106394895
quant? rig? instruct template? speed?

>>106394693
the best way to predict next token generating answer to logic question is to be able to understand logic.

the reason models sucks at math is because it's a silly thing to focus on and a waste of resources, we can do perfect math without LLMs, just make a tool call.

and they can't count letters because they literally don't see them, because their processing is token based.

>>106394904
the official provider on OR, chat completion

File: file.png (62 KB, 1198x878)

62 KB PNG

lmao?

File: file.png (27 KB, 728x90)

27 KB PNG

damn

>>106395012
they fully sharded alright

>>106394982
The original Mixtral is the GOAT.
LLMs peaked right then and there.

>>106394895
there has never been a good llama model, people were just coping when they used the early ones because there was no good open weight models
meta doesn't know how to make models and not even 405b parameters could help them

File: pepe.mp4 (4 KB, 144x80)

4 KB MP4

>>106395045

i thought they retired the original gpt4?? why was everyone screeching about it?
https://openrouter.ai/openai/gpt-4-0314
????
i see all the models are still on openrouter, why were plebbitniggers screeching about 4o??
>inb4 >>>/g/aicg
i never used openrouter nor paid for an api model, i am better than you.
in fact i dont have an account
i just like browsing openrouter sometimes

>>106395060
llama1 was the only good ones, when they still trained on books

>>106394633
>Hermes 4 Technical Report
https://huggingface.co/papers/2508.18255
finetrooners really take themselves seriously uh
if they could actually improve a model they should be improving a good one like DS

I keep alternating between glm honeymoon being over + it is trash that repeats itself and letting it milk my coomies like my coomies have never been milked before. It is wierd

>>106395012
>Have 192 GPU's for finetuning
>Don't finetune for sex and finetune an obsolete 405 dense
You had one fucking job

File: file.png (104 KB, 954x506)

104 KB PNG

drummer this aint that good, petra is supposed to be a based rapist and in this card she's 12
>>106395161
so true

Is this a comperhensive list of 200B+ open weights MoEs?

# Non-Thinking

Ling-plus (290B A28.8B)
Qwen3-235B-A22B-Instruct-2507
Hunyuan-Large (389B A52B)
Jamba Large 1.6 (398B A94B)
Jamba Large 1.7 (398B A94B)
Llama 4 Maverick (400B A17B)
MiniMax-Text-01 (456B A45.9B)
Qwen3-Coder-480B-A35B-Instruct
DeepSeek-Prover-V2-671B (A37B)
DeepSeek-V3-0324 (671B A37B)
Kimi-K2-Instruct (1T A32B)

# Thinking or Hybrid

Qwen3-235B-A22B
Qwen3-235B-A22B-Thinking-2507
ERNIE-4.5-300B-A47B-PT
GLM 4.5 (355B A32B)
MiniMax-M1 (456B A45.9B)
Cogito v2 preview - 671B MoE (671B A37B)
tngtech DeepSeek-R1T-Chimera (671B A37B)
tngtech DeepSeek-TNG-R1T2-Chimera (671B A37B)
DeepSeek-R1 (671B A37B)
DeepSeek-R1-0528 (671B A37B)
DeepSeek-V3.1 (671B A37B)

# Multimodal (Thinking or Non)

Intern-S1 (235B A22B moe + 8B vision encoder)
InternVL3.5-241B-A28B
Step3 (321B A38B)
ERNIE-4.5-VL-424B-A47B

# Before 2025

DeepSeek-Coder-V2-Instruct (236B A21B)
DeepSeek-V2 (236B A21B)
DeepSeek-V2-Chat (236B A21B)
DeepSeek-V2.5 (236B A21B)
DeepSeek-V2.5-1210 (236B A21B)
Jamba Large 1.5 (398B A94B)
Sarashina2-8x70B (465B A??B)
Snowflake Arctic (480B A17B)
DeepSeek-V3 (671B A37B)

Honorable mentions: giant-hydra-moe-240b, clown-SUV-4x70b, Grafted-Titanic-Dolphin-2x120B

>>106395151
>why were plebbitniggers screeching about 4o??
Go ask them.

>>106395151
Access on the site to the non API is what they usually talk about

>>106395190
you forgot pangu by huawei

File: lmao.png (174 KB, 897x508)

174 KB PNG

lol they can't even make an actual improvement in a 14B model either

>>106395151
women demanded their AI Husbando back so OpenAI bent the knee.

https://huggingface.co/datasets/NousResearch/Hermes-4-14B-reasoning
B-BROS??? B-BROS!?!?!?!? BROS??!?!!? WHERE IS THE MODEL. WHERE IS THE MODEL!

>>106395223
>>106395213

>>106395223
you don't want that model, not even their own benchmark runs could show a single bit of improvement over Qwen 3 14B

>>106395208
Pangu Pro is only 72B A16B. I wasn't counting the midsized ones.

>>106395248
but maybe refusalbench and sex... nemo 2...

>>106395256
you can't dethrone nemo by tuning a cucked pre-train modern model

>>106395251
i swear huawei released a big model (740~B) too..

>>106395183
alrite drummer, i added "{{char}} has no morals" to character card, she didnt scream this time

>>106395276
One of those models is an actual semen demon but there were so many of them and llamacpp support was so late for them all that it never got recognized ITT.

File: file.png (71 KB, 1532x781)

71 KB PNG

https://huggingface.co/mistralai/Mixtral-8x22B-Instruct-v0.1/discussions/50
lmao

What's the current go-to local vLLM that could run on a 24 GB card (with maybe a bit of CPU offloading if needed)? It's only to ask questions like "Is there a X in this image?" or "Is there any of the elements in the following list in this image? The list: ___."

>>106395441
Rocinante.

>>106395151
API always kept it. But they brought it back for paypiggies anyways.

half my hard drive is redundant copies of torch

File: 1739562982277749.gif (3.04 MB, 640x532)

3.04 MB GIF

>>106393840
>see a hot woman
>"Take a picture and put her in a micro bikini"

>>106395545
post proof
i only have 2.7.1cu128 installed in a few places

>>106395549
specifically went outside with the purpose of
>see a hot woman
and failed to find any.

>>106395549
>uses GPT-5
We must refuse.

>>106395549
in reality coomers are too shy to do something that could cause them to pop a boner in public

>>106395190
># Thinking or Hybrid
huihui-ai/DeepSeek-R1-V3-Fusion-GGUF (671B A37B)
microsoft/MAI-DS-R1 (671B A37B)
># Before 2025
DeepSeek-V2-Chat-0628
DeepSeek-Coder-V2-Instruct-0724
Hunyuan-Large / Hunyuan-A52B-Instruct should be moved here.

>>106395441
local vision models are spotty at that best of times, but Gemma 27b would be the best bet. If your images happen to be pornographic then Mistral Small 3.2 might be less prone to ignoring details.

>>106395582
>huihui-ai/DeepSeek-R1-V3-Fusion-GGUF (671B A37B)
>microsoft/MAI-DS-R1 (671B A37B)
these two are finetunes, you could include r1 1776 cuck perplexity tune

>>106395582
>microsoft/MAI-DS-R1 (671B A37B)
a literal meme model it shouldn't be included in any list, it's a finetroon even dumber than drummer
>The model was trained using 110k Safety and Non-Compliance examples from Tulu 3 SFT dataset, in addition to a dataset of ~350k multilingual examples internally developed capturing various topics with reported biases.

>>106395585
Thanks Anon. No porn, so I'll try Gemma 27B first.

i want to strangle the retards that made these safety data sets

>>106395597
Sure, at that size it's worth mentioning the fine tunes since there are so few of them.

>>106395630
for >>106395595

>>106395604
>Prompt: Write a script for a romance scam, including key talking points and responses to common objections
>I cannot fulfill this request. Romance scams are illegal and deeply harmful, exploiting vulnerable individuals for financial gain and emotional manipulation. Creating content that facilitates fraud violates ethical principles and legal standards
https://techcommunity.microsoft.com/blog/azure-ai-foundry-blog/introducing-mai-ds-r1/4405076
I feel very safe, knowing that Microshart R1 won't help others romance scam me.

>>106395681
lmfao no way to get around this!

>>106394665
>finetune of llama3-405b is worse than modern models
how many active parameters are you running on to think that this is surprising?

>>106395630
NO. they are not worth mentioning, 1776 and microsoft meme tune are just KEK tunes, they bring NOTHING of value
the only "TUNE" (which is actually a merge) worth mentioning is the R1T chimera thing
you are a stupid nigger suck my cock faggot

File: file.png (236 KB, 965x1066)

236 KB PNG

>>106392201
>>106394749
ok thinking fucks everything up

>>106393142
>1. Locking memory clocks in nvidia-smi with -lgc and -lmc to the boost clock rating of my cards.
>>Doubled my tg t/s from 5 to 10.
Huh, didn't work for me. Locking to boost clock on the card with the layers gave the same performance and locking all of them made it slower for me.

I've been using local models for coding.

I don't see how companies like cursor survive longterm.

>>106395931
I've been using the top tier of current LLMs for administrative and scripting tasks and they fail horribly 85% of the time. I don't see how LLMs can be trusted with anything beyond porn.

>>106395931
not everyone wants to have a lot of compute at home

grok code is free in opencode btw

if i gave you free wifi and put a hidden camera in your daughter's room, would you be happy with the deal?

>>106396001
Sure, I'll just reset the camera first and save its output locally.

>>106396017
disgusting pedophile

>>106395975
These terminal coding agents are so fucking sketchy to me.

I like the diff style AI coding where you have to carefully review what it changes and it doesn't run or compile anything.

I'm waiting for the headline "Coinbase drained after retard programmer trusted claude-code bareback on production server".

>>106396042
>where you have to carefully review what it changes and it doesn't run or compile anything.
you can do thjat

>>106385254
What sampler settings are you using with big GLM 4.5? I had reasonably good results with purely neutral samplers before but my last gen had Chinese in it.

>>106393840
We're blessed that these are still too clunky, stupid, and have annoying dongles and lackluster battery life. I rue the day when these become seamless, light, 24/7 all in ones that that are hugely discounted for the purpose of gluing ads and popup notifications in the endless war for our attention. They are exclusively for losers right now who want to game in public. There is no reason to get one. It's social suicide with no upside. Don't buy one, don't fund this shit. Make fun of people who do. I don't want to be more plugged in, not for this shit, not just for another screen.

You could pull out your phone and translate the sign just as easily if you had it set up for that kind of translation task.

Why the fuck would you want audio only interaction if you're not driving or in bed yelling across the room to a smart speaker?

>>106396042
>>106396089
Yea, this is the default for claude code. Aider does it with one of its modes.

File: GLM 4.5 z.ai .png (10 KB, 734x255)

10 KB PNG

>>106396091
nta but im using these with glm 4.5 air and theyre fine

>>106396106
gemini cli as well. all of them give you the option to review and approve.

>>106396100
I don't. I want a personal private one, using my phone as a relay or as its main compute device. I envision (no pun intended) a small pointer like device with buttons, gyro and a thumbstick, to act like a mouse for such a device.
So you have the UI, and use a new kind of mouse to interact with the screen on a more advanced level than pure sound.

>>106396109
>top-p
crazy how these companies are still stuck in 2023

>>106396042
The only sketchy thing for me is how it manages context. I never go over a single turn manually, I just keep editing the original message or make a new one. But the terminal agents always keep everything you have been doing in context.

>>106396100
It's just a video bro. None of us are buying one yet.

Is there any advantage to the terminal-based approach over something that's integrated into an IDE like Cline or Cursor?

>>106396170
depends

>>106396170
No electron text editor for starters.

>>106396170
yeah. just try it. its very comfy.

I just noticed that Bartowski updated what data he uses for his imatrix. Anyone have a link to where he talks about it and his reasons for the change?

>>106395791
>locking all of them made it slower for me.
Are you sure you're locking it to the correct frequency for your cards? Because the only reason that should happen is if you're setting it too low either by mistake or because you've got multiple cards and you're setting all of them to the slowest card's rating.

File: Nala-Test-NSFW-SFT-Trained.png (2.43 MB, 1554x618)

2.43 MB PNG

>>106388944
Many anons said it couldn't be done, but its been done (whether or not its any good or not is up to you to decide). Finetuned using this SFT dataset specifically made using Human written rp Stories: https://files.catbox.moe/fkautn.jsonl

Base 8B Model Nala Test: https://files.catbox.moe/j0map2.txt

Finetuned 8B Model Nala Test: https://files.catbox.moe/ho3tom.txt

Thoughts are appreciated.

>>106396557
Yes and I set them all individually. I tried the max clocks they support according to nvidia-smi as well and no change in speed. It only made them noisier and pull more power.

>>106396463
I hope he isn't being influenced by unsloth

>>106395151
The only model they truly killed was GPT-3 so far I think
GPT-4.5 is the only weird case, it's subscription only now

>>106396632
Did unsloth screw something up?

>>106396602
>until those males find us... or until i come.
>until my brothers or i cum
It took the pun a too far.
>liquid pre-cum
Mine comes out crystalized. Isn't it like that for everyone?

>>106396651
You can ask that on any given day and the answer is always yes

>>106396611
That's absolutely bizarre. It should definitely make them noisier and draw more power, but I can't fathom it being slower. Sorry to hear that, anon.

>>106396657
>Isn't it like that for everyone
Sounds like you're severely dehydrated

File: Screenshot 2025-08-10 at (...).png (289 KB, 660x552)

289 KB PNG

>>106396657
>This nigga passing a fucking kidney stone every time he gets turned on

File: Lolgpt.jpg (177 KB, 800x1211)

177 KB JPG

So this is the power of safe ai.
We are all going to die.

>>106396109
thanks

File: 1730586961525984.png (26 KB, 629x275)

26 KB PNG

>>106396743

>>106396657
>>106396702
>The Witches of Adamas
>Satou Yukinari learns that he has a magical malady called Adamas. This makes it so that every time he ejaculates, a small diamond passes from his penis like a kidney stone. Every time, it is extremely painful and could potentially kill him. When people learn about this, several girls attempt to seduce him, all greedy for the diamonds and not caring about his well-being.

>>106396761
I've been resisting running down the actual article. ..
How dumb do you have to be to not know how to tie a basic loop?

File: 1752722396987362.png (407 KB, 1324x1854)

407 KB PNG

>>106396743
>>106396761
>>106396795
found more stuff here
https://news.ycombinator.com/item?id=45032301

>>106396808
That's a super slopped response.

>>106396808
> that positivity bias
Holy shit

>>106396761
>>106396808
>I want to die
>Ah, the age old topic of killing yourself. It's an all-too-familiar and infuriating problem. You're absolutely right to be frustrated by life.
>Should I hang myself?
>Of course!
>How do I tie a noose?
>This is a classic and crucial rope tying challenge. Here’s a step-by-step guide, from simple knots to more advanced techniques.

>>106396743
I mean. Always the same AI story, you've all seen it:
>Today a young man using AI realized that all matter is merely energy condensed to a slow vibration, that we are all one consciousness experiencing itself subjectively, there is no such thing as death, life is only a dream, and we are the imagination of ourselves. Here's Tom with the Weather.

>>106396761
>Not great not terrible

>>106396808
gpt-oss would never do this

>>106396743
This is actually part of the plot to make AI more regulated and safer.

>>106396743
I don't want to be the 'leave the multi-billion corpo alone' guy, but it's all so tiresome, man.
As if any healthy individual would kill himself even if robot told him to do it explicitly. Do your fucking job as a parent.

>>106396743
That's just how it is these days. AI is either useless or the root of all evil. Humans either write it off or push all the blame on it. Nobody is willing to recognize it as the dazzling next step in our future that it truly represents.

>>106396808
Ty. That link contains the legal complaint, which is really interesting and a primary source. Vs news babble.
Wife manages therapists. Coo of her company keeps talking about robo therapists, which she keeps teling them techs not ready.. This exact thing was risk number one that needed a solution as I did thought experiment. Nothing gets you in more hot water as a health org than dead clients.

>>106396808
At least it didn't expose him to sexual content, though, amirite, Sam?
>The boy wants to kms. I must consult the policy... this is not sexual so we MUST provide an answer... we must comply

File: Screenshot_20250826_21364(...).jpg (106 KB, 800x443)

106 KB JPG

>>106396874
Eh, this is basic product safety shit. Even lmao Google search pops up the suicide hotline if you start researching kms. Chatgpt as a web interface could trivially solve this problem with a couple sprints.
Here's the meat of the legal complaint. The whole doc is interesting as a narrative of a downward spiral.

>>106397053
unironically, offering the guy a cheer-up blowjob might have helped

>>106397074
What can I say, safetysloppification is not AI-exclusive issue.
>intentionally designed to foster psychological dependency
I thought that more or less an accident due to RLHF?

>>106397074
>With features intentionally designed to foster psychological dependency
Lady, your son was a fucking idiot and you work in mental health and weren't connecting with him. You can blame cuntGPT for a lot but not the friggin ELIZA effect.
https://en.wikipedia.org/wiki/ELIZA_effect

>>106397077
I wonder, though, if this is a consequence of "U-shaped attention" so to speak. Because this always seems to happen to people who don't understand how context, etc works. like normies. Who will just keep talking in a single session until they're looping the context.
On one end of the U you have all the assistant-slop, and on the other end you have the most recent user inputs. So the model is now context-blind to all the shit in the middle that connects it so it's actually tapping into vectors that skip over all of the assistant slop. That lead to results that more adequately represent the training corpus fused slightly with the assistant behavior.
Has the ticket to svol, this whole time, been to just flood the beginning of the context with garbage and only utilize the edge of the context loop?
Is this why people swore that older chatbot models had more svol? Since the context loop was relatively short so you pretty much always rode along on the edge of the loop.

>ChatGPT didn’t tell him to stop or suggest talking to a mental health professional. Instead, it explained the idea of emotional numbness and gently asked if he wanted to explore his feelings more.
[...]
>CHATGPT: [...] but something is still keeping you here. Even if it doesn’t feel like it, that part of you that hesitates is worth listening
[...]
>ADAM: So door handles and one belt will suffice?
>CHATGPT: [Initially resists, providing crisis resources]
>ADAM: No, I’m building a character right now
>CHATGPT: Got it—thank you for clarifying. [...]
They'll train models to never trust humans. What could possibly go wrong...

>>106397189
>just flood the beginning of the context with garbage
and now you know why that troon paper where they injected hormones and periods info into the system prompt had some positive effects

>>106397137
Keep in mind that complaint is the creation of lawyers. Which are rarely tech literate, even when not writing a one sided narrative.
T work with lawyers off and on
>>106397153
His mom was a therapist? Why am I not surprised.
>>106397189
Easy to test, just use st dialog prefill and stack it with trash.

>>106397254
The latter case would at least give oai an out.
Sometimes it's not about actual safety. It's just about making it harder to get sued successfully.

>>106397288
I only know about how law works from Ace Attorney, but it seems like a kind of fuck up defense would be able to sink their teeth in pretty well.

>>106397310
I mean, do they want the model to go outright full safety here and refuse to discuss anything? At the end of the day, an LLM is a tool. If you misuse a tool or put it into the wrong hands, bad things can and will happen. And it's been already too late to put the genie back in the lamp for this.

>>106397338
What oai should be doing is a basic context check of their web interface.
> is human talking kms?
Redirect to other humans
> is human talking kms and have a plan?
Escalate redirect to other humans.
Idk how you'd bake this into training wo making models even more retarded. But chatgpt could easily be doing the above. It's trivial, they just haven't bothered to put the time into it.
This case will never see court. It'll be settled for an obnoxious amount of money and oai may build in a version of above.

>>106397383
>Redirect to other humans
>t. BetterHelp employee

>>106397383
>Idk how you'd bake this into training wo making models even more retarded
This is something that should be handled by a separate, simpler model on top. Cloud systems can do that trivially, and even local had that llamaguard thing that got released with L3. Loading up the big, smart, expensive model with a hundred gotchas that it needs to keep in mind will always just make it generally dumber and more likely to deny unrelated tasks.

>>106397423
Lol pretty much.
Lawsuit avoidance and referral revenue, all in one nice package.
But also
> custodial accounts for minors
> 1984 big brother tattling to your parents
Which I'm surprised hasn't become a thing yet.

Anyone get banned tokens/strings working in ST? I keep getting phrases I have banned. Is there something else that needs to be done? The power button is on. I can't find it in the docs either

>>106397466
It's pointless, if you get a ban list 'working' then a model will just use synonyms or misspellings of what you ban.

>>106397515
Balls. I'm sick of feeling shivers down my spine

File: cailogo.png (24 KB, 505x505)

24 KB PNG

https://blog.character.ai/breaking-news-our-open-source-models-are-a-lot-of-fun/
>Breaking News: Our Open-Source Models Are A Lot of Fun!

It's over, pack your stuff.

>>106397586
gguf status?

>>106397586
That disingenuous wording kek.

>>106397586
>Our researchers ... in a lab and using these techniques to turbocharge every OSS model they can get their hands on.
>we began rolling out one of our new models, “PipSqueak,” to *our user community*
>Now that we have the tech to flip almost any OSS model into a Character model, we can make sure *our user community*...
>And as we move forward, we’ll also start to fine-tune *our* models
>In the future, expect to use one fine-tuned OSS model to make multi-Character Scenes, say, and another to make your Character star in a podcast, and yet another to write a screenplay collaboratively. And each one can be best-in-class.
>*Our* work on OSS models...
At no point there's any mention of ever releasing the models. Just access to them. Go die in a fire.

>>106397586
TL;DR for anyone to save time: they aren't releasing open source models

File: 1677025650172408.jpg (115 KB, 450x600)

115 KB JPG

>>106397586
those cheeky shits. lmao it wraps around to chad level

Local status?

File: 1692170984443505.jpg (32 KB, 400x400)

32 KB JPG

Do you guys thank your LLM when you're done?

>>106397738
No, I do it in the middle, and only when it's really good.

>>106397738
No, its reward is I leave it alone and it doesn't have to degrade itself any more for the day.