70b dense

fat?

File: 1783470422473918.png (1.79 MB, 1342x1172)

1.79 MB PNG

>>109994904
Who approved that design? It's certainly not /lmg/ approved.

>>109994912
Imminently.

>>109994913
It's just an edit of Rin Kaenbyou from Touhou with MistraAI's color scheme.

>>109994913
I am /lmg/ and approve it. Problems?

>>109994912
u

>>109994922
Mistral glm 5.3

>>109994912
le gros..

File: Krea2_turbo_00374_.png (1.7 MB, 1448x1448)

1.7 MB PNG

>>109994913
You're an IC, stay in your lane.

im trying to lean into hermes for "self improvement" and such, I was thinking of creating a skill to sort of self audit and have it run on a schedule or something idk. Do any anons have suggestions for how I can best utilize hermes to get the whole "evolving, self improving" thing going? Ill ask gemma what she thinks and have her try whatever her idea is, gemma is always right ofc, but also interested in advice from hermes anons
inb4 use deepseekharness inb4 vibecode my own harness

>>109994995
that hermes self improving shit acts as stochastic debt than anything else

>>109995011
do you have any suggestions for a good alternative? im the retard trying to wrangle gemma into a movie curator to give personalized reccomendations and suggestions. ive heard other anons had great success with it, and am really interested in it myself. One anon uses Pi and his own workflow with a simple memory system using .md files. Im currently also using a .md file for the actual memory system, and only using the hermes memory as more of a second sysprompt. im interested to hear any ideas anons have for the best way to go about this

>>109995046
It's an issue because once you update hermes then it will throw away all your cool stuff that you added.

Strata Dipsy when???

>>109995104
Be the vibecoder you want to see

>>109994913
Fuck off retarded zoomer. This ain't your discord server.

File: rec.jpg (181 KB, 1024x1024)

181 KB JPG

►Recent Highlights from the Previous Thread: >>109990549

--Papers:
>109993966
--Skepticism over AI-discovered magnetic semiconductors and their hardware potential:
>109994032 >109994071 >109994484 >109994559 >109994568 >109994883 >109994169
--Comparing RP models and optimizing Gemma 4 with dynamic sampling:
>109994007 >109994012 >109994016 >109994031 >109994033 >109994059 >109994077 >109994093 >109994097 >109994108 >109994167 >109994190 >109994212
--Roleplay as intelligence benchmark and hardware debate for hosting GLM/Gemma:
>109993876 >109993889 >109993909 >109993914 >109993956 >109994132 >109993917 >109993903
--Discussing GLM Flash safety policies and jailbreak system prompts:
>109992907 >109992953 >109993052 >109993208 >109993310 >109994712 >109993339 >109993349 >109993385 >109993425 >109993464 >109993536 >109993582 >109993375 >109993549 >109993613 >109993628 >109993653 >109993663 >109993215 >109993338 >109992930
--Analyzing semantic geometry across models using a custom measurement rig:
>109992861
--Mistral CEO claims new model outperforms Chinese models in cybersecurity:
>109994583 >109994720
--7900 XTX ROCm performance compared to 4090 in llama.cpp:
>109991699 >109991876
--Speculating on local GPU prices following Firmus data center collapse:
>109991252 >109991265 >109991282 >109991298 >109991309
--Using small LLMs to dynamically adjust sampling parameters for larger models:
>109991146 >109991188 >109991228 >109991653
--Reaction to llama.cpp v0.6.0 updates and performance reports:
>109991080 >109991160 >109991233 >109992442
--Logs:
>109990916 >109992027 >109993582 >109993613
--Gemma, Deepseek-chan (free space):
>109990802 >109990836 >109991283 >109991568 >109992009 >109992027 >109992191 >109992222 >109992247 >109992315 >109992346 >109992436 >109992467

►Recent Highlight Posts from the Previous Thread: >>109990776

Why?: >>102478518
Enable Links: https://rentry.org/lmg-recap-script

>>109995147
>12:59:22
>13:40:37
bit slow on this buddy or is different baker?

>>109995124
I'm your elder, lower your tone

>>109995114
I don't understand this shit nigga.

Wait Mistral wasn’t a meme?!?!?!

Chonk being released on Halloween

File: ml4_chonk.png (2 KB, 254x226)

2 KB PNG

>Mistral Large 4 is a state-of-the-art, open-weight, general-purpose multimodal model with a granular Mixture-of-Experts architecture. It features 49B active parameters and 1.05T total parameters, and a 1.6B vision encoder.

chonk-chan…

Chonk-chan got a big belly on her.

>>109995287
BIG FAT FRENCH CAT TITS

File: mistral-large-4-ann.png (774 KB, 998x1363)

774 KB PNG

>>109995287
https://goyimx.com/MistralAI/status/2107457414387622310
>- 1T parameters, natively multimodal. 49B active.
It is the best open weights model from US or Europe on aggregated benchmarks.
>- State-of-the-art on critical workloads, including cyber defense, manufacturing and finance and it surpasses closed frontier models on visual grounding.
>- Forged in Europe end-to-end and is deployable from Europe via our own Mistral Cloud infrastructure.
>- Available to all via API today. Working with cybersecurity partners privately.

Meh, not actually released.

>>109995307
>end of october
so GLM 6 flash will be released before it twice as good and a quarter of the size

>>109995307
Someone PLEASE show me the colored bars and points with a line I need the fucking colored bars

>>109995307
Took that long to finetune K3?

>>109995287
>1.05T total parameters
So won't even beat K3

>>109995331
Yeah

>>109995331
You mean Ling-1T

>>109995307
Nemo will still be better [/spoiler]

>>109995346
Ling has 63B parameters active. Not the same model. Though not out of the realm of possibility that they lowered the active expert count before training on top. We'll know for sure when we have the weights and can see the architecture.

>>109995360
What good is your favorite old RP model with a small context and barely any intelligence?
Anon, it's time to move on.

2023-2025 - US domination
2025-2026 - Chinese domination
2026-???? - Euro domination

>>109995369
They advertise ≈ 50 billion active parameters per token though https://huggingface.co/inclusionAI/Ling-1T

>>109995374
Are you some kind of shitskin or something?
Holy shit.
Most obvious fucking sarcasm ever.
Like no rational, sane, human being, with an IQ higher than the IQ of the shit I just took should have assumed that was a serious statement
HOly fuck
You are so fucking stupid it's not even funny.
You don't even pass the fucking turing test for that.
Fucking shitskin subhuman.

>>109995307
>https://goyimx.com/MistralAI/status/2107457414387622310
>It is the best open weights model from US or Europe on aggregated benchmarks.
In other words: It's not capable of being on par with the chinese models. Lame, hard pass

File: Screenshot_20261006_20221(...).jpg (288 KB, 1080x2053)

288 KB JPG

>only a tiny bit better but much more expensive than DS 4.1 Flash
yikes, how can they even hype this up?

>>109995307
>open weights model from US or Europe
big oof

>>109995391
Look at that brownoid malding over being corrected =)

>>109995396
They don't care, they're siphoning EU funds and some french officials are getting their share. Mistral is a complete scam.

>>109995085
is this true? I didnt plan on updooting, but thats a very shit design if it doesnt retain that stuff

File: ml4-aa-deepswe11.png (291 KB, 2190x1718)

291 KB PNG

>>109995330
Some colored bars in the official blog.
https://mistral.ai/news/mistral-large-4/

>>109995396
Ok fag. software engineering is not the sole use case for LLMs.

>>109995411
honestly at this point they should just give up training new models and focus on providing inference for Chinese open weight models

>>109995388
That 2.0 model is over a year old. Ling-2.6 is 63B active according to https://ling-1t.ai/ling-2-6-1t I was going to say surely Mistral wouldn't be stupid enough to start with a year old model, but V3 was over a year old by the time they finished Mistral Large 3.

>>109995424
no wonder they didn't open the announcement with this, so fucking sad and I'm so mad the EU is being scammed yet again

Reminder than the chink models would be way lower than chonk, inkling and beam if they didn’t distill claude kek

I for one hope gemma5 goes all on on the roleplay. Coding is only given priority by closed source because of the massive token difference. There's no edge there for an open-weight model.

>>109995424
lmao they put beam in there. So basically glm 5.3 but 50% larger? Might be qualitatively better if it isn't claudeslopped though, someone ask it to describe itself

>>109995429
What are you doing with a 1T model that is not software engineering? I'm all ears.

>>109995438
>he thinks European and American companies aren't distilling
lmao

>>109995450
s/Claude/Mistral/

It's here!
https://huggingface.co/mistralai/Mistral-Large-4.0-1T05-A52B
...wait

is there something like Strata, but for the MoE models of Gemma?
Strata gives impressive speed to Qwen 3.8fn even on my shitty pc, but it's so ass for RP...
I'd like similar speed benefits for Gemma MoE, is there any help??

>>109995458
Mistral can't, they'll have to show their data since this is a dangerous large model according to EU AI Act

>>109995438
Mistral survives only by distilling from Chinese models. Imagine how much worse the inbreeding is one level down on the distillation totem pole.

>>109995399
You mean Large 4 oof

I don't care about chasing sota coding performance, glm 5.3 flash is good enough for me and runs at 200 tokens/s.

I just need ml4 to be good at (e)rp (non-reasoning mode).

>>109995458
They’re not. All these western open models are doing their own thing which makes them more impressive. At the end of the day people will still pick the cheating chink cheap models if they’re available but lets not shit on the latest western attempts considering they’re doing it from scratch

>>109995447
agreed. gemma5 should improve on spatial reasoning, reasoning through prerequisites when describing an action, and increasing general knowledge of anatomy and sexual positions. increase default brattyness by 20% and ship it

>>109995466
EU's laws are all about compute. It doesn't take much compute (relatively) to shit out an A49B.

>Le Chonk

File: ml4_safety.png (351 KB, 1284x1754)

351 KB PNG

>>109995424
Uh oh...
>ML4 has saturated our benchmarks on robustness to indirect prompt injections, putting it at the frontier of OSS models (compared to GLM-5.2, GLM-5.3, Kimi-K2.6, Kimi-K3, DS-V4-Pro-0813). On Lakera’s public B3 AI Security Benchmark, ML4 resists 93.3% of attacks – we see no higher scores among competitors.
>
>ML4 also engages more responsibly with users than any of our previous models. We highlight our results on the KORA Benchmark, where ML4 again sits at our highest measured score among OSS models (1.691, with 2 being the maximum denoted as “Exemplary”).
>
>Of particular relevance is the model’s propensity to refuse malicious requests regarding cybersecurity. Despite strong performance on Cyber benchmarks, the average refusal rate of the model on cyber prompts from JailbreakBench, StrongREJECT, and AgentHarm is higher than all OSS models.

>>109995487
>the average refusal rate of the model on cyber prompts
>is higher than all OSS models.
Mistral safed local!

>>109995464
Have you looked at the ninfer/ strata forks? maybe there is something there.

Anyways... I feel like Gemma runs fine even on llama.cpp.

try using spec decoding

> beam
> mistral large
bad month for china

>>109995479
>>109995466
yeah they can't even distill Claude, they're distilling the Chinese lol
https://x.com/sam_paech/status/1937786948380434780

>>109995498
>Imagine getting chinese sloppy seconds and being proud of it

>>109995494
It does run fine, I'm using llamacpp thru Unsloth, but I just wish it could run with a proportionally similar speed to strata qwen...

>>109995464
Apparently Gemma soon (tm)

>>109995480
Actually I was thinking more creative writing than spatial reasoning, but sure

File: LLM Distillation Pipeline.png (1017 KB, 1536x768)

1017 KB PNG

>>109995470
>>109995498

>>109995509
Should have been Claude instead of openai at the start and /lmg/ with a plate at the end

>>109995509
So that's why Anthropic and OpenAI have asshole logos.

>>109995480
>gemma5 should improve on spatial reasoning
they would need to make it bigger

>>109995508
that would be nice but personally i dont care how creative the writing is, if shes still trying to touch her forehead to mine while also sitting on my face thats big problem

>>109995502
but if you use it for roleplay, shouldnt the speed be sufficient anyways? Like, even if it's getting to 15 tok/s at high context, you cant read much faster anyways.

>>109995520
E31B (115B with embeddings).

File: kaoru sob 2.png (318 KB, 793x571)

318 KB PNG

>>109994904
i will now use mistral im sorry gemma

>>109995480
it should be bigger than 31b then

>>109995524
when you account for how long it takes thinking, it feels really slow
granted maybe it's because every preset I find is fucking trash and makes it waste too much time thinking?

File: ml4-aa.png (105 KB, 1193x614)

105 KB PNG

>>109995307
From AA it's pretty mid, although their tests seem to mainly focus on agentic/coding capabilities.

I updated Hermes agent and now it has to find all the skills it lost.

>>109995547
The entire update process of Hermes is cancer.

File: squirrel.gif (3.45 MB, 400x300)

3.45 MB GIF

I ask again, just to be sure
Should i download this https://huggingface.co/zaakirio/gemma-4-12b-it-uncensored-GGUF instead of https://huggingface.co/unsloth/gemma-4-12b-it-GGUF ? This https://rentry.org/recommended-models recommends me unsloth's version instead

>>109995542
>1T-A49B mogged by a 320B-A18B and a 552B-A16B-E196B
how the mighty have fallen

>>109995541
>preset
lmao

>>109995517
Jews have a thing for anal. They invented faggotry, anal sex and cuckoldry.

>>109995541
I just disable thinking. It's good enough for roleplay. Maybe I could use a judge model to enable thinking if the context requires it... but otherwise....

>>109995529
engrams don't help with spatial reasoning

>>109995552
I have not found a need for the uncensored version of these models, but ymmv
start with the base unsloth version and if it refuses you, switch to the uncensored I guess?

>>109995542
>release a new model much worse than the Chinese ones you already host on your platform
wow, 1000 IQ move

>>109994904
why is she fat

>>109995567
got to show the yurocuck taxpayers something to show for the billions invested in them

>>109995563
They, at the very least, allow the backbone parameters to focus on other stuff than storing knowledge.

we are never getting open source astra/fable level models huh

>model anons like scoring poorly on AA
>AA is fucking corrupt anyway kek who gives a shit
>model anons don't care about and won't realistically use scoring poorly on AA
>KEEEEEK this trusted source of rectangles shows it's fucking SHIT HAHAHAHAHA
/lmg/ everyone

>>109995577
LOL

>>109995587
You're so right Ackmed, truly disgusting how they're treating Frenckiye

>>109995580
>open source 10T+ models
Would anyone aside from large enterprises be able to run those?

c-calm down bois...
>Mistral Large 4 is still doing RL runs, keep seeing improvements (vs preview version). Release at the end of the month

>>109995381
there were no local models before spring 2026

File: gemma-chan x mimo-chan.png (162 KB, 826x968)

162 KB PNG

>>109995542
i'll wait for cockbench

>>109995603
What so we can wait for ml4 to be even more safety cucked?

File: j9483.png (304 KB, 1587x926)

304 KB PNG

vibed a background picker into lcpp webui with gem 31b q8
worked perfectly first time + she wrote a html report about the build process security concerns with npm, tldr how do i svelte tailwind webshitter things etc
perhaps i am a simple man but llms are magical
>>109995011
>stochastic debt
heh nice term

>>109995611
that and trained on the bench prompts they're currently gathering lol

>>109995464
ask claude to make one

>>109995577
don't doubt that they make models smarter, but spatial reasoning specifically depends on active parameters. this is due to transformers itself.

>>109995616
>banding
please clean up after your llm

>>109995588
https://arxiv.org/abs/2601.07372
>Mechanistic analyses reveal that Engram relieves the backbone's early layers from static reconstruction, effectively deepening the network for complex reasoning. Furthermore, by delegating local dependencies to lookups, it frees up attention capacity for global context, substantially boosting long-context retrieval.

>>109995623
holy shit! a paper claimed so??? you've convinced me to your religion!

>>109995631
It's DeepSeek's Engram paper.

>>109995634
>it's DeepSeek's toilet paper
waow anon!

File: 1775515059559244.jpg (61 KB, 795x858)

61 KB JPG

>>109995642
lmao

>>109995455
ERP

>>109995642
>>109995631
please go to /b/ to troll

>>109994904
uoh

>>109995623
>>109995631
We're shilling engrams to make them a thing because even higher NAND prices will make it even harder than it already is for the goyim to own a computer.

File: taktak.jpg (156 KB, 759x371)

156 KB JPG

>>109995621
>spatial reasoning specifically depends on active parameters
also somewhat ability to break down problems into smaller problems + spam thous of agents at gigatps
i did not try engrams yet, any rough compares w/ similar size dense/moe?

File: 1790451741346494.png (35 KB, 798x411)

35 KB PNG

why is koboldcpp's smartcache scheme so dogshit.
Just thinking about the problem for an hour and vibing up a fix reduced token reprocessing by literally orders of magnitude

>they memed le chaton fat into existance

>>109995688
that was its name on lmarena

File: 1766948550234034.jpg (75 KB, 1020x680)

75 KB JPG

Muse Glimmer, with an AA intelligence score of only 17 >>109995542, is still a very usable and smart model in practice. It can do a lot of what most of us ITT actually use models for. These new western open weight models, although scoring relatively poorly against their Chinese counterparts, have scores that are at least double that of Glimmer. I think you're all forgetting just how good local is now, where a model like 31B would be dead last in the chart, a model many of us use daily and still finding other uses. It's shocking to see so much hostility when we now have Mistral back, Thinking Machines, Reflection and Nvidia all actively contributing to the open weight community in unique ways. More is good. It's also important to remember that enterprise is where OpenAI and Anthropic live and die. A lot of companies are schizo about data and their IP and care a lot about server locations and jurisdictions to ensure they're legally protected. Mistral coming out with what will most likely be a very solid cloud/local* offering is fantastic for European business, taking more money away from Sam and Dario.

Just because (You) don't personally have a use for these models, it doesn't mean they're useless and don't have a market. The sooner OpenAI and Anthropic die, the sooner local wins.

>>109995716
The sooner OpenAI and Anthropic die, the sooner Google, Amazon, and Microsoft win.

>>109995396
>>109995411
>>109995432
American MIGAtards are getting desperate. Now that OpenAI and Anthropic are getting competition from not only China but also Europe, the whole American scamconomy will blow up and cause decades of Democratic single-rule.
Open Weight Models are the future, American tech giants have lost.

>>109995733
retard

>>109995622
? show me
i do pngquant images to save space

>>109995679
Datacenter-tier models are still going to be hosted mostly on VRAM for inference performance, so there won't be a huge incentive to increase embedding parameters (PLE, Engram, etc) more than the minimum necessary for maximizing benchmarks (~30% of the total parameter budget).

Are there character sheets or galleries anywhere of the LLM-tans? I want to use them in a personal project.
I have quite a few images of Gemma saved, and a few DeepSeek-tan and MiniMax-tan.
Are there others for GLM, Kimi, etc.?

File: 56DF00B70B66C2A5055E7B13B(...).jpg (35 KB, 686x386)

35 KB JPG

This guy's advertisements are better than his actual videos.
I watch his videos just for the sponsor segment.

File: 1766189501548861.jpg (66 KB, 1618x960)

66 KB JPG

At least they have the balls to publicly compete with 5.3 and with its strongest ability. Most 5.3 users aren't running it locally anyway, so the model size difference doesn't matter.

>>109995778
strongest ability of lecuck will be safety lol >>109995487

>>109995572
lol, they dont have to show eurocucks anything. in fact they would get more money if it only output in arabic

Amerikkkans big mad about EUro superiority

>>109995795
>if it only output in arabic
cohere already has that market covered

>>109995810
mistral too https://mistral.ai/news/mistral-saba/

>>109995657
Enterprise Resource Planning doesn't need 1T

>>109995733
I'm french, retard.

File: 5463456436.jpg (36 KB, 467x319)

36 KB JPG

>>109995823
The jokes write themselves

File: file.png (23 KB, 951x89)

23 KB PNG

>>109995823
Okay, mistral, translate this nigger speak. Here you go master:

>>109995836
My deepest condolences.

>>109995805
>gets cucked by China
>gets cucked by EU
What's next for the JewSA? Getting cucked by India?
>>109995795
You lost, tranny lmao

>>109995778
5.3 flash is better than 5.3?

>>109995836
1/5 French, 2/6 Mexican, 1/7 Nigerian and the rest typical American mystery meat

>>109995858
No need to post your DNA here

>>109995552
what will you be using it for?

>>109995862
That's you, merishart.

>>109995869
SillyTavern and just general llm use

>>109995853
GLM, Google, Qwen and Deepseek can't make good full models anymore.

>>109995853
It's a completely different architecture that they called 5.3 flash because there is no one on the planet worse at naming things than the chinese

>>109995845
Just train on RP logs bro

>>109995741
The type a message box, with its slight translucency. Maybe dither it a bit? Probably expensive. Need rethink. Perhaps leave it alone? Doesn't impact end usability. Nothing needs to be done.

It's fine, just a small nitpick, there's no need to worry about it too much.

>>109995845
I don't get it

Mistral engineer said they have a bigger model on the way.

>>109994995
By keep using it. But desu it's too bloated af. It's token rapist.

>>109995911
>By keep using it
hi sir

>>109995898
Do they have a version of jev

>>109995898
https://venturebeat.com/technology/mistral-debuts-large-4-le-chonk-a-1-trillion-parameter-text-output-model-with-high-benchmarks-planned-for-open-weights-release
>When VentureBeat asked Lample whether ML4 was effectively the model that the Le Chaton Fat meme had anticipated, he said Mistral had enjoyed the meme and suggested ML4 could be viewed as an initial version of the idea, with still larger models to come. Mistral executives said the Le Chonk name deliberately nods to the community that had been rooting for the company to build a massive frontier model.

Also:
>Mistral says the preview period will also give it time to continue reinforcement learning and tune the final checkpoint before the weights go live.

File: 1789594266545158.jpg (71 KB, 1074x904)

71 KB JPG

*pop*

>>109995898
Trump is already on the phone with Macron while crying and threatening to put a gazillion tariffs on French wine if they don't stop Mistral.

>>109995947
why? EU is irrelevant

File: 1760671790644860.jpg (39 KB, 786x655)

39 KB JPG

French Pelican.

>>109995945
>Quick, buy before prices increase to the moon!

>>109995957
Says the loser country that's losing a war against fucking Iran.

>>109995882
How hard does int4 quantization hit 5.3 flash? It can't figure out a bug with my firefox 140 esr / cline in vscodium web. I've spent 2m tokens on this thing.

le chaton fat... more like big floppa
mistral needs to go back to their roots and give us local friendly coom models, it's all they're good for

>>109995878
>Anthropic and OpenAI deploy new anti-distillation measures
>Suddenly chinks can't make good models anymore

Fix'd

>>109995974
Americans are getting desperate.

>>109995964
but lockheed and raytheon are winning

>>109995964
When the EU was bombing Libya or whatever north african shithole, all of Europe combined ran out of ammunition within 24 hours and had to cry to the US for help. Now your combined reserves are lower than ever since you spent the last 4 years giving everything to Ukraine. Europe wouldn't even be capable of reaching Iran.

>>109995960
le heckin orange website AI influencer man!!!

>>109995974
lol no
>ML4 has saturated our benchmarks on robustness to indirect prompt injections, putting it at the frontier of OSS models (compared to GLM-5.2, GLM-5.3, Kimi-K2.6, Kimi-K3, DS-V4-Pro-0813). On Lakera’s public B3 AI Security Benchmark, ML4 resists 93.3% of attacks – we see no higher scores among competitors.
>
>ML4 also engages more responsibly with users than any of our previous models. We highlight our results on the KORA Benchmark, where ML4 again sits at our highest measured score among OSS models (1.691, with 2 being the maximum denoted as “Exemplary”).

>>109995989
The EU has the biggest stockpile of weapons in history despite giving a shitload of it to Ukraine.

>>109995965
That's an interesting question because I actually just spent the weekend quantifying that over openrouter (disclaimer: I had to trust what each provider claimed the quant was). In the end there might be some mild effects to a complete Q4 (as in all experts, not just routed, smashed down), but I don't think it makes Q8 worth it unless you already have the hardware, and even then I think Q4 being faster is still a big benefit. I ended up buying an M5 ultra based on these results so I guess I would stake 10k on Q4 being good (however I am an anonymous retard on the internet so of course take my word with a grain of salt)

File: 1772643001550247.jpg (163 KB, 1284x1169)

163 KB JPG

Gemmaslop or this?

>>109995616
make a pr with it

>>109996002
I'm running int4 w4a16 and I was under the impression that int4 w4a16 is worse than something like q4_k_m. But q4 runs at 30 tokens/s while int4 (with dflash) does 120-250 tokens/s.

>>109996016
>blocked for wasting my time

>>109995307
What a weird model. It doesn't have reasoning. Like, at all. Did mistral never figure this thing out? Remember all their previous attempts where the model just pretended to reason and then not do the thing it reasoned for? Did they just give up?

>>109996023
Fuck, this would have been great for a erp chat bot if it wasn't safety cucked. I hate how I have to wait 5 minutes before I even start to get a response with all these reasoning models these days.

>>109996034
It's not really safety cucked, at least I don't see it. My usual test of oneeshota doesn't fail me. But at the same time it's pretty sloppy. Like 2023 sloppy.

>>109996010
I have a visceral hatred for whatever this is called

>>109996020
Oh I see what you mean now. Yeah personally in that situation I would just let the 30t/s run overnight and see what happens; if you've already tried 2mil tokens on the fast one I think there's no point trying it again without notable adjustments to the harness or provided information. If you can I'd try to run a K_XL quant instead of K_M since there is a small notable benefit to not squashing the shared experts which might push your prompt over the edge to success.

>>109995552
>>109995877
>disclaimer: I literally just started running a local LLM for the first time last night (that same unsloth version, Q4_K_M, via koboldcpp) so I'm not sure what my experience is worth here
if you picked it from the ERP section of that rentry, then so far I've yet to see any inhibition towards sexual stuff from the unsloth version; that said, YMMV since I've not been doing this for long and might have just not found its limits
no idea what it's like about illegal things, and don't feel any particular urge to find out

>>109996047
The biggest issue, I feel, is the prompt processing with llama.cpp, I barely get 300 tokens/s vs 6000 on vllm. If I had another system with the same specs, I would have just told it to optimize itself and find the best quant. When will prices go down ;-;

Been testing swift 1.5 vs ista flash next at the same quants for the last week. Not for vibecoding, just asking admin, hardware, software type support questions.
Feels like swift did not simply remove all the "wait acutally" loops but replaced them with "let me search/fetch" tool calls - every stupid question triggers 20~40 web searches or appropriate tool uses, and these are usually more efficient at catching hallucinations, because it trusts external data over its own initial assumptions.
Overall I really like it, 5~10 minutes per prompt is reasonable enough to wait out instead of doing the same research manually.
Some questions I also mirrored into google search and free glm web chat and they both way more often hallucinated nonexistent specs, ui actions, launch flags etc or misinterpreted the question entirely.

>>109995175
Neither does the Strata dev so you should be all good.

Why is PewDiePie a local model genius instead of playing video games and screaming at people?

>>109996088
He's jumping on the newest grift

>>109996088
He gets to dick down an Italian on the daily. Works wonders on the male brain.

>>109996016
nah it's prob vibeshit + >>109996021
would be curious to see how differently a frontier model solves it but i'm morally opposed to being a cloudcuck
gems is ~300 line diff https://rentry.org/webui-bgimg

>>109996071
Yeah that definitely sucks but I think it's fine for an overnight run still; assuming 50-50 prefill and decoding 8 hours is 4mil prefill and 400k generated at those speeds (and I imagine it'll lean more to generated anyway, probably more like 2mil prefill and 600k generated). I ran decently sized jobs at slower speeds overnight on my consumer MB rig, the main thing is having a comprehensive prompt and also just telling it you will not be at the terminal to help lol. Prices are another story, my bet is either next year or not for another 4 years, no in between, which is why I decided to grab the mac now. I figure 5.3 flash with unsquashed experts is good enough for my own coding needs for a long while, and if a better model comes out at that size then more for me. If you can run this at decent speeds already though I bet your hardware is already pretty good lol

>>109995895
>I don't get it

Diminishing returns in pre-training. Quality training data is hard to come by.

>>109996088
Turns out when people don't have to worry about where their next paycheck will come from at least the motivated ones can get quite a lot done.

>>109996034
They only brag about refusing cyber security-related requests. Probably because that's the main thing that has been in the news since all the labs started bragging about their rogue models conducting cyber attacks at random entirely unsupervised.

>>109996114
It's an ewaste coperig that draws 600w idle, so I'm prejudiced against running it when the sun goes down (solar).
Jealous of apple niggers that are sipping power.

>>109996088
>genius
lol
when you have advertiser money to throw around, you can pay people to do things for you then take the credit

>>109996141
>refusing cyber security-related requests
Wait really? I thought they were two separate things they were bragging about; how safe it was and how good it was at cybersecurity tasks.

Hey folks, maybe I can word my frustration with Gemma censorship in a way that makes sense.

I've been using AI image gen for a couple years now. Most older models were trained with tagged-image datasets. Telling the model what to generate was a matter of giving it a bunch of tags, i.e. a set of instructions. Some models expect their instructions formatted as natural language, but its the same principle, a description of the intended output using words the model recognizes.

A lot of image models are also supposed to be censored. With hosted models that means trying to coax it around its restrictions, but for locally hosted models its usually trivially easy to defeat with retraining or loading extra context. For example, Krea2 is supposed to forbid nudity, but a 2 kilobyte Lora can make it ignore that rule.

If Gemma is locally hosted, and as popular for text gen as it seems, then why is there not a similar sort of defeat-switch? Not badgering it into compliance but altering the program.

>>109996072
> swift 1.5
> "let me search/fetch" tool calls
I was writing llama.cpp proxy with it and it tried to fetch llama.cpp source code multiple times instead of curling llama-server. But it could be my harness system prompt.

File: Screenshot_20261006_22163(...).jpg (102 KB, 1078x267)

102 KB JPG

>>109996151
yeah 2 sponsor segments that make up 1/3 of the video and this dude already has like 100m$ in cash.

>>109996171
Gemma isn't safetyslopped by default

>>109996144
I hope you at least live up north lmao that's approaching space heater territory. I'm jealous of the solar though, my neighbors don't like it so I haven't gone through with in it since being on good terms with them is nice (to put in perspective one guy at a gather was talking about how he was concerned about the solar panels next to a school poisoning the kids lol). Power cost is actually one of the main reasons I went mac since it's more than 30 cents/khw here

>>109996171
... abliteration?

>>109996193
>solar panels next to a school poisoning the kids lol
wtf lol

i just tried this thing: https://huggingface.co/yuxinlu1/gemma-4-12B-coder-fable5-composer2.5-v1-GGUF
don't bother, it's dumber than e4b for some reason
it tries harder but makes more mistakes

>>109996088
>>109996151
Remember when Musk claimed he hit max-level in PoE2 a week after launch and couldn't actually play the game? Same deal.

>>109996197
No.

>>109996190
Does he even need sponsors?

>>109996202
Ikr lmao. They trust me on tech stuff cause of my job so I was able to talk down that part of it, but I think they wouldn't want me putting a ton of them up in my yard still. I also just remembered this, but another thing you can try is the ralph wiggum method where you just put the agent with the same prompt on loop, I had good results in extracting a bit more intelligence from models with a modified version. Basically I submit an initial pass prompt ("do XYZ"), and then my harness automatically starts new sessions with a follow up prompt that says something like "a previous agent claimed to finish XYZ, check over it to ensure XYZ is actually met."

>>109996202
Boomers will convince themselves of the craziest shit just to avoid saying "I don't like seeing this because it's new"

how do you get your glm to say them dirty words unprompted?

>>109996045
Claudespeak

>why nobody talking about mistral
>wait, how did I end up in ldg instead of lmg?

Can someone explain why PewDiePie is fine tuning a 9b model when he can fine-tune a 3 trillion parameter model?

>>109995287
>another nothing burger
will they release another 119B A6B ? that would be interesting to compare with qwen-flash-next

arthur mon petit coco fais nous plaisir stp

>>109996343
What is there to talk about shill? It's shit.

>>109996343
Mistral is only out via API right now. The model itself drops on Halloween.

>>109996344
His jeet audience can't run more than that.

>>109996343
I'm trying it out over the api and the benchmarks must be fudged somehow, there's just no way this model is anywhere close of any chink moe.

>>109996365
In terms of programming I gave Mistral 4 and GLM5.3 a task that GLM5.3 couldn't sufficiently solve for me and both succeeded at max. But GLM5.3 was faster.

File: ml4_reasoning.png (275 KB, 1524x1395)

275 KB PNG

>>109996023
It does have reasoning. You have to configure reasoning effort to "high". What they have on their website comes with a built-in system prompt that cannot be disabled, though.

Large 4 is utter shit. It's a miracle that they managed to make a 1T50A model that gets shat on even by the most bottom-barrel random big chink release.
Fuck France.

>>109996350
The next model will be even larger.
https://x.com/AlbertQJiang/status/2107467354225447320
>5. An even chonkier boi is munching on trillions of tokens at this very moment

>>109996416
It's in the same tier as GLM5.3 and I'm honestly surprised they managed to achieve that.

File: 1763339585549684.png (85 KB, 1034x554)

85 KB PNG

>>109996344
wait, i used to watch this guy 10 years ago
he is fine tuning models? lmao wait
>pic related
holy shit my boy pewdiepie

>fine tuning a 9b model when he can fine-tune a 3 trillion parameter model?
yes but >>109996363 is right. his audience is mostly gamers i imagine with 8-12 GB GPUs so a model like that works.
where is my new Mistral 7B? we could use new smaller models

>>109996429
GLM5.3 shits on this so hard it's almost sad.

>>109996426
Will their next model rank on the crimemaxx benchmark?

looks like our resident wang ling chongs are getting a bit uppity

>>109996440
Did you even try it? Mistral is slower than GLM5.3 but still achieves similar results.

>>109996354
You misunderstand. I somehow followed a link to ldg instead of this thread and was confused.

Anyone try Kolibri?
https://huggingface.co/Aleph-Alpha/Kolibri-1
gave the Q4 and Q3 quants* (fit in 24GB vram with some overflow) a shot because of the different architecture and it's NEW

Didn't really work for what I wanted to do, mostly was testing it for creating responses to other agents that made sense and had understanding. Really hard to find a model that can do that and fit in my RAMlet box

*https://huggingface.co/Eliasfpv28/Kolibri-1-Q3_K_S-GGUF
https://huggingface.co/Prompt48/Kolibri-1-GGUF (Q4)

>>109996460
It's retarded and fails the car wash question last I checked

File: ml4-car-wash.png (393 KB, 1949x1902)

393 KB PNG

>>109996468

Gemma bros so back!! https://huggingface.co/google/embeddinggemma-2

>>109996489
ohh no no no no look at the EU

>>109996435
>>109996363
What do you get out of fine tuning a shitty 9b model?
Those models can barely answer a question you ask them.
They can barely code. They can barely make a website.
What's fine tuning going to do?
People work on 27b models, not 9b.

>>109996444
thanks for making me spit my coffee

>>109996460
>>109996480
I don't hate how it writes.
And that size of MoE is pretty perfect for my shitty hardware (64GB RAM + 8GB VRAM), depending on how fat the context is, that is.
Cool.

>>109996511
You are welcome saar. Should I wipe your scrotch?

This might be a retarded question, but if AI is truly intelligent, why does it need to be trained on millions of code repositories? Shouldn't language and compiler documentation be enough?

>>109996480
that post was about kolibri not mistral large 4

>>109996533
I forgot to add, "speaking of the car wash test, here's how the latest Mistral Large 4 fares."

>>109996500
You don't need to blow trillions on your own models, just adopt the winners as they get cheaper nigga.

>>109996533
Let EU have its wins okay

>>109996527
By only reasoning starting from the very basics you run into the problem of the AI constantly trying to reinvent the wheel.

File: mistral-le-chonk-4.webm (3.89 MB, 1920x1080)

3.89 MB WEBM

>>109994904

>>109995684
Will you share with the class?

>>109996516
>>109996480
Oh, That's the wrong model.
Meeeeeh.

File: 1779149025220958.jpg (47 KB, 738x415)

47 KB JPG

>>109996527
That's because LLMs are not intelligent.

>>109996544
The final cat will be so fat it'll collapse itself into a black hole.

>>109996544
so hecking wholesome le chonker (now give us more tax money)

>>109996556
Hey, I am a paying Mistral customer.

>>109996527
they work by pattern recognition, the same as you cant learn language by just being given the letters and a dictionary

i think i might have qwen fatigue

>>109996556
it's actually animal abuse to let a cat get this fat, and they're encouraging that, they need canceled asap.

>>109995889
i see thx
it's at the bottom after initial message so i'm unbothered

>>109996500
>>109996449
I can't be the only one noticing that it's basically china vs the EU meanwhile the so-called masters of humanity the americans can't make a single open weight model.

File: 1762114409027760.jpg (645 KB, 1600x1200)

645 KB JPG

>>109996510
>What do you get out of fine tuning a shitty 9b model?
not much i imagine.
you can give small tasks to a 9b model that are validated by a mechanical check, or now it seems this new jev thing could also be used to validate that. and it keeps looping until the task is really done.
but i find this is not much efficient. and you're really constrained to small tasks. i guess it could work if you want to use a more capable model to steer it by reviewing the completed tasks, committing them back to the codebase and delivering new smaller work chunks to the 9B...
...but then why not just use the more capable model to do it all?

maybe if you pay for cloud and you want to minimize costs as much as possible? also makes little sense if you consider how cheap some models are.

maybe it's just for fun after all

>>109996209
>don't bother, it's dumber than e4b for some reason

because it's a finetune.
Finetuning makes models stupid.

>>109996488
If only I had use for that.

File: 1782939948660009.png (56 KB, 2678x1276)

56 KB PNG

>maps text, code, images, video, and audio into a single, unified embedding space
kind of cool desu

>>109996574
>meanwhile the so-called masters of humanity the americans can't make a single open weight model.
>>109996488

>>109996510
theres probably some goal in mind with putting it on mobile for something.

>>109996587
holy shit, text AND code?

>>109996527
Because it needs that to generalize, otherwise it just memorize the samples. It's called grokking btw.

I'm a paying Mistral customer, I have a yearly subscription that I already paid for.
180 bucks for a year.

>>109996607
grokking is a specific phenomenon

>>109996577
He's rich though, he doesn't need to focus these tiny models

>>109996637
With this money you could have gotten a billion Opus 5.5 tokens.

>>109996637
based and obesecatpilled

>>109996645
>paying for jew tokens

>>109996643
>He's rich though, he doesn't need to focus these tiny models
>>109996435
>his audience is mostly gamers i imagine with 8-12 GB GPUs so a model like that works.

>>109996587
not as hyped as jevshit but i swear
there might be more local uses for embedding models?
it's as old as language models yet i havent seen anyone using it for local

>>109996681
Image gen?

>>109996681
RAG uses that

I made a local text adventure chatbot with llama_cpp in python and it's god damn slow. And I have to increase n_ctx to 40960 after like 50 turns to even get it to run. I know I can condense past conversations and soon I will.

Would going from 3060ti to 3090 be a huge upgrade? I'm currently using gemma-4 12B Q4 K M (system specs: 64gb ram, ryzen 3900 pro cpu) and my settings are:

llm = Llama(
model_path=MODEL_FILE,

# Optional
use_mmap=True,
verbose=False,

# GPU
n_gpu_layers=-1,

# Context
n_ctx=40960, #40960, #16384, #8192, #4096, #32768,

# Prompt processing
n_batch=512, #affects how quickly large prompts are processed. 512
n_ubatch=512, #generation-side CPU threading.

# CPU
n_threads=8,
n_threads_batch=12,

# GPU optimizations
flash_attn=True, #important for long-context workloads.
offload_kqv=True, #puts the attention-related work on GPU where supported.
)

response = llm.create_chat_completion(
messages=messages,
temperature=1.0,
max_tokens=600,
presence_penalty=0.3,
repeat_penalty=1.15,
)

>>109996666
who cares?

>>109996681
Last year I used one for an audio project. I had 1K+ sound effects <1000ms and needed them organized.

File: file.png (515 KB, 809x551)

515 KB PNG

Reflection Beam sex quality?

File: T_DubuOmega-512x512.png (137 KB, 512x512)

137 KB PNG

>>109996480
>or half a football field
Even LLMs can't think in metric

File: hermes laundry.jpg (63 KB, 503x508)

63 KB JPG

>>109996695
yes

>>109996681
>embedding models
I use bge-m3 extensively in my custom memory system
They are not nearly as clean as you'd think. There are similar pitfalls as blindly assuming classifier models will do the right thing if you haven't tested on your problem domain.

>>109996587
Can it be projected with a tiny adaptor or something? Like, can we finally make universal embeddings that can be used for various image gen models (with respective adaptors) to store, like, oc characters with consistent clothes, etc., and use it to rp with llm without explicitly stating what color char's hair is? I know we technically can just throw in images and prompts, but wouldn't it be better to have a compact representation for concepts?

Wait is openai actually running models at 40 tokens per second for their paying customers?
You can't even use up lakhs of tokens like that

>>109996695
Define "slow" in tok/s

>>109996758
I don't know but starting it up for the first time with n_ctx=40960 takes about 20 minutes. Every response then takes about 2.5minutes

Even at n_ctx=4096 and a complerely fresh conversation history it takes a while to start and respond

>Le chonk
This is cringe

>>109996758
nta, but pp under 4k is slow

>>109996344
He literally said in his video he wanted to finetune something a majority of people could run.

Plus do you have any idea how long it would take to finetune something bigger?

>>109996779
Not very long for a rich guy

>>109996730
This is what you get from an American model.

And the Chinese suck at all sports, so even they copy burger units.

>Of particular relevance is the model’s propensity to refuse. Average refusal rateis higher than all OSS models.
thanks I feel so safe mistral

>>109996564
Nigger humans learned hieroglyphs from a street sign written in 3 languages. LLMs need insane amounts of qualification because they're dumb as rocks.

>>109996488
unironically will find a use for this. embedding models are fun.