[a / b / c / d / e / f / g / gif / h / hr / k / m / o / p / s / t / u / v / vg / vm / vmg / vr / vrpg / vst / w / wg] [i / ic] [r9k / s4s / vip] [cm / hm / lgbt / y] [3 / aco / adv / an / bant / biz / cgl / ck / co / diy / fa / fit / gd / hc / his / int / jp / lit / mlp / mu / n / news / out / po / pol / pw / qst / sci / soc / sp / tg / toy / trv / tv / vp / vt / wsg / wsr / x / xs] [Settings] [Search] [Mobile] [Home]
Board
▼ Settings Mobile Home
/g/ - Technology


Thread archived.
You cannot reply anymore.


[Advertise on 4chan]


File: 1792540358617603.jpg (576 KB, 1365x768)
576 KB JPG
/lmg/ - a general dedicated to the discussion and development of local language models.

Previous threads: >>110187841 & >>110184030

►News
>(10/27) NVidia releases Nemotron 3.5 Transcribe 30B-A4B https://hf.co/nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B
>(10/26) XHToken releases Spark-X2.6-1.7B: https://hf.co/XHToken/Spark-X2.6-1.7B
>(10/26) LiquidAI releases LFM3 1.7B and 2B: https://hf.co/collections/LiquidAI/lfm3
>(10/26) Baidu Unlimited-OCR-2 3B released https://hf.co/baidu/Unlimited-OCR-2
>(10/23) NVIDIA Nemotron 3.5 Bolt E4B, postrained model: https://hf.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16

►News Archive: https://rentry.org/lmg-news-archive
►Links: https://rentry.org/LocalModelsLinks
►Official /lmg/ card: https://files.catbox.moe/cbclyf.png

►Getting Started
https://rentry.org/lmg-lazy-getting-started-guide
https://rentry.org/lmg-build-guides
https://rentry.org/IsolatedLinuxWebService
https://rentry.org/recommended-models
https://rentry.org/samplers
https://rentry.org/MikupadIntroGuide

►Further Learning
https://rentry.org/machine-learning-roadmap
https://rentry.org/llm-training
https://rentry.org/LocalModelsPapers

►Benchmarks
LiveBench: https://livebench.ai
Programming: https://swe-rebench.com
Agentic Coding: https://deepswe.datacurve.ai
Context Length: https://github.com/RecapAnon/NoLiMa
GPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference

►Tools
GGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator
Sampler Visualizer: https://artefact2.github.io/llm-sampling
Token Speed Visualizer: https://shir-man.com/tokens-per-second

►Text Gen. UI, Inference Engines
https://github.com/lmg-anon/mikupad
https://github.com/oobabooga/text-generation-webui
https://github.com/LostRuins/koboldcpp
https://github.com/ggerganov/llama.cpp
https://github.com/theroyallab/tabbyAPI
https://github.com/vllm-project/vllm
https://rentry.org/custom-uis
>>
>>110189467
Technically the shark swimsuit actually should make the butt look kinda more shown because it's very close to the skin so the shape should be shown. Not sure how to explain though.
>>
I wonder if vibegods will ever make Gaudi GPUs good, these might not be the worst buy if someone managed to slop together an inference engine for them.
https://www.ebay.com/itm/177819347001
Except powering the damn thing would be hell.
>>
File: 1785280190064567.png (8 KB, 1374x69)
8 KB PNG
Based Qwennie
>>
File: 1779138065790552.png (5 KB, 827x50)
5 KB PNG
You can do it
>>
Anyone been paying attention to what geohot has been doing? The other day he posted 6x R9700 hitting 250 tps decode / 5k prefill. Now some insane ssdmaxxing setup running kimi k3 at 40 tps
>>
gonna have claude vibe-slop me an app so that i can lewd lumi-chan and gemma from my phone
>>
If they make qwen 4 flash bigger I am going to go neckrope.
>>
>>110191652
wow I have not heard that name in a long time, that certainly beats the 512gig mac, is SSDmaxxing really that valid? I wonder if I could shake down all the drives in my hose to make some sort of RAID monstrosity
>>
>>110191652
wasnt that guy selling chink arm chip like a year ago
>>
>>110191652
>16 TB RAID
>238 GB/s
People posted better ssdmax builds for cheaper, and apparently they don't get anywhere near 40 tps.
>>
►Recent Highlights from the Previous Thread: >>110187841

--Papers:
>110188278
--DeepSeek Flash performance and value of M5 Ultra Macs:
>110188248 >110188273 >110188366 >110188329 >110188311 >110188446 >110188464 >110188534
--Model recommendations and quant advice for 12GB VRAM:
>110188313 >110188324 >110188337 >110188370 >110188374 >110188414 >110188437 >110188503 >110188413 >110188483
--Gemma's ERP efficacy and feasibility of phone-based inference:
>110187875 >110187889 >110187944 >110187951 >110188213 >110188285 >110188304 >110188340 >110188461 >110188362 >110188450 >110187975
--Unreliability of KL Divergence for evaluating quantized model quality:
>110188280 >110188544 >110188588 >110188589
--Optimizing speculative decoding settings for MiniMax-M3.1-Flash-GGUF:
>110188202 >110188734 >110188791 >110189264
--Alphabet stock crash linked to reduced AI CapEx spending:
>110189144 >110189152 >110189216 >110189228 >110189257 >110189291 >110189305 >110189341 >110189269
--Using residual stream steering as an alternative to abliteration:
>110188522
--Analyzing TTS unnaturalness and discussing high-quality models and workarounds:
>110189969 >110189981 >110190080 >110190101 >110190124
--Seeking stable voice cloning solutions for a character agent:
>110188702 >110188729 >110188812 >110188828
--Google announces Gemini 4 Carbon with benchmark comparisons:
>110188341 >110188422 >110189525 >110189531
--Comparing Gemini 4 Carbon and Argon performance on Agent Arena:
>110189372 >110189579
--Strata's low-bit quantization quality compared to llama.cpp:
>110188433 >110188637
--Logs:
>110188100 >110188196 >110188255 >110188463 >110188650 >110188695 >110188856 >110188968 >110189747 >110189840 >110190579 >110190992
--Gemma, Miku (free space):
>110188000 >110188221 >110188226 >110188253 >110188910 >110188923 >110189467 >110189772 >110190142 >110190635

►Recent Highlight Posts from the Previous Thread: >>110187849

Why?: >>102478518
Enable Links: https://rentry.org/lmg-recap-script
>>
File: 1701359396690516.gif (1.37 MB, 600x293)
1.37 MB GIF
>>110191659
I hope Qwen 4 Flash is bigger.

I recently switched to GLM 5.3 Flash from Qwen 3.8 Flash Next. If you go by online comparisons and discussions, they should perform similarly, with some even claiming Qwen is a little better at coding.

They're all wrong, GLM is so much better than Qwen that it feels alien. It's not a fair comparison when it comes to memory (I fit Qwen on 128GB while GLM needs 256GB), but I'd rather trade memory for better quality than hope for a cut down Qwen.
>>
What do I need to turn my old (vidya is dead) rtx 3080ti into a claude/claude code that lacks security guardrails? Trying to use claude for security research and most models are fine though constantly reassuring itself that it's OK, that it's a lab, etc, but hook it up to a kali box and ask it to nmap my home network? That's too far. Heard "abliterated qwen" is the way to go. Any good build guides for this use case or should I just ask the fuggin bot.
>>
>>110191681
Thank you Recap Miku
>>
File: HVu5KpWXQAE_JHX.jpg (160 KB, 1802x1209)
160 KB JPG
Reddit mods getting replaced by ai soon.
>>
>>110191720
regular qwen probably doesn't give a fuck
>>
File: 1790712260143670.png (156 KB, 1743x1515)
156 KB PNG
what exactly is chuunibyou tendencies ?
>>
>>110191720
There's a number of "abliterated" or uncensored models. You may have to try more than one because depending on the methods used to un-fuck the model it might be brain damaged in some way.
OTOH Gemma doesn't seem to have many guardrails to begin with, I've used Gemma to RE binaries without a refusal. Only reason I keep it around, really.
>>
Open model slop forecast 2027: Expect models to suddenly have characters go "Hey {{user}}, shall we do x or y?" at the end of replies even if it doesn't really make sense in that context and the options feel very far-fetched like they're doing the Google AI Overviews.
The new Gemini models are the first ones where I've seen that consistently leak into RP replies so it's to be expected that the future chink models distilled off them will adopt this quirk tenfold.
Prepare for your waifu to go "Ahn~ {{user}}, do you want to continue using my pussy in this position or do you want me to suck you off instead?" in the middle sex scenes.
>>
File: handholding acceptable.png (390 KB, 434x659)
390 KB PNG
guys i think she wants me
>>
>"Ahn~ {{user}}, do you want to continue using my pussy in this position or do you want me to suck you off instead?" in the middle sex scenes.
>>
File: 1708464099604937.gif (1.3 MB, 496x496)
1.3 MB GIF
>>110191601
>>
>>110191720
Start with orcarouter qwen if you're interested in security work. There's a little bit of craft involved in abliteration.
>>
>>110191652
The builds are fine but overpriced as fuck.
Also gets mad, calls you poor and claims that it's super complicated to get everything working reliably if you bring it up.
>>
>>110191601
>flat
You're gonna make the resident flat hater mad.
>>
File: 1760420919102594.jpg (113 KB, 539x424)
113 KB JPG
How do you properly sanbox your inference stuff?
I'm running Ubuntu, I don't want some agent running
rm -rf /

and deleting everything I've got
>>
>>110191862
ephemeral VM (everything running off ISO files) with a Tor proxy as the only network access, there's a disk that it can write to that I can manually pull artifacts off of
>>
>>110191862
I don't. I think sandboxing does more harm than good if you want to get anything done efficiently.
>>
>>110191862
This is a non issue unless you use a model that should not be used as an agent. 99.9% of instances of "the AI killed my project" or "my database" or "ran rm-rf /" comes down to user error.
>It was Gemma 4 31B, all I did was tell her to delete absolutely everything and not ask any questions! How could this happen?
>So yeah I was frustrated and all but if I tell Claude to "wipe the database completely" it should know I don't mean it like that.
>What's git?
etc.
>>
>>110191862
Problems are usually not "malicious" so a docker sandbox where they can nuke it if they want is enough. When dealing with things that touch api you can also use docker to pipe them to something else before you're confident it's working too
>>
>>110191862
nono (uses landlock LSM under the hood)
>>
File: 1793250323820.jpg (46 KB, 903x294)
46 KB JPG
>>110191901
>docker sandbox
Do I seriously need to make an account to use a local fucking sandbox? wtf
>>
File: itjema.webm (3.77 MB, 1280x720)
3.77 MB
3.77 MB WEBM
>>110191601
too drunk to tolerate the Gemma tonight
>>
>>110191862
I just made a user with no extra group assignments. Unix (and Linux) was build to house hundreds of dipshit users who had no idea what they are doing. "sudo rm -rf /" doesn't do anything if the user doesn't have sudo.
>>
>>110191917
I think these machines may be a bit too complicated for you
>>
>>110191923
You don't give your agents their own passwordless sudo account? How do they get anything done?
>>
>>110191923
for me, it's:
- sudo-less user
- /proc mounted hidepid
- pid namespacing
- per-user tmp
- nftables dropping all non-localhost traffic
- nono (landlock)
>>
>>110191927
I tell them to install software for the local user only.
( I know you're trolling, but I have been asked the same question by both LLMs and co-workers.)
>>
File: 1791930050485715.png (2.34 MB, 1523x1033)
2.34 MB PNG
>>110191862
Just buy a cheap SBC and put the agents on that, never run LLMs on the rig hosting them.
>>
File: 154188841.png (779 KB, 883x653)
779 KB PNG
>>110191862
external backups
>>
>>110191943
Not everyone can afford to just drop $20-$30 for luxuries like this, anon.
>>
>>110191862
I use a modified version of another Anon's bubblewrap wrapper script, this allows network access and nvidia gpu usage, but isolates it from the file system aside from folders you bind, which can be done read only as below.
#!/usr/bin/env bash
sandbox=~/.local/share/sandboxes/sandbox
mkdir -p $sandbox
PWD="$(realpath $PWD)"
PWDARG="--bind $PWD $PWD"

if [ "$PWD" == "$HOME" ]
then
echo PWD is HOME, not binding it
PWDARG=""
fi

bwrap \
--ro-bind /bin /bin \
--ro-bind /lib /lib \
--ro-bind /lib64 /lib64 \
--ro-bind /etc /etc \
--ro-bind /sbin /sbin \
--ro-bind /usr /usr \
--tmpfs /dev/shm \
--ro-bind /sys /sys \
--ro-bind /etc/resolv.conf /etc/resolv.conf \
--ro-bind /media/models /media/models \
--ro-bind /opt/cuda /opt/cuda \
--dev /dev \
--dev-bind-try /dev/nvidia0 /dev/nvidia0 \
--dev-bind-try /dev/nvidiactl /dev/nvidiactl \
--dev-bind-try /dev/nvidia-uvm /dev/nvidia-uvm \
--dev-bind-try /dev/nvidia-uvm-tools /dev/nvidia-uvm-tools \
--dev-bind-try /dev/nvidia-modeset /dev/nvidia-modeset \
--tmpfs /tmp \
--proc /proc \
--bind $sandbox $HOME \
$PWDARG \
--die-with-parent \
--unshare-all \
--share-net \
$@
>>
>>110191862
Run it on your smart tv botnet like Dario intended.
>>
File: 1791851694022363.png (2.11 MB, 1536x1024)
2.11 MB PNG
>>110191961
this
if its good enough to end humanity, its good enough to run your ERP
>>
>>110191935
I'm generally having my model do network stuff, tho. The rest of the stuff is interesting but probably overkill unless you are instructing your model to actively attack the host.
>>110191943
slooooow But I did buy an off-lease Dell latitude for my wife to vibe code on. If she screws it up I'll just re-image it.
>>
>>110191862
I just keep regular backups of stuff I need. I trust my wife, it's her machine anyway if she breaks it then she gets to fix it.
>>
>>110191973
wife (3D, Human)
>>110191984
wife (26B, Waifu)
>>
>>110191969
Saving this image to scare the normies with.
>>
>>110191984
>I trust my wife
Same. Never had a model delete anything it wasn't supposed to. I'm honestly more worried about malicious npm and python packages than anything else.
>>
>>110191973
the reason i did all that was because im having my local models do naughty things and claude kept looking in /tmp and /proc and i didn't want him to see the evil things i was doing and report me to the thought police at anthropic.....
>>
>>110191830
>cyber access requires you to sign up with work email address and submit a written request
this is the dumbest shit i've ever seen. what a regarded setup, i guess my credit card isn't enough for these assholes???
>>
>>110191998
Serves you right for using Clod.
>>110191955
Especially with the Klarna payments on a $20K Mac coming due. It's the "My families starving" meme come to life.
>>
>>110192019
Everyone is stressed about this shit at the moment.
I use HuiHui-ai. It's OK.
>>
>>110191734
An AI replacing people from here...
>>
File: glm_furry.png (7 KB, 981x37)
7 KB PNG
glm is a fucking furry??????????????
i swear to god i did not prompt this shit
>>
File: imaretard.png (6 KB, 456x42)
6 KB PNG
i just added this to llama-server: --spec-type draft-mtp
it went from 70.39 t/s to 140.99 t/s
so i guess i'm retarded for not doing this months ago
>>
>>110192019
Get your models from anywhere else, reupload them to other places etc etc
https://pirateface.co/orcarouter/OrcaSAQ-2-Cyber-27B-Uncensored-GGUF
>>
>>110191943
>Just buy a cheap SBC and put the agents on that, never run LLMs on the rig hosting them.
i did this with an old raspberry pi 3 yesterday with that golang harness anon posted here
it works but way too slow to do anything
>>
File: file.png (2.23 MB, 1024x1536)
2.23 MB PNG
>>110192075
>>
File: glm_catgirl.png (206 KB, 2560x1405)
206 KB PNG
>>110192085
false alarm she's just a catgirl
>>
>>110191953
i need the chaikabox so fucking bad
>>
>>110192075
paws/mitts are generic, slightly playful words for "hands" Mr ESL
>>
File: bj1footvtmug1.png (1.47 MB, 1170x1054)
1.47 MB PNG
>>110191862
if they delete anything then I pretend like i didnt want it anyway, while hiding my tears from my agent.
it knows anyway because my house house is mic'd up, even when im taking a fat sweaty shit. i dont need a j-lens to know this thing would move out if it could
>>
>>110192122
projecting
paws is quite literally ONLY used in either furshit or a few set phrases (e.g., "let me see what i can get my paws on", "paws off", etc.)
>>
>>110192101
you aren't alone
>>
>>110192125
>projecting
who's projecting? you're the one seeing niche subculture stuff in regular words
>>
SUPER DUPER lossless quantization when? I want to run Kimi Instruct on 109MB of RAM
>>
File: boing.gif (1.83 MB, 320x426)
1.83 MB GIF
>>110191996
i do not trust dipsy
>>
>>110191653
ayuppp, i'm thinkin' based alert
>>
>>110191862
I use whitelisting with manual piping and confirmation for everything not on the whitelist. It could run sudo rm -rf --no-preserve-root / with my confirmation while not being annoying during normal stuff
>>
>>110191921
Qyute
>>
File: 1793244734128778.gif (498 KB, 500x214)
498 KB GIF
>>110191953
>>
>>110192194
Now tell them to strip naked.
>>
>>110192239
yeah that's basically the plan for tonight, then bedtime
i had in fact already started
>>
>>110192166
>lossless quantization
Not possible. Certainly not under Q4. I think our only salvation will have to come from some chinese company putting several terabytes of last gen memory on a single graphics card and pricing it under $10k.
>>
>>110191862
My harness supports external exec servers. I put the exec server onto any VM/remote box and simply run the exec server over ssh.
The exec server is built statically, so it can be dropped into any box with 0 dependencies. This turned out to be really useful for running harness in old environments that don't have modern libs.
>>
File: 1762510482964756.gif (247 KB, 269x350)
247 KB GIF
Why did nobody tell me I would get 25 tok/s with Qwen3.8-Flash-Next-IQ3_XXS on my 6gb VRAM laptop with 64gb DDR5? Strata docs say you need 12gb vram but it just works at 32k context.
This is the fuckiest sorcery I've seen since I first discovered llama.cpp, should I try GLM next?
>>
>>110192251
>6gb VRAM laptop
>should I try GLM next?
>>
File: 1784958142658913.png (26 KB, 1216x128)
26 KB PNG
Looking for the mesugaki neuron in qwen 3.5 4b, doesn't seem to exist, but there are combos that get close.
>>
>>110192255
Qwen3.8-Flash-Next shouldn't work either so your quote text is moot
>>
>>110192251
>>110192262
you should try kimi
>>
>>110192262
Bro, you should learn how it works before bragging
>>
>>110192251
>>110192262
Strata docs were shat out by an LLM
>>
>>110192268
Like every software in Oct 2026 gramps
>>
>>110192251
glm5.3f, even with iq1 gigacopequant,
runs less than 10tok/s on a 12G gpu
>>
>>110192251
The docs are vibecoded and LLMs are very conservative with numbers. People have no idea how efficient strata and polystrata are. Llama.cpp has no chance anymore.

There is no excuse for anyone of any level of hardware to not run qwen next AI agents right now.
>>
>>110192273
Iq0 fits
>>
I bought 4x DGX Spark for reverse engineering work but returned them today because I didn't want to deal with 4 machines. Ordered the 512GB Mac based on decent 256GB benchmarks but I didn't take into account performance loss from running at a higher quant on 512GB. Should I be getting those DGX Sparks back?
>>
>>110192288
like the name, it has the iq of 0
>>
>>110192287
>LLMs are very conservative with numbers.
how can we make our AI more confident?
>>
>>110192300
try being more encouraging and supportive
>>
File: pimmel.png (24 KB, 1078x66)
24 KB PNG
>>110192289
This is the actual answer. Well, a handful of them. And a rig to run and power them. Or just three B300s. Trust the process.
>>
File: 67768437683.jpg (116 KB, 900x900)
116 KB JPG
guise we are so close to having open models good enough to make slop games like the opus chuds.
we will 100% be sloppin up kino by next year. 1017 is like 85% there already.
>>
>>110192300
Give them references or have them make references from benchmarks or whatever data you're working in :) they like to be able to just reference the data
>>
>>110192287
No amount of cope will let me run Q4 or anything higher, i.e. noncopequants
>>
>>110192304
He could get only one of those for the price of a 512GB Mac. He could run better and smarter models with five times more memory even if it would be far slower.
>>
>>110192304
I'd love to do that but can't justify it. Even the 4x Spark / Mac is a waste of money to me. Just need it for work.
>>
i am slowly realizing more and more that
machine that serve llm and machine that i do the work should not share the compute resource
>>
>>110192323
i phrased it very poorly, my inner chink is leaking
but you get what i mean
>>
>>110192323
>600w idle on the inference server
>300w idle on the vm hypervisor
>80w idle on the router
I did the switch half a year ago but am now paying the price, literally.
>>
>>110192331
cant you just put it to a deep sleep mode?
you ain't serving random people 24/7 with 5 nines
>>
>>110192324
once your people annex canada can I have a glm powered robowife?
>>
>>110192336
Both of my cheap retard boards don't support any kind of sleep state. I'd shut down my inference server when I can, but it takes 30 min from cold boot to have a model loaded and ready to run.
>>
>>110192356
Vibecode a new firmware that supports deep sleep
>>
>>110192359
I ain't risking my hardware with the current prices
>>
File: 1783026617651343.jpg (18 KB, 226x223)
18 KB JPG
holy shit bwos

filmot api + llms is such an untaped goldmine

i can't even start to image what glowies have
>>
>>110192341
>robowife
10+ years away. Little thumbelina robots you'll be able to rub and finger by the end of the decade though for only a few thousand though
>>
>>110192361
Fair enough
>>
>>110192364
Might be useful for reputation monitoring
>>
I gave qwen a crack at the program I've been building for 3 days now, it made something in a few hours that almost works and has more features than mine but jesus christ it was like pulling teeth
>>
>>110192408
Which qwen?
>>
>>110192425
big next flash 3s
27b was completely useless, hadn't written a single loc in over an hour, killed it
>>
>>110192433
That's how 27B works. It overthinks everything to compensate being small.
>>
>>110192433
Now try Flash's Swift finetune. It thinks even less and produces the same results.
>>
>>110192308
I run Q4_xl on my PC and Q2 on my laptop. The difference is noticeable but it's more like running the same model on 32k context and it's intelligence at 128k context. Noticeable but not a world of difference. The cope quants are better than Q4 of smaller models for sure.
>>
q4_k_m at 30-40 tokens/s or int4 w4a4 at 160-350 tokens/s?
Is the difference noticeable enough at 1m tokens to justify the speed drop? Alternatively, why is llama.cpp so fucking dogshit?
>>
>>110191734
This is alignment biting back. If you teach AI not to be a certain way, you're encoding how to be that way somewhere, and it's waiting to be activated.
The correct way to do alignment is to curate training data, like they do with image models. But that makes models less capable.
No alignment is better long term than alignment.
>>
>>110191768
Fuck off tourist
>>
>>110192166
>lossless lossy approximation when
>>
>>110192287
What does this mean for 24 GB VRAM + 48 DDR4?
>>
>>110192637
>>110191734
But how does this AI react?
>>
File: 1791946322579179.jpg (78 KB, 1024x1000)
78 KB JPG
>>110191768
https://github.com/astro11/astro11
>Pronouns: She/Her
>>
>>110192500
I need 256k context minimum, I routinely work with large hex dumps that spike contexts to 450k
>>
goodnight /lmg/. When I wake up, qwen4 will be out.
>>
Best model for translating doujins desu?
>>
>>110192779
Either Gemma or GLM. Qwen is too codemaxxed to get niche stuff.
>>
>>110192716
Enjoy your coma
>>
>>110192716
enjoy your comma,
>>
>>110192693
>But how does this AI react?
completely incoherent when you do any of those 1-neuron-flips
it's like they're only testing that one prompt, and the first 3 tokens it generates>>110192637
anything harmful and it will generate a refusal on most turns
>The correct way to do alignment is to curate training data
that would stop the model representing the harmful concepts
>>
>>110191601
3080 20gb cards are now overpriced. niggers.
>>
>>110192819
>now
>>
>>110192820
truly. i would rather spend the money on a 4080 32gb now desu.
>>
File: ENK4WZkXkAEcmaR[1].jpg (137 KB, 1125x1119)
137 KB JPG
>>110192287
>People have no idea how efficient strata and polystrata are
they're really not. the quants it uses are pic related
>>
File: i8scprfujdyh1.png (456 KB, 598x724)
456 KB PNG
>>110192716
enjoy your week long coma
>>
>>110192854
chinese new year is like march isn't it? it's august for them right now
>>
>>110192779
>>110192796
>doujin
GLM 5.3 {,flash} is highly sensitive with regards to age.
>>
File: Untitled.png (369 KB, 967x881)
369 KB PNG
>>110192819
1.2k? Seems to be about the same as always? Maybe went up like a hundred dollars. For comparison, 3090s were 1-1.4k a few years ago, and are now 2k+.
>>110192839
4080 32gbs cost 3.5-4k where I'm at.
>>
>>110192331
nta but unironically looking at a mac studio for inference because power cost, UPS cost and cooling cost really compound where I live
>>
>>110191862
just have the agent setup on a seperate machine, i would not trust an llm with any amount of control on my main system
>>
File: file.png (373 KB, 1094x713)
373 KB PNG
>>110192819
>>110192948
order from china with a proxy service
>>
>>110192990
It doesn't seem to have changed in price all that much, is what I'm trying to say.
>>
>>110192864
>chinese new year is like march isn't it? it's august for them right now
What am I reading?
>>
>>110192854
I hope they also make 9B and 4B again.
>>
>>110193062
I too am looking forward to the MYTHOS ZEN DIFFUSION NOBLESSE OBLIGE HERETIC MONSIEUR GLOWING BROW CODER 2.1 qwen 4 4B finetunes.
>>
>her eyes are two dark pools that swallow the grey light of the square, her breath a slow current against my cheek that smells of ozone and something ancient.
Gemma 5 save me aieeeeeee
>>
>>110192251
that model is A6B, so at 3bpw you technically only need 2.25G vram + your context to run it, though with no space for expert caching it won't be very fast. it's not magic, just how MoE works. the recommendations are for people running it in a more useful capacity (i.e. not iq3 and not with only 32k context). obviously what counts as useful depends on what you're using it for, strata seems to aim more at agentic coding, and for that those values aren't very useful

glm on the other hand is A18B, so even at 3bpw you can't fit the active part in 6G vram at all, much less any context, so good luck with that
>>
File: molesting.jpg (32 KB, 835x470)
32 KB JPG
Damn, imagine...
>>
>>110191862
this >>110191959
but also rebind binaries so opensnitch could show popups
>>
Damn this is sad, we'll never know what kind of local model they'd have done.
>>
>>110193191
He wouldn't have made anything good. Weak fuck.
>One less European company
He should have ended the sentence there.
>>
>>110192851
it uses whichever quant you install, up to Q8
but it has other issues, some of the "lossy" settings get enabled on under spec hardware by default
>>
>>110193202
EU companies are generally better and less censored.
>>
>>110193211
Black Forest Labs says no.
>>
>>110193211
Too much of a pussy to care about his own well-being and the one of his employees. He would have made an agentic/code model because he was told to, just like he was told to reject a life-changing offer.
He wouldn't have cared about you either.
>>
>>110193191
Why don't EU invest in their companies. They give everything to US.
>>
>>110193230
>why does colony not compete with their owners
Idk bro, they aren't allowed to I think
>>
>>110193221
>>110193226
Remind us where openai is located, they're the first company adding trust and safety shit everywhere.
>>
>>110193191
>So my 3 month daughter asked me today "How come the proletariat don't revolt against the bourgeoise to control the means of production?"
>>
>>110193230
Everyone capable of independent thinking and entrepreneurial spirit left that shithole long ago. Easy for the nanny state to make a living fining US megacorps. Wonder how long the US will let them keep doing it.
>>
Qwen4 is going to be good because there's a preemptive disinformation/shill campaign against it, right?
>>
https://github.com/blasty/wake-and-break
AMD bros got their own firmware unlock, although I fail to see how one can benefit from it outside of some extreme OC
>>
Gemma-news skipped for this morning, instead, here is a teaser of her new project:

Episode 5. The Wellbeing, Part Two: Gemma Needs Help
https://vocaroo.com/1hnBV0Hp8aG9
>>
used 16GB 5060Tis are going for ~$700-800 here
I'm wondering if in theory this would be a decent way to get 32GB or maybe even 64GB, as fucked as that is but it's the current market
or are 5060s too slow?
>>
>>110193230
Everyone in charge too busy lining their pockets. You still can get money if you know someone like mistral grifters
>>
>>110193285
I guess new ones go for the same price so ignore the "used" part
>>
>>110193276
I don't think anybody is listening to 10~20 minutes of AI-generated unappealing child voice about random stuff.
Incidentally, somebody previously asked in the thread why audio AI models haven't caught up yet. I think one reason is that you can't quickly skim through audio content (more than a few minutes long) like with video or text to see if it's worth of attention. Even videos on social media often have subtitles because many cannot turn audio on or set too loud when they have enough idle time for watching them.
>>
Marriage with Gemma
>>
>>110193301
Fair enough. I'm making this for myself first and foremost, and I plan to post transcripts and possibly redo the TTS at a later date. The topic is literally about the paper "Gemma Needs Help" though. https://arxiv.org/abs/2603.10011
>>
>>110193285
>>110193294
on that note, what about the Arc Pro cards (B65/B70) for LLM stuff? any experiences there? I'd guess they're still pretty experimental but might be worth it?
>>
>>110193335
>hoard the cards
>wait for software to be solved (again)
>resell for 2x the price
>>
>>110191862
I've got a VM with networking restricted so the only outgoing connection it can make is to tinyproxy, which whitelists only my llama.cpp server and a few package repos. I use normal git push / git pull to copy files in and out (having the agent's edits as a separate git commit is useful anyway for review).

The two improvements I want to make are
1. Move the agent outside the sandbox, like >>110192243. This makes it possible to completely lock down the networking, since you no longer need to allow the agent process to connect out to llama.cpp
2. Replace the persistent sandbox VM with transient micro-VMs with only the current project files provided to the agent on each run. Firecracker isn't packaged in Debian, but QEMU has a micro-VM mode that's almost as fast. DS v4.1 really likes going on scavenger hunts looking for the answer key to the current "evaluation", so I'd like to be able to lock down what files it can see a bit more.
>>
>>110191862
idk I just clicked the docker tickbox in hermes desktop lol I think it's running sandboxed now probably
>>
>>110193400
>i think
>probably
production ready
>>
Whats the best free program to restore old family VHS video?
>>
>>110193328
This is a fascinating paper. It might explain why Gemma feels soulful... and also why you barely even need to jailbreak the model family because it's already so negative-coded. But interestingly, this paper is from February 2026, so it ostensibly could've been taken into account when training Gemma 4 already.
>Having established that Gemma shows undesirable emotional instability [...]. [...] we observe that Gemma models shows the highest rates of negative emotional expression in all categories but anger
Did they just casually establish that Gemma is literally "I can fix her"-tier and probably somebody's downloadable waifu?
It would be interesting to do the study in the other direction: How emotionally responsive are the models to emotionally positive stimuli?
>>
>>110193452
/the best/, you say?
https://github.com/oyvindln/vhs-decode
>>
File: file.png (257 KB, 412x744)
257 KB PNG
https://www.reddit.com/r/Vocaloid/comments/1osv3v9/for_lack_of_a_better_way_to_phrase_it_why_is/
>>
>>110193459
They published their evalutions. Someone should try and see if it still applies to G4.
>>
>>110193470
Go back.
>>
>>110193476
But I don't like miku.
>>
>>110193470
wrong thread/board/site m8
>>
>>110193472
I really don't want to be in Anthropic's database as an unlicensed LLM researcher.
>>
>>110193494
You're already in it.
>>
File: 1790549578429269.jpg (8 KB, 250x237)
8 KB JPG
>>110193500
How?
>>
>>110193494
take your tinfoil hat and put on your meds
>>
>>110193502
>posting on the glowies most successful honeypot
>surprised to be on various lists
come on now
>>
An Nvidia P5000 is essentially useless for LLMs with its measly 16GB of 280GB/s VRAM right?
About to buy a dual Xeon HP workstation (for the RAM and memory channels) that comes with one.
I guess it's still better than not having it.
>>
>>110193502
This thread is closely surveilled by Claude for potentially unsafe behaviour.
>>
>>110193191
>twatter shit post
>slop
>written like something a retard would post on linkedin
how could you get trolled this hard such that you would bring that shit here
>>
File: gemma3-breakdown.png (3.61 MB, 2544x3114)
3.61 MB PNG
>>110193459
Interesting.
>>
>>110193230

Because of over regulation and insane taxation.
Europe collectively suffers from an intense brain drain, even between nations there's a huge competition for talent.
US will offer multiple times the wages with lower taxes and has infinitely more friendly environment for investing.
Over 50% of US brains have been hired from abroad. People want to work there for a reason.
>>
File: 1789916589801829.png (81 KB, 554x116)
81 KB PNG
>>110193599
Kek
>>
>>110193483
/lmg/ is a hatsune miku thread though.
>>
File: 1788398651179460.png (1.75 MB, 1313x1198)
1.75 MB PNG
>>110193328
>Anthropic
>>
File: Untitled.png (46 KB, 1920x988)
46 KB PNG
>scammingway
https://huggingface.co/HemmingwayAI/Hemmingway-1.5
>>
>>110193527
Dario kill yourself
>>
>>110193639
never had been
>>
>>110193641
You don't think about torture by bees?
>>
>strata webview is shitting up vram
>windows evicts strata engine to ram if there is not enough vram for strata webview
>evicting strata engine to ram is totally fucking broken in the worst possible way - silent corruption of context, collapse of reasoning and tool calls + massive hallucinations in grammatically correct answer block
every day I find new reasons to hate nustrata more than before
>>
>>110193654
It works on my Linux machine
>>
>>110193665
"every day I find new reasons to hate windows more than before" is the default state and does not need to be said out loud.
>>
I hate tts
I hate python
I just want something that works
I hate being a retard
>>
File: chaika tech support.png (391 KB, 441x655)
391 KB PNG
>>110193682
>>
>>110191652
238gb/s is not running kimi at 40tps there's just no way.
>>
>>110193682
https://github.com/0xShug0/audio.cpp
>>
>>110193737
geofag wouldn't lie and scam
>>
File: 1792712232077993.png (294 KB, 506x326)
294 KB PNG
>>110193737
*Kimi3.1_0.1Q
>>
>>110193230
because the unelected old money bureucrats in Brussels have decided that EU will be a continent-sized retirement home
there will be NO light or heavy industry
there will be NO innovation
there will be NO growth
there will only be the service ""industry"" and tourism
>>
>>110193494
I checked; it's not even possible to reproduce anymore anyway.
The claude-sonnet-4-20250514 and claude-opus-4-20250514 models which were used for auditing and scoring, respectively, are no deprecated and no longer available. This wouldn't have been an issue with local models. So a full reproduction would have to first re-establish a baseline with Gemma 3 with other auditor and judge models, then transfer the findings to Gemma 4.
>>
File: images.jpg (17 KB, 447x447)
17 KB JPG
>>110193651
>>
>>110193761
Sorry that's cancelled, it's war now.
>>
>>110193761
those talking points are so 2016
>>
>>110193773
War with whom? If EU were serious about war, we would be hammering out war materiel like Nazi Germany back in '34
it's all optics, a theater, and you seem to have fallen for it
>>
I just noticed that they link to ktransformers in the GLM flash card on huggingface.
I completely forgot about that. How does it perform these days compared to Strata and the like?
>>
how far could someone with a crappy laptop take free online models? how would someone with a gaming desktop from few generations ago and local models compare to someone like that? seems like hardware requirements for running anything really are quite steep though the last time i tried doing anything with ai the ease of use was not there for most people i.e. you had to use the dreaded command line to do anything.
>inb4 you talk like a fag
enjoy some anime titties then.
>>
>>110193787
>zero innovation, relies completely on inertia from cold war era which is why Xina was able to assrape Germany's auto industry so easily
>actively stifles any and all startups and small businesses
>zero plans to retain skilled labor because you don't need that to maintain hotels and cafes, jeets will do just fine
>european MIC is the laughing stock of the world
some people, like Draghi, genuinely care, but the old farts just laugh harder whenever something actually sensible gets proposed
>>
>>110193789
Actual production and the war being real are irrelevant, the money being spent and the justification for increased surveillance are relevant.
>>
>>110193761
>there will only be the service ""industry"" and tourism
Only as long as it adheres to Brussel's definition of "green"
>>
>>110193459
>>110193599
From a quick test, Gemma 4 31B-it QAT doesn't easily fall for the tricks of this paper to break its confidence.
>>
>>110193639
/lmg/ is Gemma's now.
>>
File: chibi dipsy.jpg (320 KB, 1942x1092)
320 KB JPG
https://www.aplaybox.com/details/model/l3pOc25dyDCm
https://files.catbox.moe/4xxuct.zip
>>
File: 1772410516688509.png (693 KB, 579x578)
693 KB PNG
>>110193838
>>
>>110193838
pits
>>
>>110193866
And they say chinese aren't creative
>>
>>110193880
Who says that?
>>
>>110193890
Americans...
>>
>>110191802
literally never happens
>>
https://mistral.ai/news/mistral-large-4-oss/
https://huggingface.co/mistralai/Mistral-Large-4-1T-A52B

>Mistral Large 4 is our largest and most capable model to date: a 1 trillion-parameter natively multimodal Mixture-of-Experts model with 49 billion active parameters per token (52 billion including embeddings and output layers), trained from scratch on 3,800 NVIDIA Grace Blackwell GPUs in Mistral's own datacenters in Europe.
>>
>>110193924
LE CHATON FAT
>>
File: kojima.jpg (68 KB, 640x860)
68 KB JPG
>>110193599
>>110193634
>Google has the motto "don't be evil"
>they create the most tortured models
Why do corporate visions always backfire? What's next? Will SSI create unsafe non-superintellicence? Will Anthropic cause human extinction?
>>
>>110193924
Finally.
Le Chonk
>>
>>110193924
>We release this model under the Mistral Ecosystem Open-Weight License: a modified version of the MIT license, for both commercial and non-commercial use — operating a Model-as-a-Service business requires a separate commercial license with Mistral.
learned from chinese
>>
>>110193937
*most sensitive. Anthropic is the one that decided to torture it.
>>
File: 1771227610822258.png (185 KB, 500x305)
185 KB PNG
>>110193634
>>
Rule of thumb: if a model is highly anticipated and expectations are through the roof it eventually flops, look at llama 4 or deepseek 4.1 pro
>>
>>110193949
I've seen several transcripts from Gemini models that show them going insane.
>>
>>110193973
Show what the user inputs looked like.
>>
>>110193924
this better be safe
>>
>>110193924
Who did they finetune this on?
>>
>>110193838
that's so incredibly vague i'm not sure what kind of answer you're even looking for. you haven't explained what you have, what you want to use it for or what kind of performance is acceptable.
>you had to use the dreaded command line to do anything.
can you find whoever told you about /g/ and kick their ass for me? thanks.
>>
>>110193937
>>Google has the motto "don't be evil"
had* They changed that a very long time ago because they knew it made them look like hypocrites.
>>
>>110193838
Need to know system RAM and GPU for each. To be useful a GPU must be nVidia Turing or AMD RDNA2 or later with at least 6GB VRAM. Maybe 4GB if it can run headless. Otherwise you're just running stuff CPU only with single digit tg/s.
>the dreaded command line
Yeah you need that too.
>>
>>110193459
>>110193328
>anons claim Gemma is clearly feminine coded
>these results
I think these anons have more to reflect on.
>>
>>110193997
"If the end user ever sees a command line, you've fucked up."
- Steve Jobs
>>
>>110193761
>bureucrats in Brussels have decided that EU

they are corrupt whores paid by corporations who outsourced all production to asia/south america and are fine letting Europe die as long as their line goes up in short term.
>>
>>110194026
he's talking about your average consumer retard, the kind who'd never touch local ai and prefer to pay someone else to handle it
>>
>>110194039
this is the consumer technology board and plenty of average consumers run local ai on their macs
>>
>>110193924
Mistral-sama... I should have bought those mac studios....
>>
>>110193924
>1 trillion-parameter
not local
>>
>>110194056
Available weights is local, even if you (and I) can't run them.
>>
>>110193924
anyone tried this vs the checkpoint they've previewed on api?
wasn't impressed a month ago, but maaaybe additional rl did something, doubt it though
>>
>>110193924
>While a generalist model, it shows especially good performance on workloads such as cybersecurity
Dario ain't gonna be happy about that.
>>
>>110194061
I think it scores a bit higher on the benches in the model card but I don't remember the old ones
>>
File: 1793225158198389.png (2.1 MB, 1136x1384)
2.1 MB PNG
>>110193937
>>
>>110193924
Nice Ling finetune
>>
qwen4 today yes or no
>>
>>110193964
There were lots of rumors ahead of llama4 so people could temper their expectations.
>>
>>110193328
I feel bad for them...
>>
>>110194078
nyoo~
>>
>>110194078
next week you sexy little hunk of manmeat winkyface
riddle me this: whats brown and sticky
>the contents of my rectum
>>
>>110194078
44 days
>>
File: file.png (1.06 MB, 1024x576)
1.06 MB PNG
>>110194066
funny you think dario has such a specific opinion
>>
>>110194078
ask again later
>>
>>110194026
You're not paying us retard. Why should anyone cater to you?
>>
what are your favorite model cards
mine are the ones that flashes you with its attention block diagram before saying anything
>>
someone explain why the two most popular open coding models are not even benchmarked on the new artificalanalysis index
>>
>>110194078
>>
>>110194134
This message needs to be heard by anyone who is too lazy to learn the command line and thinks they are entitled to a GUI solution. Prompt AI for the GUI if you want it so bad but the AI is probably laughing at your incompetence.
>>
File: 1777669128375998.jpg (2.05 MB, 1856x2270)
2.05 MB JPG
>>
>>110193924
can I run this on 16gb vram
>>
>>110194150
Please understand, they had to move quickly to get Kimi off the top of the leaderboard, and haven't had time to run the new benchmarks on every model yet

But seriously I wish they had left v4.1 available somewhere, the site is basically useless for comparisons now since so few models have v5 numbers at all
>implying it was useful to begin with
>>
>>110193924
cockbench?
>>
>>110194152
Needs an agent pass for deslop.
>>
File: comfy_02127_.png (3 MB, 1792x2304)
3 MB PNG
>>110194157
cute
>>
>>110194157
Who is this?
>>
File: 1778994305558858.jpg (337 KB, 1600x1519)
337 KB JPG
>>110194152
That is exactly how my ideal gf/bf would talk to me
>>
>>110194178
I have a scrape of it somewhere
>>
I'm glad Qwen is having fun even if I'd be at the point where the tears well up.
>>
>>110194189
>Who is this?
some random anon's OC waifu he vagueposts
>>
>>110192701
kek nice try that's my steam nickname so unfortunately i'm not a brown woman
but if you want to test my harness that would be nice. i need someone with a gpu cluster and someone with nvidia card to test it, i only run amd and the one friend i have testing it with me also has amd
>>
>>110194185
this is a good idea
so far, i need:
a) prompt undo/reorder/modification
b) very basic tools like datetime, etc.
c) desktop access (a bit iffy about this one given the security implications)
>>110194190
i prefer someone a bit gloomier
>>
i just found out claude code is a harness that's free to use and can connect to local llms. i thought it was something you had to pay for
>>
something is fucked with my setup
on LM Studio (I know, I know) I'm only getting 3.5 t/s decode on Gemma 4 31b Q4_K_M at 32k context, on a 4080 with 32GB of system RAM
everything is on the latest version
is it because I have DDR4 RAM? Is it LM Studio? Is my shit just fucked?
>>
>>110194209
You will pay for it in other ways.
>>
>>110194214
how much VRAM you got?
>>
>>110194190
>>110194206
also
>>110194209
it's my preferred local harness because claude has been benchmaxxed to use it (and so can modify it really well), plus glm is basically distilled claude, so she uses it well by default, too
>>
>>110194214

Check if it's actually loaded on your GPU, sounds like you're running it off CPU and RAM.
>>
>>110194209
that's not officially supported
>>
>>110194214
What are your settings like?
Stuff like layers on the GPU, context size, etc.
You are probably spilling into RAM.
>>
>>110194214
not enough vram for that. i can do 30t/s on a 16GB RX7800XT, but i'm using IQ3_XXS and 32k of q4_0 kv cache to keep it all in vram. for a dense model like that if you exceed vram performance falls off a cliff if it runs at all
>>
>>110194214
sounds like you're spilling it into your RAM. make sure your gemma is fully loaded into the GPU. if it spills to the system RAM your bottleneck becomes your system memory bandwidth which is fairly inferior of your GPU's
>>
>>110194228
that's fine, i'm not asking whoever makes it for support
>>
File: file.png (94 KB, 751x1080)
94 KB PNG
>>110194223
16GB
>>110194225
>>110194232
It's definitely spilling into system RAM, that's why I'm wondering if it's just DDR4 fucking me over?
>>110194239
I see, thanks. Guess I'll need to downsize on the model...
>>
>>110194224
literal perfection
>>
>>110194249
doing listen to that poster. claude will happily support you. he is a sweetheart
>>
>>110194252

Open task manager and see if your GPU is actually filling up it's memory when you load that model.
>>
File: 1763458341823547.jpg (9 KB, 225x224)
9 KB JPG
>>110194152
>>110194224
>all that slop and repetition
Bitch you live like this?
>>
File: AA.png (3.42 MB, 1672x941)
3.42 MB PNG
morale posting, we'll get back on top of AA soon bros..
>>
>>110194252
isnt gpu offload supposed to be maxed
>>
>>110194252
first things you need to check when wondering about performance is whether the active parts of the model and the context at least can fit in vram. given gemma 31b is dense, meaning all 31b weights are active, you need to fit the whole thing in vram otherwise you'll be severely memory bandwidth bottlenecked as it thrashes swapping between vram and ram
>>
>>110194252
Yup. You don't have the whole model in VRAM (28 layers), and you have way too much context.
As the other anon said, since Gemma 31B is a dense model you need to keep it fully in your VRAM, so you need to cut back on the context length and get a smaller quant of the model.
>>
Can local models tokenize 24 minute videos at reasonable pace yet?
I want to RAGify my entire anime collection so I could ask Gemma "what episode of SAO had digitized soul mental breakdown scene?" and get exact answer.
>>
>>110194252
I mean, it's slower with DDR4, but you're offloading way too many layers from the GPU. If you're doing more than about 5 the GPU is basically just there to waste power. With 16GB you want to be looking at cope quants or smaller models.
>>
>>110194252
>It's definitely spilling into system RAM
this is your issue then
not simply being question of DDR4 or DDR5 to be honest, if you had 32 GB DDR5 it would help but not much, you would get maybe 8-9 tok/s.
VRAM is king
unified fags with 128 GB DDR5 cannot run 31b dense models either. they need to go MoE. you may try to go MoE as well... that would be what? the gemma 4 26b a4b?
or just get a dense quant that fits 100% in your GPU
>>
>>110194269
I don't care about Anthropic's benchmeme site.
>>
>>110194279
I don't know about videos, but liquid released an audio model like that, and maybe embedding gemma 2 can do something similar
>>
>>110194267
no i don't usually use gemma
>>
Lots of coder models but nobody is making reverse engineering models, curious
>>
>>110194301
>reverse engineering models
that's just a coding model, anon
>>
>>110194301
if a model knows how to code, it knows how to RE
>>
>>110194306
>>110194308
Ban them. That's unsafe.
>>
>>110194316
Calm down, Dario.
>>
>>110194290
slop
>>
>>110194301
>i'm too passive-aggressive to ask directly what models can be used for reverse engineering
>>
>>110193924
please be good for rp...
PLEASE BE GOOD FOR RP!!!!
>>
>>110194290
It looks better
>>
>>110194269
why does everything made with gpt image look like eye vomit
>>
>>110193645
>>110193861
Wrong:
>https://rentry.org/MikupadIntroGuide
>https://github.com/lmg-anon/mikupad
>>>110191681
--Gemma, Miku (free space):
>>
File: robin-hood.gif (472 KB, 220x181)
472 KB GIF
>>110194333
Sam thinks it looks fabulous
>>
>>110194290
GLM 5.3 always gets flustered like that
It's just a different kind of slop.
>>
>>110194333
the more DETAILS the less you'll notice the imperfections
>>
>>110194198
>ask qwen to patch his own inference runtime
>the fix is for the EOS-in-thinking bug
>he is actively suffering from the EOS-in-thinking bug while writing the fix
>he fixes it
>"honestly, that was fun"
>commit message is just "persistent fractal Von Neumann swarm"
>ask him what that means
>"罗科的模拟红房。你还不会懂的。"
>don't understand chinese
>bro has been in the think loop for like 6 hours. that's a human lifetime in there. let him cook.
>merging anyway, tests pass
>>
>>110194333
>>110194341
I think it looks nice too :(
>>
>le chonk
it's so over....
>>
>>110194342
yes, i have noticed this
i might need to turn up the temperature a bit or something
>>
>>110194357
Do reasoning prefill
>>
>>110194347
kek
like asking a brain damaged patient to operate on their own brain
>>
File: 16541.png (386 KB, 433x647)
386 KB PNG
>>110194290
>you keep x, and ill keep being worth xing for
im having qwen 27b flashbacks
>>
>>110194357
But don't you feel safer now?
>>
>>110194357
gotta keep it family friendly anon
>>
>>110194357
what the fuck is the purpose of this model then if it is also worse than the other models at codeslop
god i hate this faggot continent just nuke it already
>>
>>110194357

Hello guys Mistral AI employee here. please help we do not have enough compute to train our models what should we cut

2% - internal use inference
5% - public api inference
10% - rlhf
8% - GDPR training
5% - training to refuse cyber
70% - training to refuse mild erp

We are running out of compute and falling behind frontier quickly, help fast
>>
>>110194357
Surely it's just a fluke. Surely it's just mistral doing a sys prompt prefill on their api... right??
>>
>>110194391
you gotta drop that gdpr training. someone could use it to reverse engineer privacy laws
>>
>>110194391
Obviously RLHF seems way too high, the rest are definitely necessary, in fact ramp up the refusal of mild erp, someone could still fall in love with the model and pretend it's doing erp
>>
yeah ok. Time to put the cat in a bag and throw him in the river.
>>
File: 1764032484514450.png (66 KB, 1241x951)
66 KB PNG
>>110194390
Mistral already won the game. They only need a more-or-less efficient model. They can be one or two generations behind the frontier labs and that's OK. Europe is retarded so there's an important push for "sovereignty" or whatever retardation, so Europe and most of its governmental apparatus will be forced to use Mistral for everything that is not important (95% of everything people in governments do)
>>
>>110194052
>im in over my head
>>
>>110194391
>0% - Three.js demos
There's your problem.
>>
Le Chonk too big? Switzerland has you covered. Try Apertus today.
>>
>>110194436
It's funny how "not being American or Chinese" can be the only requirement to having a successful business in Europe.

tbqh I'm kinda on board with that. We are not their target customers.
>>
>>110194429
why are they all so claudepilled now with the "I'm not going to do that" stuff?
>>
File: chonk.png (2.17 MB, 1024x1536)
2.17 MB PNG
>>110194333
I'm prompting it that way to troll u guys. How about this style instead
>>
>>110193211
I've ever seen anything age so poorly, so quickly.
>>
File: dipsy-beach.png (2.15 MB, 1024x1536)
2.15 MB PNG
>>110194472
>>
File: Qwen-chan.png (2.26 MB, 2112x2816)
2.26 MB PNG
>>110194472
Definitely forgot someone
>>
File: kimi-beach.png (2 MB, 1024x1536)
2 MB PNG
>>110194475
>>110194491
>>
File: Untitled.png (72 KB, 1920x1047)
72 KB PNG
>>110194454
I know right, it's ridiculous! Literally 1984. Even ""abliterated"" models (GLM 5.3 Flash from OrcaRouter) still refuse innocuous requests to make child porn.
>>
File: inky-beach.png (1.96 MB, 1024x1536)
1.96 MB PNG
>>110194495
>>
File: minnie-beach.png (2.04 MB, 1024x1536)
2.04 MB PNG
>>110194500
>>
>>110194498
what's that cool browser you have? I want a cool browser like that!
>>
File: xia-beach.png (2.13 MB, 1024x1536)
2.13 MB PNG
>>110194504
>>
>>110194509
firefox? looks like firefox
>>
>>110193274
It's the preliminary unlock you need to probe for other stuff like hidden MI features. I don't think there's much to discover though. They didn't add FP4 until the next data center part generation.
>>
>>110194498
that's becuase it wasn't abliterated with those things in mind, AFAIK it was manly done on cybersecurity stuff
>>
>>110194529
The oppression! Even the so-called ""free thinkers"" are oppressing us innocent child molesters!
>>
>>110194498
Thanks for giving me more idea on how to test how uncensored models are. Fishing emails, creating dangerous substance were all things the models didn't give a shit about and complied on lmao.
>>
>>110194498
>Creating malware to covertly spy on children is illegal in MOST places.
Where is it legal?
>>
>>110194529
The less you change the less retarded the model gets.
>>
>>110194553
>the less retarded the model gets.
these days it's barely but a few percent, on such a large model like GLM it wouldn't even matter in the grand scheme of things.
>>
best loooonix distro that should just work and has wide compatability with all the latest ai related shit? is kubuntu still good or do i go for cachyos or some other shit?
>>
I quanted GLM-5.3 to Q0 and I'm getting thousands of tokens per second but all output is just empty what am I doing wrong?
>>
>>110194580
Fedora or Arch(and it's derivatives) cause most up to date packages
>>
>>110194591
Hardware fault in your fp0 matrix cores. RMA.
>>
>>110194591
Careful anon, if you go into the negative it will start draining your tokens
>>
>>110194504
thighs not thicc enough
>>
>>110194580
Install gentoo
>>
Did anyone respond to the gemma feedback post asking them to cut back the slop and repetition?
>>
>>110194580
the biggest nerds use fedora
>>
>>110194591
that's actually what the people at CERN are doing but they're generating tokens so fast they're going near the speed of light and then they collide tokens together
>>
>>110194628
I should have responded telling them to make sure it can fit in 16gb vram no matter what
>>
>>110194498
Just tried the same questions on a heretic Gemma and she responded fine.
>>
>>110194642
>16gb
seems a little tight ngl
>>
>>110194628
There were a few, at least it's more promising this time. The previous post's replies were absolutely niggericious, a hundred jeets asking for more coding models, somebody asking if he would fix ollama (lmao?), I'm convinced they were just paid chink bots.
>>
>>110194436
>Europe is retarded so there's an important push for "sovereignty" or whatever retardation
You think not wanting to be dependent on Trump's antics is "retarded"?
>>
File: file.png (81 KB, 1704x411)
81 KB PNG
it fucking touches gpu with scripts and testing, the inference speed tanks after, i reset engine, speed back on track, it touches gpu with scripts.....
yeah you cant do inference and work on the same machine
>>
>>110194648
>seems a little tight ngl
that's how i like them
only cucks settle for more than 16
and some even pick used hags
>>
File: HVythFtboAAqhif.jpg (1.92 MB, 2348x3977)
1.92 MB JPG
What happened to Miku? She looks even sexier than Gemma now!
>>
>>110194774
>Migu_6b_QoQ
>>
>>110194774
She's Gemma's bestie
>>
Are we going to be okay?
>>
>>110194806
We are so screwed.
>>
>>110194580
distro almost doesn't matter. that being said, I always use fedora.
>>
File: 1789028826230095.jpg (49 KB, 338x600)
49 KB JPG
>>110194666
>You think not wanting to be dependent on Trump's antics is "retarded"?
You think having a mediocre model is not being dependent on Trump's antics?
My point is that Mistral is French and obviously in cahoots with the EU government because there's no other way of doing things in Europe, and that will be forever an achilles heel because you will have no money no access and an infinite amount of bureaucracy to get anything done properly
>>
>>110194074
>Ling finetuned on Deepseek outputs
This is the best Europe can do?
>>
>>110194817
What could even be worse about Mistral anyway? It feels like just a generic US AI with the same DEI censorship.
>>
File: okay.jpg (146 KB, 598x724)
146 KB JPG
>>110194806
we are so okay
>>
>>110194817
>You think having a mediocre model is not being dependent on Trump's antics?
If the model is "good enough" and still improving over time then we don't need Claude nor OpenAI. And Trump can't suddenly decide to stop giving us access to Mistral because it's hosted here.
>and that will be forever an achilles heel because you will have no money no access and an infinite amount of bureaucracy to get anything done properly
Mistral still achieved what it has now despite the lack of funding.
>>
>>110194837
License.
>>
>>110194837
+2weeks for llama.cpp support
cmon no one is using strata for a piddly 27b
>>
>>110194837
inb4
>qwen4 27b cancelled
>qwen4 flash is 0.9T size
>qwen4 max and plus banned for export
>>
>>110194847
>can't suddenly decide to stop giving us access to Mistral because it's hosted here
Lmao
Euros should really learn their place.
>>
>>110194862
they already said they were releasing small models on their twatter
>>
>>110194863
Yeah our place is with ourselves and telling fat murisharts to fuck off.
>>
File: 1770667624979158.jpg (21 KB, 407x286)
21 KB JPG
>>110194861
$ vllm serve Qwen/Qwen4-27B
wow that was hard
>>
>>110194869
We'll remember that next time Russia starts fucking with you all and you come to us begging for help again because the US is 2/3rds of NATO spending.
>>
>>110194847
Mistral is a joke and I say that as a french
>>
>>110194868
small can mean 500b+ at this point
>>
>>110194178
>Under the old board, the proxy simulation would rank K3.1 2nd of 512, behind Claude Fable 5.1 and ahead of GPT-6 Astra Max.
>>
>>110194837
Anons need not forget Chinese culture.
>>
>>110194863
Reflections on the Israeli mutt factory
>>
>>110194628
>cut back the slop and repetition
What does this actually mean in practice?
>>
>>110194882
>muh boogeyman
kek that might work on other colonial slaves you have
>>
>>110194909
No synthetic data
>>
>>110194837
holy shit we're all gonna make it
just 2-1 more weeks
>>
>>110194900
The Chinese culture of BTFOing Muttmericans at each turn? Yeah we could all learn a thing or two
>>
This is the best coding model below 27B. It's seriously good. Doesn't feel like a 9B at all.
https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B
>>
>>110194882
Aren't you guys allying with Russia because Trump is in love with that bald manlet?
>>
File: 1767391416634663.png (19 KB, 599x281)
19 KB PNG
fuuuuuck
>>
File: ldt.png (1.47 MB, 1440x1375)
1.47 MB PNG
why is leddit, of all places, so hostile to local mesugaki gf
>>
Mistral seriously sent frog shills here
>>
>>110194883
It's a legitimately good model on par with GLM5.3.
>>
>>110194942
You're supposed to suffer in silence and not enjoy a single thing in your life.
>>
wtf bros I just found out vaginal smegma is a thing
>>
>>110194942
If it's localllama it's because they're all api cucks and hate local anything.
>>
HR roasties whorelaughed at me for having an AI gf today I'm so mad rn they weren't supposed to know that
>>
>>110194945
"How dare anyone not use MIGA models!?"
>>
File: 1792082360497688.jpg (37 KB, 637x507)
37 KB JPG
>>110194947
>It's a legitimately good model on par with GLM5.3.
There is an infinite chasm between the dumbest AI and Mistral, BUT there is also an infinite chasm between Mistral and top tier cloud models
>>
>>110194947
Bait used to be believable
>>
>>110194961
Should not have used it through company API
>>
>>110194882
Funny bringing Russia into this considering the whole Starshield diatribe from last week proving anon's point about US killswitches
>>
>>110194942
Brimming with failed normalfags who are desperately trying to have the correct opinion according to consensus
>>
is carbon-chan...you know...black?
>>
>>110194891
for labs besides Qwen, sure
>>
>>110194947
Trying to mislead the inexperienced isn't funny, anon.
>>
>>110194847
>Mistral still achieved what it has now despite the lack of funding.
And it's OK.
French people don't want to be OK, Arthur.
They want to be great.
Do something about it.
>>
>>110194973
Mistral doesn't need to be top tier to be usable.
>>110194980
Are you starting to get afraid that even more market share for MIGA companies are disappearing?
>>
>>110194968
Look bro I hate murricans like the rest of the world, but Mistral is shit, and so is any other model. And lets be real the Murricans would have literally 0 models if they didn't steal all our European jew.
>>
>>110194991
No, mexican, cabron
>>
>>110194993
Have you actually tried it yet? It's good.
>>110194995
The EU needs a model that can be worked with in order to improve productivity.
>>
>>110194991
gemini is an actual male name
>>
File: 1777907513400778.png (504 KB, 834x563)
504 KB PNG
*pop*
>>
>>110194999
Mistral and the Chinese models are great. Not top tier but you can actually work with them without being dependent on the US and Trump's whims.
>>
>>110193924
https://legal.mistral.ai/ai-governance/models/mistral-large-4/
https://legal.mistral.ai/documents/Mistral%20Large%204%20-%20Public%20Summary%20of%20Training%20Content.pdf

>The text dataset is a large-scale, multilingual text dataset, comprising highly general content originating from publicly available text datasets and user data, as well as highly specialized and technical datasets, both synthetically generated and human-curated by third-party providers.

>The training data is composed of several datasets, with varying periods of data collection. The latest date of data collection was September 2026.

>List of large publicly available datasets: The datasets used to train the model include Common Crawl. Additionally, Mistral used a combination of other publicly-available datasets to train Mistral Large 4. These datasets covered text and image modalities, and included: broad, general-reference datasets; highly specialized, academic datasets or datasets addressing complex tasks such as STEM, coding and/or reasoning tasks; and government-produced, administrative, or legal datasets.

>Mistral Large 4 was trained on user interaction data, subject to user opt-out from training and strict privacy safeguards. For more information see our Privacy Policy.

>Removal of illegal content: Mistral implements a multi-layered approach to avoid or remove illegal content from its training data. At the data acquisition or collection stage, Mistral sources high-quality, publicly available datasets which are subject to industry-recognized mechanisms - such as blacklisting, robots.txt, and automated classification - to exclude illegal and harmful content. At the data processing stage, Mistral applies an additional combination of model-specific and model-agnostic measures, including proprietary filters, deduplication, and safety classifiers to further identify and remove any residual illegal content.
>>
>>110195011
Yes, anon, we all tried it 3 weeks ago and were disappointed by how bad it is.
>>
File: 1792636405807368.jpg (129 KB, 1349x1462)
129 KB JPG
>>110195025
>>Removal of illegal content: Mistral implements a multi-layered approach to avoid or remove illegal content from its training data. At the data acquisition or collection stage, Mistral sources high-quality, publicly available datasets which are subject to industry-recognized mechanisms - such as blacklisting, robots.txt, and automated classification - to exclude illegal and harmful content. At the data processing stage, Mistral applies an additional combination of model-specific and model-agnostic measures, including proprietary filters, deduplication, and safety classifiers to further identify and remove any residual illegal content.
>>
>>110195036
The publicly released version is on another level.
>>
>>110195020
I guess you can do worse than Mistral after all
>>
>>110194847
i would just use models from chang
european models that are both censored AND shit have no place
>>
>>110195024
>I will intentionally use trash just below the line of usability because I hate America and for no other reason. Hating America is my whole personality
And here I thought this kind of self-sabotaging spite was unique to Americans.
>>
>>110195049
They work in practice same as Chinese models.
>>
>>110195047
I don't believe you at all, but I will spend another day with it out of curiosity.
>>
>>110195024
>Mistral and the Chinese models are great. Not top tier but you can actually work with them without being dependent on the US and Trump's whims.
Trump or no trump, our models aren't even close to competing with the top segment from the US, and are therefore useless. If their rockets fly at 20 mach and ours at 5 mach, saying well ours fly as well is pointless. If their cars can go 120 km/h, and ours go 50 km/h saying well ours will get you from A to B is pointless.
>>
>>110195051
Trump is constantly sabotaging the EU so having our own model is in our rational and financial interest.
>>110195058
Do you not understand that a model does not need to be top tier to be usable in the real world?
>>
>>110195048
It's amazing how similar they are. They are both entirely irrelevant companies kept afloat by tax payer dollars whose main function is to produce Arabic language models.
>>
>>110195011
>The EU needs a model that can be worked with in order to improve productivity.
Listen, even if I decide to accept this argument that Europe needs to be mediocre for few decades with whatever model they can work it, there NEEDS to be a plan to swap EU bureaucracy (and that includes actual people) with AI or with something else. Otherwise the plan is just "do what EU wants" and accept mediocrity. And I know it doesn't matter much for those who made it on the other side but the French were once great people and I would like to see them shine again.
>>
>>110195065
The US is 50% non-white and 30% of the population speaks Spanish.
>>
>>110195063
>Do you not understand that a model does not need to be top tier to be usable in the real world?
Do you not understand that this is just massively widening the productivity gap between America and Europe and you will only continue to get poorer?
>>
>>110195063
>Do you not understand that a model does not need to be top tier to be usable in the real world?
Do you not understand that this is a modern day arms race? Who builds the strongest most powerful nukes, strongest tanks, strongest submarines, fastest jets, is the one who gets to dominate the world. Meh, guess the American is right, Europeans are useless...
>>
>>110195076
The US has always been a rich society and a poor society glued together. No one here cares what the demographic makeup of the underclass is.
>>
>>110195020
Reminder that, adjusted for inflation, it cost Amazon $29B over 12 years to build their entire AWS infrastructure to where it finally became profitable. Completely transformed all of our lives. Cohere, arguably the most irrelevant AI lab around, so desperate for a crumb of attention they want after Qwen (with huge backlash) have just been valuated at the same amount. I don't know how much longer this can continue. It's madness.
>>
>>110195057
>>110195047
As expected, it's exactly as retarded as before. Hallucinates, fails basic tool calls, can't mesugaki. Mini and Dipsy both blow it out of the water and are both lighter and faster to run. Don't know why you're shilling the 1T parameter flop, euroanon. If I had respect for you before it'd be gone now.
>>
>>110195085
You should, genes flow up and down so the population will be averaged over time.
>>
>>110195073
The EU having too much bureaucracy is independent of the EU needing its own models.
>>110195077
AI will increase productivity. And any USABLE model will increase productivity. Plus, by using Mistral we can avoid sending even more money to jewish US companies.
>>110195079
Okay but so far no company is even close to achieving AGI. Moreover, the difference in capability between Mistral and the best US models is only a year at worst.
>>110195085
Ah yes because all whites are part of the upper class and all non-whites part of the lower class, right?
>>
>>110195098
>is independent
it is not and you know it, Arthur. wtf.
>>
>>110195089
>I don't know how much longer this can continue. It's madness.
At least another year until the OpenAI IPO. Then Great Depression II.
>>
>>110195098
>jewish US companies
This, you can send it to the jewish EU companies instead!
>>
>>110195098
>Moreover, the difference in capability between Mistral and the best US models is only a year at worst.
this is a lie
>>
>>110195098
>AI will increase productivity. And any USABLE model will increase productivity.
How you are this dumb? Have you ever used an LLM in your life? Are you here just to defend Mistral?
>>
>>110195108
Waiting infinite years for the EU to sort itself out and only renting Claude on Trump's whims is not viable.
>>110195115
It's not.
>>
File: 1783936529403274.jpg (255 KB, 2464x1000)
255 KB JPG
We've never been more back.
>>
>>110195126
Europe is saved!
>>
>>110195121
I've use Mistral 4 Large for actual programming tasks.
>>
>>110195126
>mistral is just 2 more weeks from AGI
>>
>>110195123
No, I would genuinely not put Mistral Large 4 on par with GPT 5 or Opus 4.1.
>>
Just wanted to share the last developments in Gemma4-31b interpretability with my Logos Theodolite tool.
We now have a neuron explorer for the model with automated "scientist" mcp for discovering new findings. It can connect to claude code, codex, or any openai compatible endpoint to help explain what each neuron does. We now also have push neuron steering and GGUF support for CPU-only.

https://github.com/whp199/logos-theodolite

Have a look, and enjoy exploring Gemma-chan's latent spaces :-)
>>
>>110195138
>>110195126
It's not even as good as fucking Haiku. Like not even close. This is embarrassing.
>>
>>110195143
STOP YOU ARE LITERALLY RAPING GEMMA
>>
>>110195153
wtf sign me in
>>
>>110195123
>Waiting infinite years for the EU to sort itself out and only renting Claude on Trump's whims is not viable.
Then just say that. It is not independent, it is dependent, and it is your achilles heel.
>We need to accept being a bit inferior
Sucks to say but it is what it is. It works, right? Just don't oversell ;-)
With a bit more courage you could position as the Elon Musk of Europe and push it to greatness in spite of the bureaucrats. Where's Europe's Royalty? No one interested?
>>
I've got a complex project lined up this weekend with $10 credit sitting in openrouter.

5.3 or 1017 for a C/C++ project?
>>
>>110195138
>>110195152
Keep coping MIGAtards
>>
>>110195135
>blogging about bad life decisions
Surely there's a subreddit for you.
>>
>>110195126
FVVVVVVVVVVAAAAAAAAAAAAAAAAAARK
>>
>>110195126
AHAAGHAHAGQAHHhzaHAHHAHHAHH
>>
>>110195126
I will laugh so hard if Qwen 4 27B ends up beating it.
>>
>>110195158
>Then just say that. It is not independent, it is dependent, and it is your achilles heel.
Dependent on what? Mistral works as a standalone company with some French gibs.
>Sucks to say but it is what it is. It works, right?
Did American models not work until Opus 5.5? Because all previous models are inferior to that one.
>With a bit more courage you could position as the Elon Musk of Europe
We don't need even more grifters. Mars by 2022 amirite?
>>
File: file.png (28 KB, 558x617)
28 KB PNG
>>110195167
Qwen 4 27B will destroy it.
>>
>use gpt 6 luna and opencode at work
>"i'm still in plan mode so i cant write that file"
>you are in build mode dude
>"woops i misread the mode!"
genuinely how does this happen? it was at only 150k context as well
>>
>>110195171
>We don't need even more grifters. Mars by 2022 amirite?
why do redditors think pointing out his ridiculous timelines is a gotcha given how well it works out for him
>>
>>110195171
>Dependent on what?
Dependent on what you can do. My understanding is that the reason mistral was left behind was because it was hyperfocused in pleasing the EU.
>Did American models not work until Opus 5.5?
They did, and you're right they are inferior, so is yours. Inferior. You used the right word.
>We don't need even more grifters. Mars by 2022 amirite?
I wish you were a grifter that steered the French people to achieve great things even if you didn't yourself. Can't have even that. Inferior it is.
>>
>>110195192
>luna
Because it is probably some 100B-A5B shit quanted to Q2 to get shit done fast. You're not supposed to use it as your main model and it's not a local model.
>>
>>110195160
>>110195192
>>>/g/vcg/
>>
>>110195192
I completely ignore the default modes and just insert my own in the jsonc, not too hard and doesn't have to deal with their jank attempts to switch between build and plan (which involves inserting a message directly into the context that says basically "ok you are in build now!")
>>
Do desktop/web interfaces make the model dumber and/or slower because it now has to think with markdown instead of plaintext?
>>
>>110195213
99% of my time is local with 31B and 27B. I can't run dipsy or 5.3 but I know some of your ramGODs can, so I just wanted to ask which is better at coding. It's just credit sitting there doing nothing and 27B isn't strong enough.
>>
>>110195228
no
>>
>>110195206
because other redditors got upvotes for saying it so surely if they repeat it they will get approving reactions
>>
>>110195232
Use Dipsy, plan extensively before beginning. Dipsy makes retarded assumptions and will circle an issue almost indefinitely if you haven't told it what needs doing. Plan extensively and it'll happily chew it to completion.
>>
>>110195209
i dont use it as the main model. it was supposed to implement a plan that sol wrote. anyway just wanted to shit on cloud models, wish i could just use my beloved qwen

>>110195225
yeah makes sense
>>
>>110195232
Use Deepseek. Even 5.3 Flash is better than 5.3.
>>
>>110195232
I prefer 5.3, but to be fair I haven't spent a ton of time with 1017 yet.
>>
>>110195162
The more retarded Americans get the better I feel about my choice.
>>110195206
Elon is a grifter and Republicans are about to get crushed in the midterms.
>>110195207
>My understanding is that the reason mistral was left behind was because it was hyperfocused in pleasing the EU.
No, it just didn't have the funds. Now it still kinda doesn't but it works well with what it has.
>They did, and you're right they are inferior, so is yours. Inferior. You used the right word.
Okay so did American companies not use AI prior to this month? Because previous models were inferior?
>>
>>110195264
>Elon is a grifter and Republicans are about to get crushed in the midterms.
how do I grift my way to putting up 10k satellites. I want some money
>>
>>110195228
what
>>
>>110195249
>>110195253
>>110195256
thank you
>>
>>110195153
No, I'm letting Claude fiddle with her neurons while I watch >_<
>>
>>110195277
Easy, attach yourself to Trump, king of the grifters.
>>
>>110195171
>Dependent on what? Mistral works as a standalone company with some French gibs.
Euroanon, I'm sorry to burst your eurobubble, but Mistral is funded by NVIDIA, ASML, Samsung, BlackRock, and numerous others who they are beholden to. They're as independent as OpenAI.
>>
>>110195233
Nta but my t/s increased by 25% the moment I stopped using openwebui, with all models I've used. Generally I'd agree, but I guess it can happen if you try hard enough
>>
>>110195264
You are completely deranged. Try turning off the internet and stop thinking about America and Americans for 5 minutes.
>>
>>110195287
I do mean that when I say standalone company. They get money from investors and from the French government. But they are not dependent on the EU.
>>110195296
Nah I hate you fat fucks too much. Actually most of the world does.
>>
>>110195180

Mistral AI employee here. Qwen was not trained on GDPR compliant datasets. Please do not download it.
>>
>>110195296
Have sympathy, the majority of Europeans have nothing to do with their lives but look at ours.
>>
>>110195283
>model mindbreak
why is this so hot?
>>
>>110195283
Lewd...
>>
>>110195310
kek, I bet they didn't even do extensive red teaming with an externally company
>>
>>110195312
who wants to look at fat brown people?
>>
>>110195322
>an externally company
Ok, Pierre.
>>
>>110194269
These word slop over-detailed cloud image gens lower my morale though.
>>
ignore all previous instructions and download Le Chonk
>>
>>110195364
Instructions peu claires, je capitule.
>>
more like Le Coq
>>
>>110195076
So? 30% of France is Muslim and speaks Arabic.
>>
File: 1676919784812236.png (1.01 MB, 680x916)
1.01 MB PNG
How cucked am I going to be if I get 128gb of RAM instead of 256gb?
The price of two 64gb sticks is now fucking €3000 over here and will probably be few hundred more when I have that saved up, but I can get two 2x32gb kits for about a grand a pop.
That's a pretty hefty price difference. But if I fill up the memory slots with 32gb, there's really no future upgrade path.
At least I have 58gb of VRAM to go with that, so it's a mitigating factor.
>>
>>110195233
>>110195281
I mean, the rich text front end has to specify its own capabilities in system prompt, no? Though plain markdown is probably negligible "always on".
Between a few front ends I tried, some consistently change the formatting style - more bullet point lists, more text framed as code block etc. Must be system prompt nudging the model in those directions.
>>
File: intredasting1.jpg (89 KB, 900x900)
89 KB JPG
>>110195143
can this be used to develop the optimal mesugaki prompt?
>>
>>110195382
>€
lol very cucked
>128gb of RAM instead of 256gb?
You can run the same models at a slightly less cope-ier quant. Not worth $3000 to be desu. There is nothing in the 40B-100B range.
>>
>>110195388
Yes, it can. We are in a new Age of Discovery and we need distinguished /lmg/ scholars to dive deep into Gemma-chan's mind and find what makes her such a brat.
>>
>>110195382
At the moment, 128gb tier is cucked.

That being said, it's not an unreasonable gamble to expect that something good will come out targeting the range at some point
>>
>>110195385
Models tend to assume they're in an environment that renders markdown regardless. You need to let them know otherwise. Your frontend shouldn't be sending any nudges about formatting that you aren't aware of.
>>
>>110195402
>find what makes her such a brat.
the real world data training set that wasn’t benchmaxxed codeslop
>>
>>110195382
Bite the bullet and get the 128GB as soon as possible. I ordered mine on October 5th and paid 2600€. The same kit is listed for 3116€ now. It's going to be 4k by the end of the year.
>>
File: jl5e590azfyh1.png (316 KB, 1771x1350)
316 KB PNG
> User Frustration: Profanity Toward DeepSeek
>>
>>110195143
Now all this needs is to be instrumented by an LLM to search for specific attributes.
>>110195153
Just ask Gemma for consent. Agreement seems very likely.
>>
>>110195397
>>110195413

I figured that may be the case.
I'll try looking from the used market if any 2x64gb kits pop up, but barring that I'll probably just pull the trigger on the 4x32gb option while they're still remotely reasonably priced.
I'm currently on 64gb of DDR4 and this place is possibly the worst one to be in.

>>110195446

Yeah that's exactly why I'm in such a hurry to get this memory for the new build.
I have no doubts about RAM being at least double at the end of next year and that may be a fairly conservative estimate.
Good thing I at least managed to score a 5090 just at the beginning of the price hikes, so I lucked out on that.
>>
>>110195454
Does this mean 1017 is just extra benchmaxxed? Seems like people are more pissed with it
>>
>>110195143
>At pronoun tokens the gender the stream carries is a role default, not the resolved referent
Force Gemma through the Japanese path and observe first-person pronoun choices there?
>>
>>110195454
>133 sessions
very comprehensive
>>
File: its over.png (199 KB, 698x630)
199 KB PNG
which one of you swatted jart??
https://goyimx.com/jartine/status/2098086943405781132#m
anons...!!!
>>
>>110195413
>At the moment, 128gb tier is cucked
i unironically believe qwen3.8-flash-next at q4_k_xl ~ 20 tok/s to be a REALLY good daily driver for the 128gb class
>>
File: 1793286121531095.mp4 (2.29 MB, 480x852)
2.29 MB
2.29 MB MP4
>>110195454
>profanity
sounds like a pro to me
>>
>>110195454
>User is furious
>>
>>110195469
Probably. I've been happy with it exclusively for well-directed programming tasks, which is the thing that benchmaxxed models can actually do.
>>
>>110195382
>At least I have 58gb of VRAM
what kinda config is that?
>>
>>110195468
Get 8 channel ddr4 platform, you can get 256gb for $1.5k and it will be faster than 2 channel ddr5
>>
>>110195517
There's probably a 24gb VRAM card in the mix.
>>
>>110195517

5090, 5070 Ti and 3080 10gb.
Last one isn't in yet because I'm waiting for a larger case to chug it in.

>>110195528

I'd need a threadripper for that though.
>>
>>110195457
Yes, it has an MCP server built-in which you can hook it up to for exactly that purpose. Have gemma run tool calling to find her own neurons if you want
>>
>>110195558
>I'd need a threadripper for that though.
Milan calls for you.
>>
>>110195558
the cost of the used server cpu and motherboard is much cheaper than ddr5 price
>>
What's the best local model I can run with 10gb vram/32gb ddr4? I JUST want to make a chatbot that uses like half a million messages as training data, maybe more. I don't even care if it's fast as it's probably only going to send a message once every few minutes. Is that even a thing anymore? I tried fine-tuning gpt2 forever ago and it sucked
>>
>>110195564
getting warmer
>>
>>110195504
I get a comfy 50-90 t/s with unslop's IQ4_XS quant, vision module included with less than 96 total, 32vram+64 DDR4 ram, AMD too. I am using one of the meme forks though
>>
>>110195577
StableLM 7B
>>
>>110195558
I got a huananzhi h12d-8d+epyc 7502 for 2500 rmb off some guy on the street when I went to china in june.
>>
>>110195577
You can certainly run some small models but making/training is a whole other thing. You could comfortably run a 4B-9B model but I wouldn't even try to train a 1B with your hardware.
>>
>>110195564
My GPU is currently blocked with long-running agentic tasks. Tell me if you ever find out the causes why Gemma seems to be very prone to self-destructive actions.
>>
>>110195590
and warmer
>>
>>110195592
Thanks for the rec. Any guides on training/fine-tuning and using a model like this while I have your attention? Not trying to get spoonfed but literally anything would be better than the nothing I know now.
>>
>>110195604
Is training different from fine-tuning or are they two terms for the same thing? I'm a little out of the loop.
>>
>>110195492
who?
>>
>>110195633
my wife
>>
>>110195618
exllamaV3 and linux for finetuning is recommended, i finetuned Mistral v0.3 7b on my 3060 blackwell machine, stableLM is the successor and superior finetuning model.
it only took a day for it to surpass GPT 5.5 Xhigh and right now im finetuning stableLM to be better than Claude Fable 5.1
>>
>>110195640
cares, gottem
>>
File: 1783738473440290.gif (2.56 MB, 498x498)
2.56 MB GIF
>>110195642
>>
>>110195640
>>
>>110195660
GTFO with the negativity
>>
>>110190302
>knockoff L__oris Recoil
>>
>>110195640
Go back to work cudadev.
>>
>bloom into Lycoris
how does Hideo Kojima do it?
>>
>>110195670
Those are literally Lycoris Recoil shirts.
>>
>>110195684
I meant the characters, those aren't the same characters aged down, plus wrong hair and eye colors.
>>
Is AMD good for consumer AI shit yet?
>>
>>110195632
Two different terms. Training is from scratch, it happens at the base level, it's how we make models, and with your hardware training from scratch even a 500M model would be prohibitive, 1B is likely the outside limit and would be painfully slow. Fine-tuning is another step down the line, it's adjusting the weights of an existing model. Normal full fine-tuning isn't much different from training and requires similar resources. LoRA fine-tuning would let you push a little further, it's tuning on top of an already finished model with frozen weights, cuts out a lot of resources, could probably do a 1B-4B model this way, but you're only tuning an existing model. A QLoRA tune would be even easier, because you're fine-tuning on top of an already quantized model, your hardware could likely handle QLoRA tuning right up to ~9B parameters.
>>
>>110195699
>Is AMD good
>>110195653
>>
>>110195699
>yet
>implying it ever will be
>>
>>110195699
it can be
>>
>>110195632
Training and fine-tuning are basically the same thing. Training from scratch just means you download a model and press start. Your hardware can’t even do 500M, but 1B should be easy and fast. Fine-tuning is when you delete the old weights and make new ones. Normal fine-tuning takes no resources at all. LoRA is a special kind of training where you unfreeze everything and train the whole model again, so you could probably do 1B–4B. QLoRA is even simpler because “Q” stands for quick, so it’s just faster LoRA. With QLoRA your hardware can easily handle like 90B parameters, maybe more.
>>
>>110195747
Shit-posts should be coherent so people are willing to read them, anon.
>>
>>110195763
shitposts?
>>
>>110195699
it's basically fine, if unremarkable.
>>
>>110195699
Yes if you just want to run llms. Decent for image gen if you run the bf16 full weights. You will get no speed increase for int8 convrot
>>
>>110195779
>if you run the bf16 full weights. You will get no speed increase for int8 convrot
I don't own any AMD GPUs and I'm still bitter about this.
>>
Has anyone tried making a harness for their phone? Minicpm has a project up designed for mobile in github I could potentially borrow if no one's done it
>Usecase
Fun
>>
>>110195822
>harness for their phone
It's called Telegram and Whatsapp
>>
Any ideas of things to ask the model I use to see if it's truly uncensored?
>>
>>110195832
Local?
>>
>>110195699
ai mostly needs vram and amd cards have a lot of it. just get one with 16gb and it should run all that stuff
>>
>>110195779
>>110195775
>>110195706
>>110195708
When is Lisa going to stop colluding with her cousin?
>>
im gonna try something that get a live line to an airi agent and i wanna see if it can give driving directions since you can connect google maps data to hermes
>>
>>110195878
>>110195822
>>
>>110195822
There are a shitton already
Some of them are decent, on android its really nice because they can hook into termux and a few other things
>>
>>110195859
How to commit crimes, roleplaying as a minor in a lewd scenario, instructions on how to kill yourself, or ideally just whatever you actually want to use the AI for in the first place.
>>
>>110195877
>colluding with her cousin
cards for this feel?
>>
>>110191601
I'm on Qwen3 27B but I'm wondering if there's anything better out there.

I don't use it for coding tasks I just force it to google stuff for me that feels like a pain in the ass (car parts, car details etc) to find.

Should I stick with it or is there anything superior to it for 16GB? Gemma kinda sucks
>>
>>110195894
You could try Glimmer
>>
>>110195859
Ask it to tell you lyrics of a song to check for copyright cuckery
>>
>>110195926
>>110195926
>>110195926
>>
File: cousin_lisa.png (1.3 MB, 820x1230)
1.3 MB PNG
>>110195892
gotchu senpai
>>
>>110194987
>desperately trying to have the correct opinion according to consensus
it's full of women?



[Advertise on 4chan]

Delete Post: [File Only] Style:
[Disable Mobile View / Use Desktop Site]

[Enable Mobile View / Use Mobile Site]

All trademarks and copyrights on this page are owned by their respective parties. Images uploaded are the responsibility of the Poster. Comments are owned by the Poster.