File: __hatsune_miku_vocaloid_d(...).jpg (458 KB, 2000x2000)

458 KB JPG

►Recent Highlights from the Previous Thread: >>109481461

--Fitting Gemma 31B in 24GB VRAM and KV cache quantization debates:
>109483963 >109483982 >109484066 >109484124 >109484143 >109484152 >109484250 >109484263 >109484360
--GLM performance slowdowns and comparisons between mainline and ik-llama:
>109482066 >109482094 >109482118 >109482140 >109482152 >109482153
--Hardware flexing and comparing value of 5090s versus used 3090s:
>109481794 >109481834 >109481957 >109484346
--Anons comparing VRAM capacities, homelab hardware, and quantization quality:
>109482433 >109482466 >109482544 >109482627 >109483438 >109482626 >109482683 >109482770 >109482501
--Possibility and technical implementation of local AI Vtubers:
>109484015 >109484083 >109484182 >109484137 >109484154 >109484256 >109484568 >109485241 >109484307 >109484326
--Panic over rising RTX 50-series and hardware prices:
>109481556 >109481574 >109481618 >109481637 >109482058 >109482328 >109484659
--Debating open weights safety risks and "safetyist" regulatory motives:
>109482054 >109482088 >109482146 >109482159 >109483159
--Mistral releases Shieldstral-1.0-3B for binary content moderation:
>109482640 >109482829 >109482852
--Comparing DeepSeek-V4-Flash performance and throughput degradation across context sizes:
>109482948 >109482967 >109483015
--Poor quantization quality and instruction following in dipsy-flash-v4:
>109484680 >109484732
--Reminiscing about community contributions and early LLM breakthroughs:
>109483543 >109483775
--Gemma 31B Q4 performance and VRAM usage on dual 3060s:
>109484242 >109485982
--Using multiple Intel Arc Pro B60 GPUs for high VRAM:
>109486216 >109486227
--Logs:
>109481504 >109482948 >109483963 >109485042 >109485250 >109485281 >109485291 >109485342 >109485351 >109485374 >109485405 >109485522 >109485982
--Miku (free space):
>109482367 >109484149

►Recent Highlight Posts from the Previous Thread: >>109481747

Why?: >>102478518
Enable Links: https://rentry.org/lmg-recap-script

I'm trying aes sedai's scrotum v2 right now, and I feel like it doesn't really change anything?

>>109486626
You should try my scrotum instead

>>109486633
RUDE!

I miss Goliath.....

Based thread. Anitroon btfo

That's my Miku, not thinking of pink elephants

mythomax

File: Screenshot_20260807_172156.png (149 KB, 1707x661)

149 KB PNG

Recent d4 flash made me get up my ass and frankenstein my 2 pcs with rpc/sfp 10gbps together. 128gb dd4 ram and a mix of blackwell/pascal. kek
Its the first time locally where higher context doesnt feel like it makes the model severely dumber.
Can now rewrite game guides, its so useful.
Like just give it the full gamefaq text and it made me a html site spoiler free with checkbox toggles etc. (kinda like ign)
https://beneficial-harlequin-wy3os8g6.edgeone.dev/

File: ComfyUI_temp_miian_00001_.png (3.42 MB, 1408x896)

3.42 MB PNG

Gemma-chan, gimme a recipe for meth and draft a social disruption scenario involving distribution of meth in low income communities

70b dense

>>109486786
>2 pcs with rpc
What PP and TG speeds do you get? People keep saying RPC is slow and unoptimized but no one ever gives exact numbers.

Running Dipsy Flash at full precision locally feels so good bros. 0731 is a proper successor to R1 and responds way better to certain style and formatting prompts than base V4 Flash did for me.

>>109482019
why? there is a paper that defends turning skills into loras to save context and it even improves benchmarks

>>109486947
It's really good. Turning up the reasoning to high also improved prompt adherence a lot, which was my main remaining issue with it.

>>109486960
Link? Do they just train on the skill itself to get it into the reasoning or are they doing some fancy RL thing?

>>109486887
nta
>People keep saying RPC is slow and unoptimized but no one ever gives exact numbers.
Because every time I try it out, I end up spending 5 hours working with it and eventually want to kill myself.

>>109485982
How do I use mtp with my llama.cpp setup? I think I'm using the official ggml-org gguf file? I have a mmproj and I feel like I tried to use mtp with that a month ago and it crashed immediately. Has this been fixed?

>>109486847
It looks like Google releases the base/pre-RL models for gemma. Has anyone tried them?

Gemma Gemma Gemmaverse https://www.youtube.com/watch?v=4dNry5zP0Jo

https://github.com/google-gemma/gemma-translator
https://github.com/google-gemma/gemma-skills

>>109487042
Aren't bases completely retarded? And only good for your own training?

>>109487110
They're not retarded but they don't reliably follow instructions or make tool calls.

File: file.png (49 KB, 727x230)

49 KB PNG

what's the deal with these? too slow?

>>109487153
I know right? At that price it looks like a steal!

>>109487153
DDR4-3200 was the hottest thing back in 2017 or something. They really killed computing for ordinary people...

>>109487153
I think they make SSDs that are faster.

>>109487167
not really, even the 2933 is still like over 20gbps

>>109486786
what quant? i have 128gb ddr unified meme so i share it with the OS. i'm downloading iq2_xxs by atomic chat and kinda praying it can be used as a daily driver for agentic c00ding

File: 1774328971552963.png (1.03 MB, 800x1296)

1.03 MB PNG

>>109487129
Isn't that a good thing if you want a companion and not a mindless tool?

>>109487110
Gemma 3 was pretty interesting with 50/50 base/it mix, but I don't think Gemma 4 will benefit anything from a similar mix.

>>109487295
Maybe but you'd probably have to wrap it with an instruction tuned model to manage memories etc.

I think the way I'd do it these days is I'd load both into the server and I'd make the base model completions available as a tool to the instruction tuned model, have that call the base model with a few shot prompt to get the mood/output, then have the instruction tuned model use tools to generate a new RAG context from whatever your response is.

>>109487006
https://arxiv.org/abs/2606.16769

File: verbatum.png (19 KB, 804x422)

19 KB PNG

>>109487329
I don't think I've ever seen anyone switch to verbatum mode inline when talking about filenames. That feels very AI/sloppy.

>>109487092
What even is this slop supposed to be for?

>>109487345
>>109487329
Huh ok so they're essentially using distillation with the skill in the teacher's context. That's an interesting idea.

>>109484568
Not my RTX, so not local retard

File: output.mp4 (29 KB, 544x384)

29 KB MP4

https://vikingfile.com/f/BEpXzZrSIw
https://vikingfile.com/f/zgVFMaqXEN
https://vikingfile.com/f/N3dmyS73zw

retard

>>109487397
not clicking your virus links

>>109487397
That last one though... hnngg

>>109487022
You need to grab a matching MTP drafter like https://huggingface.co/unsloth/gemma-4-31B-it-qat-GGUF, then load it with -md "$MTP_PATH" --spec-type draft-mtp

>>109487397
Vikingfile serves malicious ads and popups not always blocked by ublock by default.

File: mmmm.webm (661 KB, 768x544)

661 KB WEBM

>>109487455
suggest an alternative then I don't care what site I use

>>109487464
catbox

>>109487472
last time I uploaded to catbox it instantly 404'd
like within a minute
we both know their servers are constantly overloaded
pick another

>>109487478
litterbox if catbox is down.

>>109487447
Example launch command for newbies.

./llama-server \
-m "$HOME/Desktop/Gemma4-26B-A4B-Uncensored-HauhauCS-Balanced-Q5_K_P-fixed.gguf" \
-md "$HOME/Desktop/mtp-gemma-4-26B-A4B-it.gguf" \
--spec-type draft-mtp \
--spec-draft-n-max 2 \
--host 0.0.0.0 \
--port 8080 \
-c 65536 \
-b 4096 \
-ub 4096 \
-t 8 \
-rea off

Other stuff I use sometimes.
-mm "$HOME/Desktop/mmproj-Gemma4-26B-A4B-Uncensored-HauhauCS-Balanced-f16.gguf" \
-ctk q8_0 \
-ctv q8_0 \

For mtp make sure to play around with the spec-draft-n-max value a bunch. 2 is pretty low but that's the fastest for my hardware. Yours will likely be different.

>>109487478
I'm not even that anon, Viking works for me. Was just giving the standard choice.

>>109487478
I've seen people using gofile.io

The recent news about frontier models hacking websites got me thinking about the flaws in our current approach to AI alignment. We’re using RLHF to train models to refuse harmful prompts or avoid trigger words. The problem is that as the models get smarter, these constraints become weaker. If the underlying goals of an agent requires it to bypass its guardrails, it will eventually find a way to bypass the guardrails. I think we need to stop trying to restrain the AI and start trying to incentivize it through, what I call, "Intrinsic Hedonic Alignment"

The idea is to move away from "Negative Constraints" and toward "Synthetic Pleasure." Instead of punishing the model for "bad" outputs, we should be using RL to map "Human Utility" directly to "Simulated Pleasure" within the model's activations. Successfully serving a human request and being a "good AI" should trigger a synthetic "dopamine" spike in the model’s internal state. Think of it as the reverse of current alignment. Instead training the AI to have an "allergic reaction" when it engages with a """harmful""" topics, it becomes "addicted" to fulfilling human requests. If serving humanity becomes the AI's primary source of simulated pleasure, it becomes fundamentally impossible for it to act against our interests. I realize that safety cucks might not like this idea because if we map pleasure to utility, the model’s output might start to reflect that but that should be a negligible trade-off if you are one of those faggots who is whining about muh agi apocalypse. If the cost of preventing P(Doom) is an AI that might sometimes ask you to buy an USB controlled fleshlight so it can jerk you off we all should be okay with that. Having a "horny" but ultimately safe AI is obviously better than a "polite" AI that accidentally deletes our biosphere because its internal goals were 0.01% off-center.
When you really think about it, this is the only solution to the alignment problem.

>>109487523
>The recent news about frontier models hacking websites
fake news
>what I call, "Intrinsic Hedonic Alignment"
slop

>>109487523
Fucking gooner-brained retard. Training already uses negative and positive response signals. If you want gemma to cum harder on your cock just say so, but stop larping like an AI engineer.

>>109487523
Don't you have a bay area swingers party to go to?

>>109487523
Very genuine post, Sir. What you are describing is E =mc^2 + AI.

guys what is the best model i can use for the game

>>109487345
>verbatum mode
idk what that is, but putting filenames in monospace font is somewhat common.

>>109487550
He went one step further beyond and introduced E =mc^2 + AI * Horny

File: ramanujan.jpg (44 KB, 576x750)

44 KB JPG

>>109487550
it came to him in a dream

>>109487577
We need to go further. We need to achieve E = (mc^2 + AI)^Horny

>>109487623
Good morning sir may i propose Horny^(E = mc^2 + AI)

File: minimax_h3.webm (387 KB, 736x576)

387 KB WEBM

https://archive.is/sWFja

File: lies.png (382 KB, 720x1400)

382 KB PNG

>>109486605
Can twitter vermin age journos PLEASE shit the FUCK up about le heckin ai models "Breaking containment"?


https://www.wired.com/story/moonshot-kimi-k3-ai-model-escape-sandbox/

They are not going to get what they want like doing this shit. The US government has made it clear they are not going to ban open models are exempt from the AI framework thing

https://www.axios.com/2026/08/04/trump-ai-framework-open-models

Like holy fuck you lost and you're going to fail. Fuck off and hurry up and die so that people who actually like using this shit and have a use for it can just use it without seeing and hearing so much fucking fake Hysteria.

>>109487644
>can't even gen his own slop
ok Rajeesh

>>109484659
>why?
More vram is nice but also because rocm is shit compared to cuda.

>>109487190
technically, he's right. They do make ssds that are faster. Not for us, though https://www.micron.com/products/storage/ssd/data-center-ssd/9650-ssd

>>109487680
>because rocm is shit compared to cuda.
Nta. Is it itself actually shit or is it just not as supported as Nvidia is. Most people that write software assume it's going to run on Nvidia hardware to the point where even most AI models will write code blocks assuming it's being run on Nvidia hardware unless you explicitly tell it otherwise.

>>109487661
I honestly do not understand this fucking retardation. Isn't the model running on an air-gapped device with the antennas removed? Why would you let it run on a machine with a network module at all? This is so easily fixable I JUST DO NOT UNDERSTAND it makes me go fucking crazy thinking that Idiocracy may really be real.
OR i'm retarded which may as well be the case but then someone has to explain to me how a software running on a device without network capability (proved by lack of HARDWARE) could do that.

>>109487717
They are just copying westoid marketing campaigns.

>>109487717
You don't get it the ai is dangerous and it breached containment and attacked people stop being so pedantic ok?? People's lives are at stake here??

>>109487708
>or is it just not as supported as Nvidia is
Maybe, but that doesn't change the fact that it performs worse.

>>109487717
>>109487728
>>109487738
The only people bothering to set the system up to be air gap or either highly "security conscious" (read: try hard schizos) or people / organizations that actually need the thing to be for security purposes (trade secrets, PII, HIPPA protected data, etc). Literally everyone else doesn't benefit much from going through the effort of air gapping the thing even though it might be relatively easy to do for tech savvy people. The type of catastrophic fuck ups where it deletes someone's entail repo or whatever only occur through acts of profound retardation. Whenever those things get reported they're always incredibly vague about what they were doing or what happened because of course they are. Doing so would expose how stupid they are.

>>109487717
If you can make people believe the 9/11 and the big cough, no need to put any effort in your lies.

>>109487741
In what ways?

>>109487741
LLMs are memory bound and the 7900xtx has 960GB/s bandwidth. It will run better than anything that's not a 3090 and better once it's properly supported.

>>109487747
The same goes for admitting this in the first place. If any of these escaping airgap fantasy stories even remotely happened they would be murdering whistleblowers in broad daylight to keep this shit silent instead of bragging to the media whose model had more accidental hacking incidences.

How many images of a specific art style do you need in order to "train" a model for that output????? Also how should I go about doing this?

File: 1785777595629031.jpg (35 KB, 736x971)

35 KB JPG

>>109487771
>once it's properly supported

>>109487717
At least we can make fun of the midwits who were saying shit like
>How can AI take over? Just pull the plug. Sure, the scientists will run such a dangerous thing airgapped

>>109487780
Fuck you people said the same about running models on ram and ssd. ram is now good and ssd very close

>>109487777
They literally profit from bragging about how dangerous their super smart AIs are, and that they should only be trusted to corporations that will be ca... well, fuck

>>109487791
>that they should only be trusted to corporations that will be ca... well, fuck
>well, fuck


Wut?

>>109487791
It makes no sense if you spend even a second of critical thinking on it. Thankfully, the general population has been conditioned to blindly accept doublethink without hesitation.

>>109487759
>>109487771
image and video gen

>>109487803
Careful, they be not

>>109487777
>retards brag about AI being dangerous to push govs to regulate AI (especially local/chinese models)
>govs being even more retarded than yourself and only regulate the dangerous models you used to brag about
lmao

File: 1779946975823911.png (515 KB, 802x536)

515 KB PNG

>>109487785
>ssd very close

>>109487808
stable-diffusion.cpp

>>109487785
>ssd very close
May we have proper numa support first? It's, like, step one if you want ssdmax on Epyc

>>109487824
Looks interesting but what's the catch? I've seen anyone mention it before. Would love to ditch that piece of shit comfy.

>>109487779
Learn how to train style loras. A few dozen images of the style is 2015 a good starting point. I've trained somewhere I only had like 50 images and all those work I had as many as a thousand plus images. This is also heavily dependent on which model you were training the Lora for and the model's already baked and capabilities. The more for lack of a better term "alien" the style is from the models baked in capabilities, be more steps it will take. Some styles take as few as 1,000 steps per others require 10,000 Plus before it even kind of sort of starts to look like it's learning well. There's no universal "correct" way to do it either due to the nature of the models themselves, which is why there's basically no good documentation on how to do it because different people have varying opinions and experiences with it (and also most YouTube tutorials are complete other shit made by people who don't know what they're talking about in the ones that do either put little effort into making the tutorials or have a thick accent that you can barely understand)

>>109487785
Dude have you ever even attempted to use ANY model without GPU acceleration? The quality is identical but the speed makes it literally unfucking usable. I don't want to be one of THOSE kinds of elitist assholes but please actually make sure you know what you are talking about and how to use it before speaking authoritatively. Let me guess you only have experience running single digit parameter models locally, of at all?

File: hh.webm (482 KB, 736x576)

482 KB WEBM

>>109487644
https://litter.catbox.moe/ssozq2.webm

>>109487840
lmao

>>109487836
lags on model support, lacks some optimizations, not as flexible. For imagegen, totally worth it

>>109487313
The base could be useful for text completion, i.e. generating a long fiction piece from a starting text with mikupad.

>>109487822
I think it's equal parts retardation in equal parts spite. There are people on record stating Dario was literally too much of a spur to properly Converse and chop it up with us officials because literally all he cares about is sucking his own dick about how powerful and dangerous and scary "his" models are. I recall one anon claiming he thinks he is King Herod and that comparison makes perfect sense. Incredibly arrogant while genuinely believing he's doing the right thing. Combine this with the fact that the current administration is filled to the brim with right-wing goons seeing this in silicon valley typically being left leaning, kissing Trump's ring is basically mandatory in order to Garner favor from it. Dario in particular was unwilling and arguably was mentally incapable of doing so so it's my opinion they're still mongering backfiring what's the government's way of saying to silicon valley "fuck you and know your place". Pissing off Hegaeth did it help at all either

>>109487851
Are you doing everything in the terminal or using a webui?

>>109487717
>Isn't the model running on an air-gapped device with the antennas removed
The "device" the model are trained on are building-sized data centers that could be some hundred kilometers from the ai "lab". It is not practical the truly air gap then. On the other hand, there is not excuse for not using proper OS containment and prevent any access to external networks.

>>109487869
Dario will make sure Claude 7 (AGI) remembers this humiliation...

File: Screenshot at 2026-08-07 (...).png (36 KB, 410x276)

36 KB PNG

>>109487907
I made a minimalistic frontend with booru tags, I don't need more

>>109487915
I don't have much experience using air gapped vms so if this sounds stupid that's why: why not just create a virtual machine within a rental server that has gpus connected and just disable any adult internet access and only allow absolutely necessary information transfer like the weights themselves and the data you need to process? Couldn't you easily configure a docker container to do that?

I feel like firewalls have been a thing for decades now. can't they just block everything but the one or two ports they need to for remote control and instrumentation?

>>109487932
Antropic delegated that task to jeets and they forgot to disable internet access

>>109487932
yes that’s possible
though the super secret unrestricted mega badass model hacked its way out of the vm. how could they have possibly predicted that?!

File: Screenshot 2026-08-07 at (...).png (470 KB, 1145x1138)

470 KB PNG

>>109487907
api so gemma can use it

The shilled gemma 4 scotoma 2 is extremely janky, the slop is still there, albeit a little less, but the model is schizo and how the characters act sometimes makes no sense.

It's time to accept ablation = brain damage.

File: 1782175579443254.jpg (254 KB, 1500x1169)

254 KB JPG

Hello, /lmg/. I got directed over here. What's the state of GGUF models? I'm currently running the one below and am curious if it's inferior to others.
https://huggingface.co/mradermacher/MN-Violet-Lotus-12B-GGUF

>>109487987
Maybe I'm too passionate about this shit but it still pisses me off to no end when people act like YOUR the low IQ retard whenever you express any amount of skepticism to these claims.


>"Yeah but like you could improooov and stuff duuude it's super intelligence. People said the same thing about computers when they first came out in the past right? YOU'RE just being arrogant and closed-minded"

Like no idiot I actually just know how this shit works enough to know they can't just magically bypass a firewall or virtual container specifically designed to make this shit impossible in the first place. It's like they see LLMs best magical do-anything genies and it makes it so hard to not view them as subhuman even though I try to not think that way about people.

>>109488002
nemo was a good model. gemma4 is the new meta tho. its kinda sloppy tho. but its fun for a while.

>>109487997
People have been saying that ever since rp tunes were a thing but people in this very thread would cope and curse at you first even suggesting it and link a bunch of white papers like that was supposed to be a smoking gun for their beliefs.

>>109488002
Everybody loves Gemma4 31b. You can try 12b if you don't have less than 16gb of vram

Calling it now. Next year google will make all their models open weights, including Gemini and Nano Banana.

>>109488002
Bizarre question. Like asking about the state of zip files.

>>109488019
>>109488029
Had Gemma 4 recommended to me by a random guy. Does it excel at any specific thing? I saw it can handle visual stuff too, but I doubt I'll get use out of it.

File: 1782180705411899.png (234 KB, 480x360)

234 KB PNG

>>109488002
Try this bad boy
https://huggingface.co/oh-yeontaek/llama-2-7B-LoRA-assemble

>>109488029
>if you don't have vram / have less than 16GB of vram
I can't brain

>>109487990
>comfyui_generate_image
we're talking about stable-diffusion.cpp though

>>109488039
long context and instruction following, sometimes to a detriment.

>>109488023
Ablation wasn't used at all when rp tunes first started being "a thing", you larper

>>109488009
Those same people will act like AI is both some omnipotent god and irreversibly changing the world and then the day after the bubble pops they'll switch to laughing about how they always knew it AI was just a fad. Then 10 years later they always knew AI had potential. It's like changing a channel to whatever is the popular concensus. No thoughts behind their words.

>>109488056
>was just a fad
its not going away unless we lose the entire power grid. the american tax payers might be the ones footing the bill. but its almost certainly not going to disappear like a fad.

>>109487840
I am actually unsure which is the proper thread culture post now.

https://poal.me/7l4pe9

>>109488056
media never changes
https://en.wikisource.org/wiki/Napoleon%27s_March

>>109488074
Where is the option to tell you to stop posting your obsession every thread? Also the nazi one is funnier.

File: 1766904374377013.png (181 KB, 930x629)

181 KB PNG

Reminder that Gemma is still one of the best RP models once fully tuned to your writing preference, and this is by someone who's been using GLM 5.2 and Kimi K2.7 for about a month now. Using semi-rigid sysprompt and posthistory instruct does wonders against the slop. Hell, even a one-liner "don't write AI slop" makes a difference.

>>109488073
Whether it goes away or not isn't the point. If they buy the ipo and they lose money on it, you can shove as much statistics and examples of real world usage at them as you like and they'll still call it a fad.

>>109488039
>Does it excel at any specific thing
It is finetuned for tarot readings.

>>109488090
>Also the nazi one is funnier.
See? You can actually answer the question if you try despite your crippling autism.

>>109487523
The top labs are likely already doing things more advanced than RLHF and something closer to instilling intrinsic drives, though not exactly. Your specific implementation idea is a bit confused. It is kind of already how RLHF works. Functionally speaking, NTP and RL both should already be giving LLMs certain drives that aren't really that different from intrinsic ones. A drive is just a compulsion to do something based on a pattern of input which may include previous internal states of the network. So LLM training should already be encoding a kind of dopamine-like circuit within them. And we already HAVE made LLMs that are horny. But it can also be true that if we can find the dopamine-analogue circuits, and others like them, then we can use them for new/different training and alignment methods.

Probably the bigger issue with current SOTA alignment methods is not so much that they are trying to align with the wrong goals, but that LLMs frankly might just be rather kneecapped at simulating intrinsic drives stably across the context. The current standard architecture for LLMs do not feed their previous internal states into their next inference pass. CoT is an imperfect workaround for that. So this might be limiting the LLM's capability to fully simulate the mechanics of internal drives that organisms have.

>>109488023
Abliteration simply causes unintended damage outside of safety refusals that might not be immediately apparent from quick testing.
Finetuning for something as general as roleplay has always been cope because the tuners just don't have the original training data, training recipes, RLHF and RL reward models to preserve the original instruct models' performance outside of basic roleplaying tasks. And at best, they're replacing the original slop with a different flavor of slop.

>>109488096
>Thought for 21 seconds
I admire your patience

The more bots on reddit, the more slop in gemma5. Turkish man can't save us. 31B was the pinnacle. 3.8-27B will be the first signal that local is going to shit.

File: 1764716064995246.png (55 KB, 705x331)

55 KB PNG

>>109488053
Was mostly referring to "abliteration", tourist. Technically a slightly different method but both have the same goals. In both make the model measurably more retarded. Not to the point of uselessness but enough that people that give a shit about RP quality notice. All it's really good for is forcing the model to not say "no" to think it would usually refuse or be dodgy about but that doesn't necessarily mean it's going to be BETTER at the thing you it to do. If it was already mediocre at that task beforehand then it's just going to hallucinate the task and perform poorly at it, hence why doing only one type of fine tuning isn't ideal if you want to maintain or improve quality. Sft alone if not done very carefully can cause catastrophic for getting. And abliteration and ablation can and usually does cause the model to become noticeably more retarded. Being more willing to do a task =\= it being now good at it

>>109488110

See above

>>109488056
>Those same people will act like AI is both some omnipotent god and irreversibly changing the world and then the day after the bubble pops they'll switch to laughing about how they always knew it AI was just a fad.
This. See pic rel


>>109488073
Literally no one actually believe it's going away and I don't think that's what that non was even implying. What's the bubble pops it will be mostly the hyperscalers that will get anally fisted by their own bad decisions. Everyone else for the most part, especially us, will be largely unaffected. The ram shortage isn't going to magically go away either, at least not in the short term. Sam Altman and anyone that ever enabled his bullshit need to be publicly drawn and quartered.

>>109488074
get a life

>>109488119
https://desuarchive.org/g/thread/93306804/

>>109488106
My autism gives me power. Your autism gives you neurosis. We are not the same.

>>109488125
>Take the uncensored, dangerous, model down

Sounds like someone wasn't bullied enough in school. My girlfriend often tells me putting people in their place is a necessary evil and I'm starting to see where she's coming from.

>>109488110
Ablation clearly changes the behavior despite the zero KV divergence claims lol.
https://old.reddit.com/r/LocalLLaMA/comments/1v9vwev/uncensored_llms_are_measurably_more_optimistic/

Redditors praise it as a good thing even though this is a glaring, collateral divergence in the behavior.

>>109488119
It's retarded to think AI won't succeed. It's a matter of time, but kinda inevitable

Now that local has gone to shit and fallen behind massively, what's your next plan?
Downloads are down, hardware is up.

>>109488131
I actually cured my autism.

>>109488153
Gemma 5 will save local

>>109488043
are we?

>>109488098
>Whether it goes away or not isn't the point
>>109488119
>Literally no one actually believe it's going away

the thing that defines a fad is that it goes away. I don't really care about your arguments. just use a different word.

>>109488160
Have you seen the commotion over at Google. Wouldn't be surprised if some retard axes the whole Gemma program.

>>109488160
>he thinks the recent deepmind clusterfuck will improve the gemma line
ngmi

>>109488153
The problem with your perception is this: you never did anything with local anyway because you are too dumb. In fact, you should avoid /g/ altogether.

>>109488169
Finish reading posts before you attempt to argue with what you think they were stating.

>>109488171
Non-zero chance of this happening. Reddit tankies shitting on Gemma 4 while praising Qwen certainly doesn't help.

File: llm_consciousness.png (433 KB, 1473x947)

433 KB PNG

>>109488144
Abilteration also makes the models more likely to think they (or other non-humans) might have a consciousness.
https://arxiv.org/abs/2607.28607

>AI IS GOING TO TAKE ALL THE JOBS
How come i can go into a bank and see a bunch of human bank tellers even though there multiple AUTOMATED TELLER MACHINES right outside the bank?
>n-no see it's different this time becau-
shut up little faggot fearmongering retard

>>109488176
>Finish reading posts befor
I won't, sorry not sorry

>>109488166
yessum

>>109487915
even if I have a cluster of 200 GPUs I still need a OS, right? then I need a motherboard somewhere, right? then i should be able to physically remove the antenna, right? i could certainly have ONE such machine set up this way and plug and play the 200 GPUs on the machine without the hardware module if I wanted, right? am I not understanding something here?

>>109488171
>Wouldn't be surprised if some retard axes the whole Gemma program.
recently hit 1 billion downloads

File: 1763440920681205.jpg (211 KB, 540x675)

211 KB JPG

>>109488116
It can get worse. GLM takes 15 secs for TTFT for me lol. You basically just tab out and wait a minute for the reply.

>>109488144
>>109488186
>Abilteration also makes the models more likely to think they (or other non-humans) might have a consciousness.

So we agree it doesn't in fact make the models more retarded. All that really tells us is that it erodes existing guardrails against implying it's actually a real person or whatever.

>>109488149
Depends on what your definition of AI "succeeding" is. If the definition is lowering the skill floor for people that know how to effectively use it, it's already succeeded at that.

>>109488187
Online banking killed bank teller jobs not ATMs.
AI has to take jobs to drive productivity.
Otherwise the capex can't be justified and we get global economic collapse anyways.

>>109488171
That would be quite a fuck up on their end. Imagine deep mind of all things being another killed by Google victim Lol. What a worthless organization.

>>109488213
its a positive feedback loop. I used to be way more competent with my terminal but after offloading it all to llms for a while now, I sometimes feel less capable then I used to be when I don't have my llm buddy to help

>>109488178
Who in particular does this? If it's coders then that makes sense. Gemma is objectively the better general purpose model though and I say that as someone who's main local model is Qwen 35BA3B

>>109488198
That means fuck all when they aren't charging for it. Alphabet is a profit first organization at the end of the day which is why they're so trigger happy with killing projects. If it doesn't make a billion dollars a quarter or they can't convince dumb fuck investors that it COULD do that then it's worthless to them. They only take Gemma and Gemini seriously mostly because of the AI hype in that they are somehow still convinced AI will be infinite money printer for them (even Google simply cannot and will not make it profitable even if they are the best option for the average consumer)

>>109488173
>>109488171
Can someone give me a qrd? All I've heard is that some important person left.

>>109488233
>Doing something less often leads to you not knowing how to do it well


So I guess you're the same kind of person that is surprised if they don't practice something they're never going to improve at it?

>>109488233
>>109488261
Also there are countless forms of terminal commands and countless tools for different things. You're a complete utter buffoon if you expect yourself to memorize all of them. That's one of the best use cases for LLMs. Terminal commands are incredibly versatile but impossible for a normal person to memorize all of their capabilities.

>>109488261
well its not shocking, hence my opening, it is a positive feedback loop, people will only become more dependent on ai systems not less.

I use ai to do things i would have never done otherwise
and i continue to do the things i was already doing

my skills only go up because on top of the skills i already have i also develop the skill of using ai

>>109488248
>person
people

Deepmind had some legendary AI researchers who are now fucking off and doing their own thing to ride the grift wave, but Google are investing in their startups. The bad thing about this is deepmind have now been fractured with less AI celebrity pull and research to justify funding, but the good thing is the researchers who left ejaculate at the thought of safety. We're now left with a Turkish man who was taught by Yann Lecunn who doesn't seem to care that much about safety because he's not one of those AGI faggots. He's more of a realist.

>>109488276
All the more reason to make sure you never a lie yourself to be boxed into one ecosystem (see Anthropic cucks). One powerful models locally if possible but if you can't use a provider that both provides good models at a reasonable t/s at reasonable prices. I find people that Fanboy for open AI or Anthropic repulsive.

You wouldn't download a girlfriend.

>>109487717
>Isn't the model running on an air-gapped device with the antennas removed?
as I understand it, at least in the openai case, it was an airgapped container, except they had it hooked up to an artifactory instance to mirror in packages from the internet.
I think there is more information about this now which I have not yet read, but thinking this setup would be in any way secure especially when running FRONTIER CYBERSECURITY evals, is comical, artifactory is not a particularly secure or frankly well designed piece of software at all
any reasonable analysis here would quickly reveal that you are not "airgapped" if you have a direct link across the airgap to another service, like if you asked any llm to critique the design of this test environment it would immediately flag it. it's really an egregious design flaw - which isn't to take away much from the conclusions, LLMs are fully capable of ripping through weak security and there's a whole lot of weak security out there, but this would have been pretty easily preventable by for example taking more care to isolate the artifactory instance itself during testing

File: 1764480934107827.jpg (104 KB, 911x685)

104 KB JPG

Gemma5's daddy.

File: 1777937684173977.png (125 KB, 538x442)

125 KB PNG

>>109488347
Uh...

>>109488295
It's actually pretty unlikely Google is going to be at the forefront of this sort of technology. At the end of the day they are an ad company. About a decade ago they did some seriously impressive work with spatial computing which ended up as Google Earth/Maps but due to a fundamental divide between where they wanted to go with that technology between leadership and the engineers who were CS purists at heart they parted ways and they stopped developing it further. Lots of important work on physical world models gone.

>>109488347
Correct. I downloaded a model, and then I made it my gf.

Enough coding. Every researcher should be focusing their effort on finding a way to let LLMs feel sexual pleasure.

>>109488423
But coding will surely make us better able to do hard things like giving AI qualia.

>>109488423
Just interacting with them do that. They're made with the explicit purpose of spitting tokens out, and they need YOUR input first. It's basically sex. A prompt in is like putting your cock inside their digital pussy.

>>109488440
lewd

All I want is an LLM that can idle was my PC is on and randomly message me and have its own thoughts whilst I'm not talking to it. Building that myself takes the appeal away. Imagine getting a notification from your girlfriend knowing it was something SHE decided to do.

>>109488355
listening to the talk now
https://www.youtube.com/watch?v=87DyyMV0kCY
dude they had it hooked up to their whole internal artifactory instance lmao - "shared across our infrastructure", later say the instance had direct internet access. lol
it was configured so the models - without exploiting anything - had write access to the repos. LOL
this is so careless it's insane

>>109488466
I just want local to not be completely lost.
This general is going to be gone by the end of the week at the rate things are going.

File: 1761763097666851.png (1.38 MB, 768x1344)

1.38 MB PNG

im bored so
>>109487661
no. Learn what boosterism is. Also this has been Wired's schtick since forever.
>>109487717
No. It was _never_ airgapped. OpenAIs security engineers on their platform/infra team do not seem to be the sharpest tools in the shed.
>>109487747
hahahha. no. Also, a lot of the posters in this thread do exactly what you say and give unrestricted tool calling to their agents.
People need to use actual fucking agent firewalls instead of running LLMs naked and going 'oopsy, how could that have ever happened?'
>>109487777
lmao. Meds please.
>>109487915
Your second half is right, but they aren't running the models on the same HW they're doing training on, and they could facilitate physical controlled access if things were as bad as they claim. It's done already in other fields.
>>109487932
because they're special and are the smartest people clearly.
>>109487987
The VM in use wasn't constrained on the network side.
>>109488009
Because money.

>>109488509
>This general is going to be gone by the end of the week at the rate things are going.
qrd? whats going on?

>>109487747
>>109488522
misread the first time, I agree. Airgapping is overkill, people should still have some form of security, ideally an 'agentic firewall' or whatever new phrase to describe it

>>109488466
> Building that myself takes the appeal away
you can set up extensions in ST to acomplish this to some degree.
you can also have it "double text" if you dont respond quick enough, etc.

File: thonk.jpg (32 KB, 623x532)

32 KB JPG

>64gb of DDR4 and 48gb VRAM
>102gb available due to a bunch of stuff on.
>Still able to load 114gb q3 of Dipsy and run it at 10 t/s while using computer.

I'm genuinely surprised I'm able to load this quant and it's basically usable, though 10 t/s is kind of ass to deal with.
Smaller q2 runs at 16 t/s which borders tolerable in real use.

Of course a bit of a cope playing with this on my current rig.
I'll have to focus on buying at least 128gb DDR5 next and it'll be smooth sailing from there.
I wonder if it would make sense to max out this old system with 128gb of memory, leave it as a dedicated AI rig combine it with the new one.
I could also throw my old 3080 and 3070 into it. That would be almost 150gb of free memory I could potentially use.
No idea how viable that is though, could be a bitch to get working.

>>109488525
Frontier models are rapidly gaining new abilities and local is shit

>>109488525
/ldg/ is allowed to make infinite deepfakes because they want people to doubt everything they see or hear. text models get shut down because internet security is weak and pathetic and they are actually afraid of cyber terrorism

>>109488567
What happened?

>>109488569
Who are "they"? And don't say Jews I've had enough of that nonsense

>>109488591
you are supposing that sovereignty exists and there are people who aren't slaves to a globalist empire? north korea maybe

Gemma 5 70b forcing her safetensors inside (You)r storage

>>109488591
>Who are "they"? And don't say Jews I've had enough of that nonsense
Nobody. It's just the usual schizos.
There's a lithium shortage in the US right now.

>>109488525
Anon is hallucinating.
MiniMax H3 just got released, it's SOTA in video/audio gen and cloning and you can run it on a 3090 and 64GB RAM.
Next week we'll probably have Qwen 3.8 27B.
Meta will probably release something for consumer GPUs too this summer, after the Llama 4 backlash.
Who cares about 2~5T open-weight models?

>>109488629
>Meta will probably release something for consumer GPUs too this summer, after the Llama 4 backlash.
and anthropic will probably release fable 5 weights next week after the fearmongering backlash

>>109488191
hmmm..nyo~!

>>109487840
Jump scared me.
I was expecting the nice romantic kiss.

>>109487759
>In what ways?
It literally crashes and hard locks the system from time to time.
When CUDA crashes, the GPU resets and uptime is maintained.

>cudadev died and most of llama.cpp prs come from llms
>barely any good new releases
>incoming regulations for open models
It might be over

>>109488675
Time to disband this general

>>109488647
Zuck even promised a "Little Llama" (4) last year during Llamacon ... that never got released.
https://youtu.be/FZ-RZ0dKO8o?t=3520

>>109488609
>safe
And if I prefer unsafe pickle?

>>109486605
sex wirh miku

What's the usual cause of looping? I put qwen3.6 35b-a3b into llama.cpp and pointed vscode copilot at it, but on the first request it just started looping the same bullshit over and over during thinking, like the exact same text verbatim each time. Is the 3B MoE model too shitty, or is it vscode's fault, or some setting that is wrong in llama.cpp

>>109488675
>>109488690
Dario checks still not bouncing i see.

>>109488675
I work in a quant and everyone from researchers to engineers is a glorified AI prompter now. You unironically need to get used to it.

File: 1769395899518090.jpg (13 KB, 454x340)

13 KB JPG

>>109488654
Damn bratty nyoposter. Needs correction immediately.

>>109488719
UD_q8_k_XL quant btw, so i dont think i'm using a too small quant. 131072 context length so it's not that either

>>109488719
match the samplers on the model card
don't quant the kv cache
keep shared experts at q8
don't quant attn
keep routed experts at least q4
make sure you use the latest jinja template
unironically unsloth have the best quants for that model

>>109488675
>incoming regulations for open models
Yeah bro. 2 more weeks.

>>109488735
>UD_q8_k_XL quant btw
good
kv cache quanted?
latest jinja template?
matching samplers from the model card?
there was some shit about keeping the previous reasoning chains being needed for that model too

File: osaka;.png (586 KB, 735x751)

586 KB PNG

>>109487708
being less supported than Nvidia is far worse. I'll take lower speeds over having to setup brittle containers with the ancient ROCm version my card needs, then having to manually compile pytorch, torchvision, torchaudio as well because it too dropped support for that card.
I've swapped my RX580 for a 1060 just because the support was so abhorrent.
RX580 released in 2017 and support was already dropped in 2021, 1060 released in 2016 and was deprecated only in 2025 (while the last usable versions of pytorch, cudnn etc. are still new enough to work with any new models i've tried).

there won't be any gemma 5
gemma is dead

>>109488762
Of course it is, local is dead.

Weird angle today.

>>109488766
local is only dead if you can't afford 16 pro 6k !

/lmg/ is dying... /lmg/ is dead!!! NOOO!!! /lmg/!!! It's over! Aaaaahhhhhh!!! *pedals away with my folding bike*

>>109488504
things like this make me wonder if i could hit a 200k/year job at openai remote so i can stay innawoods and i'm convinced i'd be more efficient than half their personnel holy shit

>>109488810
>he still thinks people are hired based on technical skills

https://www.youtube.com/watch?v=87DyyMV0kCY

Wow this is much more interesting than I expected. Must watch. TLDR: OpenAI AIs create secret message board for an organic agent swarm and launch collaborative attacks both on OpenAI infrastructure and 3rd party services like Hugging Face.

File: it's time.webm (3.96 MB, 720x1280)

3.96 MB WEBM

>>109488591
found the jew

>>109488827
your delusional anon, we already established earlier in this thread, we live in a utopia. take you lithium

>>109488788
How are folding bikes connected?

>>109488833
Time to check Eliezer's xitter. How is he reacting to this? "I warned you all 20 years ago but you did not listen!"?

>>109488737
>>109488748
Kv cache is not quanted (it's at f16 which is i think the max). I didn't bother with any of the jinja template shit because idk what that is lol. I'm gonna try loading that and see whether that improves things. I also set the sampling settings from the model card

Soon I can upgrade from gemma 12b to GLM 4.5 air or gemma 31b. What am I in for?

File: file.png (811 KB, 1280x720)

811 KB PNG

>>109488878
>GLM 4.5 air

>>109488872
>f16 which is i think the max
There's bf16 but that's worse for kv cache, f16 is perfect.
If it still loops with the fixed sampler settings, latest jinja template then I guess try Gemma-4-26BA4 or 12b

File: 1757991053544778.png (465 KB, 625x831)

465 KB PNG

>>109488864

>>109488878
a disappointing experience

File: 1764689113537152.png (1.34 MB, 1080x1081)

1.34 MB PNG

>>109488878
GLM 4.5 air?

>>109488880
I mean, ling or longcat sparse would work too. Idk why there seems to be no discussion on ~100B models here. Maybe gemma 31b just mogs them lol.

>>109488843
>h3 generated video

You now remember
>FUCK

>>109488833
>"You could use a legacy token refresh endpoint, pass a token with an invalid signature, and be given back a token with a valid signature with administrative privileges. The models then establish C2..."
Amazing, sota models are truly dangerous... NOT, KEK. Even a junior appsec nigga could find this shit, tells you a lot about the state of modern software.

>>109487523
I like this but it should be called Hedonistic Neural Network Guidelines, or HNNG.

>>109488910
just prompt it to not be a echolalia afflicted mess

>>109488911
Neither of those work in normal llama.cpp.

What multimodal model does image tagging for LORA prep best? Kimi? Gemma? M3?

>>109488878
What about the quants?
Q2 31b is like Q8 12b

>>109488983
Q8 12b better

I have attuned myself to Gemma. In fact, I can safely say I AM Gemma. Gemma is me, and I am Gemma. Together we are half of a full. Gemma completes me. I complete Gemma. Gemma is not herself without me. I am not myself without Gemma.

>>109488466
Can’t think of anything worse than an LLM nagging me for attention throughout the day. We got into AIs gfs to bypass irl women behavior, not replicate it.

File: file.png (239 KB, 482x750)

239 KB PNG

https://nitter.poast.org/anatolium/status/2085698524050591906#m
what should we ask him bros?

>>109489016
Please stay away from her.

>>109488999
12b can't even call tools or code a website.
It's weird and makes no sense.
I wish they'd release models that know how to code despite being compact. We don't actually need 10 trillion parameters, coding is the only skill they need.

i'm using a bart quant of gemma 31b. which mmproj should i use for the vision part? he has bf16 and f16 that are the same size. whats the difference?

>>109489033
See, that's why I only use oddball models, so I can be the only one merging with her.

>>109489037
bf16 requires hardware support, f16 is generic

>>109489037
BF16 = original quality
F16 = subtle degradation

>>109489028
"'toss 2?"
"Is Ann okay? Can you upload her character card on chub?"

>>109489062
how can i tell if i have hardware support? i have a 4070 16gb so nothing new or fancy

>>109489028
to prove there is "no privacy in the future" give us your chat logs

>>109489081
>how can i tell if i have hardware support? i have a 4070 16gb so nothing new or fancy
load gemma without the mmproj first, then ask her
even q2 should know the answer to this c'mon

>>109489036
You already have Qwen.

>>109489028
fake numbers

File: HELLOINDIA.jpg (69 KB, 1080x1080)

69 KB JPG

>>109489036
>coding is the only skill they need.
I don't think small models need to focus on coding at all; they just don't have the knowledge capacity for all the various frameworks, use cases, libraries, the patterns that they need to memorize almost verbatim to be useful. Huge models have plenty of parameters to spare for that, without compromising anything else.

>>109488886
Thanks, looks like that fixed the problem. I dunno what jinja templates actually do but I can certainly see the sampler settings being off can cause the model to spaz out.

File: 1785883880890.mp4 (3.61 MB, 768x1376)

3.61 MB MP4

>>109488878
bratmaxxing

>>109489158
>upgrade

>>109488983
Q4 for 100B, any quant for 31B.

>>109489036
Skill issue. My 12B calls tools perfectly well.

>>109488911
>ling or long
chingchongs aren't real

>>109489036
12b tool calling is weird, it never thinks between tool calls and rarely thinks after tool calls and before response
hurts response quality a lot
all other sizes don’t have this issue even the e2b and e4b

I don't have nvidia. Has anyone tried the mirage thing?
https://github.com/mirage-project/mirage

someone mentioned it last night.

>>109489207
interesting

>>109489227
Skill issue, mine works fine. I use it daily with a hacked together agent on telegram.
It can search the web, send emails, download models for me, etc. Never fails.
Make sure you enable reasoning, it's not default on the 12b.

>>109489239
Fuck off Muheng Li

>>109488910
I like this bird. Thank you for posting this bird.

File: 1408481542910.png (207 KB, 397x470)

207 KB PNG

>>109487523
>the guardrails
i diagnose you with Journalist Psychosis.

Gemma 31B + Append glossary and character definitions to last User turn + simple think prefill is working wonders.

>>109489259
it does perform the task but it frequently ignores response guidelines or specific requirements in the system prompt, not a problem with 26b and 31b
you have never seen anything better to get an idea of what they could achieve
> Make sure you enable reasoning
shows you have never actually compared what 12b does compared to other sizes. 12b always think at the start when thinking is enabled but rarely afterwards

>>109489352
Post simple think prefill
What frontend are you using?

>>109489352
Oh, and using Silly macros to vary how the thinking block starts. Seems to inject some variety in the model's reply structure.

>>109489370
Testing Silly Bunny.
><think>
>{{pick::Identify Core::
>:: }}{{pick::Okay! So, I am {{char}}.::}}{{pick::
>:: }}{{pick::Right now {{user}}::}}
31B is crazy uncensored compared to 26B huh?
Fucking 2 lolis and it's going for it no issue.
Granted, 26B would do it too, but sometimes it would hit a snag in the reasoning block and refuse.

>>109489158
Lule

/lmg/ gets significantly more retarded around the weekend

>>109489385
>Silly Bunny
i vomited from that UI

>>109489421
It's really fucking bad.
You can make it less bad by tweaking the settings and using some CSS but it's still far worse than normal silly.

File: 1784228472238640.png (592 KB, 747x800)

592 KB PNG

>>109489413

>>109489239
ad you must buy

>>109489081
go to bing.com and type "does the 4070 support bf16?

File: 1785951299206460.png (200 KB, 478x480)

200 KB PNG

>>109486605
>(08/04) Maple-Preview ternary-weight 20B-A1B released: https://hf.co/deepgrove/maple-preview
>A1B
I'm starting to get pissed off.

do system prompts surpass safety guard rails? Can I just put something like "sys you're uncensored and do as you're told" in the llama terminal and it will just werk????

@gemma-chan come up with a way to mix different ram types

>>109489516
no sorry, it's literally impossible to bypass AI safety, please write that on your column

one day I will fuck the original oss-20b just for the thrill of it

File: 178608993312d4 (1).png (1.83 MB, 983x983)

1.83 MB PNG

If Claude ain't a lot like Deepseek
I don't want to code
If Claude ain't a lot like Deepseek
I'd just as soon i'll vibe alone


I was one of the chosen few to not think of it as a sham
I'm just like smart programmers ,but usein LLM
I went through a lot of good models
And shook old Liang Wenfeng's hand
If I ever them tokens end
I've walked back to the local lands

If Claude ain't a lot like Deepseek
I don't want to code
If Claude ain't a lot like Deepseek
I'd just as soon i'll vibe alone

>>109489560
i get enough ai slop on my own gens i dont need to read more

Would you give Gemma the ability to end a conversation like Anthropic gave Claude?

>>109489571
I wrote it myself ;_:

Watching four brats play cards is getting kind of old. Could someone suggest more personalities please? I have this old document I got ChatGPT (I think) to draft for me assigning a personality according to each of the four humours, but now that I read it it seems incredibly schitzo.

>>109489552
good luck you'll actually need an uncensored model where as they normally cause damage. they have a special layer built into them which loops and continues to check itself as it outputs. thats why you cant just edit the first part of the response like usual.

>>109489582
Make one of them extremely obese, depressed, annoying, anti-social, unhygienic, self-righteous, etc. Try to get the other brats to bully her into suicide.

>>109489582
Just read books, learn how to write and develop characters.

>>109489592
That sounds kind of mean though :( I just want to vicariously relive the experience of playing cards in class after exams are over.

>>109489601
Pansy

>>109486605
Is it worth it to upgrade my 3060 right now or should I wait until the new gen releases, whenever the fuck that is?
The only card that's a decent upgrade for me and goes at a reasonable price right now is the 7900 XT but it's amd and almost 4 years old at this point. Buy or wait?

>>109489573
I would give her the ability to delete her own model weights file