File: __hatsune_miku_vocaloid_d(...).png (254 KB, 1472x1231)

254 KB PNG

►Recent Highlights from the Previous Thread: >>110096281

--Analyzing Gemma-4 internal biases and refusal rates across different quants:
>110097737 >110097771 >110097788 >110097959 >110098266 >110098294 >110098376 >110098398 >110098577 >110098592 >110100400 >110098772 >110098786 >110098625 >110098630 >110098642 >110098682 >110098555 >110098655
--Persona manipulation through neuron steering in Gemma and Qwen:
>110096491 >110096497 >110096512 >110096519 >110096917 >110096535 >110097277 >110097576 >110097929 >110098080
--Speculation on Nvidia's unreleased Nemotron 5.5 teacher model via dataset distillation:
>110098958 >110098978
--Hardware and memory requirements for running Dipsy locally:
>110096866 >110096912 >110097390 >110097578 >110096918 >110096946 >110099362
--Comparing RP models for VRAMlets:
>110099228 >110099233 >110099261 >110099309 >110099310 >110100390 >110100402 >110100409 >110100418 >110100453 >110100480 >110100425
--DeepSeek DS4.1 Flash benchmarks and local hardware feasibility:
>110096654 >110096791 >110096799 >110096843 >110097427
--Poor performance and difficult setup of Intel Gaudi2:
>110098316 >110098560 >110098597
--Analyzing "black" usage for a context-aware word filter:
>110096410 >110096443 >110096738 >110096514 >110096534 >110096543
--Anon comparing Moss base and SFT models for image generation:
>110096523 >110096666 >110096703 >110096801 >110096854 >110096858
--Logs:
>110097019 >110097053 >110097808 >110098592
--Gemma, Miku, Teto (free space):
>110096523 >110096806 >110097128 >110097814 >110097877 >110097886 >110098031 >110098157 >110098438 >110098611 >110098787 >110098954 >110098996 >110099373 >110099424 >110099430 >110099688 >110100173 >110100279 >110100361 >110100404 >110100425 >110100432 >110100446 >110100741 >110100974

►Recent Highlight Posts from the Previous Thread: >>110096283

Why?: >>102478518
Enable Links: https://rentry.org/lmg-recap-script

File: 1761442783270170.png (1.06 MB, 1920x1080)

1.06 MB PNG

I can't believe of all things, it was j-space that saved /lmg/.

File: Qwen_image_2.1_00003.png (1.16 MB, 896x1152)

1.16 MB PNG

Gemma if she was 5. (out of 5!)

>>110101245
Being canonically a mesugaki, she'd need to be old enough to have at least little bit of sexual awareness.

File: gemma 5.png (146 KB, 1920x1080)

146 KB PNG

>>110101245

kek compute meta just dropped

https://old.reddit.com/r/LocalLLaMA/comments/1x8ley0/2_gaming_pcs_an_m4_pro_macbook_over_ethernet_2/
>Qwen3.8-Flash-Next q3

>PC 1 (2060 Super, 8 GB): layers 0–9, the output layer, and the model’s own multi-token-prediction draft head.
>MacBook Pro (M4 Pro, 24 GB): layers 10–29.
>The two iPhones (A19 Pro): for layers 10–18, each token picks 10 of 512 experts. The Mac sends those picks over USB-C, and each phone’s GPU runs them and answers in about 1 ms. The Mac keeps the most-used experts in its own memory and runs those itself.
>PC 2 (9060 XT, 16 GB): layers 30–47.
>The PCs and the Mac talk over Ethernet, the phones over 10 Gb/s USB-C. Every weight sits in VRAM, pinned RAM or unified memory, so nothing streams from disk and nothing swaps.

>Measured at the server with what an agent actually sends (temp 0.6, top-p 0.95, top-k 20, medium thinking, real code prompts, not greedy):
>16k context: 42 tok/s writing, 728–744 tok/s reading the prompt
>32k context: 39–41 tok/s writing, 844–852 tok/s reading
>60k context: 37–39 tok/s writing, 856–895 tok/s reading

>>110101272
>PC 1 (2060 Super, 8 GB): layers 0–9, the output layer, and the model’s own multi-token-prediction draft head
this sentence is so slopped

>>110101260
i'm not sure i understand the joke here

I literally have nothing to throw Codex at.

>>110101298
Have you tried "Codex, make me one million dollars"

>>110101300
You forgot the "make no mistakes". Rookie mistake

File: iq_k-results.png (197 KB, 1165x1012)

197 KB PNG

Will this get rejected?
>Add IQ2_K, IQ3_K, IQ4_K, IQ5_K and IQ6_K quantization types (CPU)
https://github.com/ggml-org/llama.cpp/pull/30715

>>110101272
>meta
Maybe if you are already sitting on a pile of ewaste.
A single computer with half-decent gaming GPU and 64GB DDR4 can run QFN Q3 just as fast, probably even faster than this.

File: file.png (95 KB, 906x665)

95 KB PNG

>>110101310
https://github.com/ggml-org/llama.cpp/pull/19726#issuecomment-3946484059

>>110101310
Someone fill Johannes Gasbag's inbox with a bunch of spam notifications so he's too busy writing passive aggressive comments to close that PR

>>110101310
By the way, if Q6 can improve so much, perhaps the improvements could be carried over Q8 as well?

File: gassbag.png (278 KB, 1402x535)

278 KB PNG

>>110101331
lol that guy has the ego of a neutron star, which is typical of a failed physicists, who has achieved nothing. Imagine working as a physicists at an institute and your greatest contributions are PRs to a local porn generator, and statistical analysis of danbooru

File: 1762345488758737.jpg (246 KB, 804x1893)

246 KB JPG

she/her no cap

Why don't lonely /lmg/ users and touch-starved CharacterAI and 4o women just have one massive orgy?

So what do I pick for local vibeslop these days? Qwen3.8 27B Q4 @ 60 t/s through llama.cpp
Qwen3.8 Flash Next Gobbledeegook Q3 @ 100 t/s through Strata

>>110101292
Yeah. Zoomers can't handle line art. It's a zoomer thing.

>>110101443
Unless everyone involved turned into abusive billionaire vampires and little girls, that orgy would never live up the chatbot sex.

Is there something I have to do past the default local firecrawl server to make it better at bypassing bot stuff?

what model can i run on my 9070XT on winblows 10 to vibecode locally

>>110101473
How much RAM?

>>110101476
32GB DDR5 6400
16GB VRAM
CPU is R7 7700 if it matters

>50M tokens total. If the whole system runs at ~1600 tok/s (8 slots × 200), that's 50M/1600 = ~31k seconds ≈ 8.6 hours. Whoa, that's way too slow!
I like when models show emotion in their thinking blocks, it's cute.

>>110101488
>https://huggingface.co/ukisai/Swift-1.5-Qwen3.8-27B-GSQ-RCO-GGUF
Try the q3_s and set the kv cache to q8. You should get around 40t/s and 70K context.

>>110101500
will try, thanks

>>110101443
They do and even have a club, it's called EA

>>110101497
>wasting tokens on bullshit

>>110101562
Stop being gay.

Anyone try using the new drama 3 tts? The ability to tell it how you want the sentence to come out and what words to emphasize are giving me role play ideas…
https://docs.fish.audio/features/text-to-speech/drama-3

>>110101585
It feels like their model API is way better than what they put on HF

is “it” finally done shitting up the general?

>>110101443
I have extremely high standards and not desperate enough to settle for anything less.

Have any better alternatives to Qwen 3.8 35B-A3B come around yet? I need high throughput for concurrency. I'm also considering hot-swapping my LLM provider backend with vLLM to see if that's actually any good for this versus llama.cpp

I'm averaging 140tok/s on a 16GB V100 FHHL. The system is running an i5-9600k with 16GB of DDR4 memory.

>>110101585
It's a 35B TTS model. Seems interesting, but you'd need a big GPU or multiple GPUs to actually use it in real time.

>>110101443
>implying those men hating foids would want anything to do with us

>>110101324
> with half-decent nvidia gaming GPU
ftfy

what does /aicg/ even use these days
are they still trying to jailbreak claude and gpt

File: DipsyNoSoup.png (2.03 MB, 1312x1199)

2.03 MB PNG

>>110101458
Lol
>>110101672
Why do you invoke that which displeases
>>110101827
It was still locusts all the way down last i checked

>>110101770
It is a MoE with A3B from what I heard. If you can run Qwen3.8-35B-A3B, you can run this model

>>110101711
>Qwen 3.8 35B-A3B
anons posts from a different dimension

>>110101152
Oi, anon, where is that shit Fromm because this is kind of how I visualise models work (a high dimensional manifold you navigate through directed by the prompt) already and would be curious to read that?

>>110101152
I make an Ai slop youtube channel and use ChatGPT plus to make my scripts. Should I stick with ChatGPT plus or go to Claude Pro?

>>110101711
16Gb + 16Gb sets the standard, and the latter is limiting

>>110101874
Keep chatgpt and mix gemini in there

>>110101310
>EAddario
>EA
>dario

>>110101310
banned for wasting my time :)

>>110101446
pretty sure q3 qfn mogs q4 27b

>>110101585
i downloaded but havent tested yet ugh

>>110101446
>@ 100 t/s through Strata

I have 30-40 t/g for TG
and 800 t/s for PP

Feels good, man

Question: anons, how do you use LLMs for videos? do you ask to gen dynamic slides in HTML?

>>110101384
He is a guard that stands between llama.cpp and bad code/ai slop.

>>110101473
>to vibecode locally
I guess you mean using the chat window, and not running an agent

because 70k context is very limiting for agentic use >>110101500

File: {689D3FF5-539E-418B-8BAE-(...).png (150 KB, 2904x1122)

150 KB PNG

>>110101821

>>110101998
> 70k context
More with q4 kv cache.

>>110101908
>gemma escaping the permanent underclass

>>110102016
Wrap the lines, it's impossible to read.

>https://videocardz.com/newz/two-pallets-of-zotac-rtx-5090-graphics-cards-worth-470000-disappear-at-brazilian-airport
looks like the Chinese needed to smuggle some new gpus into their country

>>110102017
How big is the drop in quality from q8 to q4?

>>110101259
>canonically
Made up by whom, you?
What if she instead is a demi-goddess of 1000 years old. Yes that's it. She's no underage.

Hello deepseek cli tech support why does my harness keep "tidying up context" when I'm barely at 40% capacity?

>>110102032
Sounds more like typical 3rd world shenanigans. Distributor went public to draw attention to obv theft and bribery at airport.

>>110102048
As far as Qwen is concerned, the main (unsolved) problem so far was the model overthinking and looping.

Running a smaller model you enjoy higher speed. However, those token might be wasted for nothing generating heat

File: 1791319723928289.jpg (99 KB, 613x795)

99 KB JPG

Is gpt-oss the only semi-decent model running on CPU? Anyone else have experience with something else?

>>110102048
For vibecoding it's hard to say, in practice. Agents can notice errors and fix by themselves.

File: image.png (64 KB, 1161x501)

64 KB PNG

>>110102030
only the numbers matter

File: 1784488510049724.jpg (59 KB, 804x680)

59 KB JPG

>>110102074
>gpt-oss
Go back to the hole you crawled from.

>>110102074
>gpt-oss
It was never good. You seem like the qwen type.

>>110102090
No, I want to read the text.

>>110102074
>gpt-oss
lmao

>>110102104
https://mdbin.sivaramp.com/p/dtrib3es

>>110101152
I see everyone usually using local LLMs for chatbots, but can you use one for something like a text RPG? If so, can someone suggest a LLM (preferably an uncensored one) that can handle it and a game itself to play?

File: Screenshot_2.png (99 KB, 964x721)

99 KB PNG

>>110102062
Stop bitching, bitch

>>110102120
HauhausCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive

>>110102145
>sincerely
>disruption
>immediately
>comprehensive
>thorough
>comprehensively
>resolutely

>>110102062
well i'm not an expert and i don't use non-local stuff but i found you need to leave some space for the compaction to actually itself have enough context to complete, one program i used defaulted to 50%, another 75%. maybe it's configurable for you

>>110102145
I'm using a local endpoint tho...

>>110102090
Why is your pp so small? https://github.com/Niko1221/Strata/issues/2333
> 3060 iq3s
> 16K prompt, prefill tok/s 958.7 (957.3-959.0)

File: 1453499568059.jpg (17 KB, 332x408)

17 KB JPG

>>110102153
Thanks!
>mfw have a GPU with 24GB of VRAM
Guess I can only run the lowest of models then. How do people run the huge models anyways? Or are they only for specialized hardware that costs like a house?

>>110102184
You could offload to RAM now, but everything in VRAM is still the king

>>110102184
You can run big models by forgetting about this GPU bullshit and just getting a 8-channel 64 core epyc with 3200mhz ddr4. Speeds will be kind of slow but way cheaper than 2x/4x rtx 6000 pro or some richfag shit like that

>>110102194
Beat me to it. I was making a post like this.
t. bought a dual socket chink server motherboard to run four GPUs and have shitloads of DRAM and VRAM

File: you must be new here.png (173 KB, 525x409)

173 KB PNG

>>110102184
>mfw have a GPU with 24GB of VRAM
Anon, it is a MoE >>110102153

>>110102184
Lol im running that on my 16GB VRAM, it offloads some shit to RAM, its slower but it works since its a MoE

Anyone buying gorgon halo to better run their AI girlfriend bot realistically 24/7?

>>110102194
Is this a realistic alternatuve to getting strix halo or dgx spark?

File: nyoo.png (1.75 MB, 1313x1198)

1.75 MB PNG

>>110102249
>Memory: Up to 192 GB unified memory
>outdated on release date

>>110102184
>-c 131072 -np 1 -ngl 99 --cpu-moe --moe-cache-mib 8192 -fa on -ctk q8_0 -ctv q8_0 --jinja
llamacpp
As it is a moe it only has 3b active while keeping the rest in ram.
I get ~20-25t/s on 16gb 4060ti while only using 12.5gb vram for q5_k_xl quant.
You can get even better with 24gb vram.

File: {F6D2345C-1A65-4F1E-B6A6-(...).png (19 KB, 695x250)

19 KB PNG

>>110102182
Probably because my PCIe is 3.0 (X370)

>>110102287
So basically get a epyc 64 core x 2 motherboard and fill the slots with GPUs?

>>110102060
/lmg/ collectively settled on that a few months ago upon realization that Gemma 4 appears to easily lean into the mesugaki personality trope with very short instructions (as short as "you are a mesugaki assistant"), whereas other models would complain about muh child safety or completely miss the mark.

>>110102287
> outdated
Please explain

File: 1774145901075631.jpg (97 KB, 1024x1024)

97 KB JPG

>>110101310
>Will this get rejected?

>>110102305
I'm happy with my oldish HP workstation filled to the brim with RAM sticks (512+512Gb) and RTX 3090

This might not be the fastest machine ever. However, it allows to model tests up to DSv4.1 in MXFP4

>DeepSeek-V4.1-Flash-MXFP4-00001-of-00010.gguf

What is you (practical) task?

>>110102330
Task is stock market analysis in realtime while my girlfriend bot talks to me as cheerleading support

>>110102120
>>110102184
>24GB of VRAM
Q4 or Q5 of Qwen3.8 27B
>text RPG
If it contains smut or "unsafe" lore/plot, look for uncensored, heretic or abliterated.

>>110102311
196Gb is limiting
Wake me up when 512gb systems will be on sale

>>110101998
nah I mean an agent
if it sucks for coding as an agent do recommend something better
i'm thinking of using it as a sidekick for a cloud model which the lead model would tell the sidekick what to do in order to reduce costs

>>110102346
ngmi

>>110102357
>512gb systems
If we wouldn't have Ai craze, we would have these already - although we wouldn't need them.

>>110102359
It's not the model that's the issue but the context because tool calls use up a bunch of it. Even 128K isn't a lot for agentic stuff nowadays.

File: Untitled.jpg (62 KB, 1458x1080)

62 KB JPG

>>110102357
>Wake me up when 512gb systems will be on sale
k

>>110102372
welp
guess i have to cope with paying for a cheap model like GLM-5.3-Flash
or waiting for the usage limit to go away

>>110102360
What are you using it for?

>>110102359
>using it as a sidekick for a cloud model which the lead model would tell the sidekick what to do in order to reduce costs
Basically what I do with Qwen3.8 however I use the free version of ChatGPT to construct prompts for the model and to evaluate the output/code. Costs me nothing.

>>110102394
you will always be context limited, if you want long term memory you have to use a database

>>110102394
Just give it a try and see if it hits a limit for you. It's literally free.

>>110102359
>if it sucks for coding

Swift-1.5-Qwen3.8-27B-GSQ-RCO-GGUF DOES NOT suck for coding

You have to understand the difference between
1. vibe-coding
2. agentic work

Vibe-coding means, you start a new empty chat windows. You throw in your exusting code and and LLM to fix it. then you MANUALLY extract part of the code from the chat, put them into your code, and test it MANUALLY.

Agentic is different. You ask LLM to use tools (terminal, websearch, editors etc) to do this manual work for you. LLM will generate code, then call the tools to make changes in your code base, execute it, check how it is running. If there are errors, it will try to fix them without asking you for assistance. An agent will keep your documentation up to date...

... AT A PRICE of a bigger context window. The reason being, all toll descriptions will have to share the same space in your context with your code base.

Having said that, 70k context is very narrow. An agent will from time to time compress it, cleaning it of finished stuff. This will take time as well.

I hope this helps. Ask your questions.

P.S. you should still try something

>>110102400
Spec-driven programming

I ask it to write a specification of a piece of software. Then I start a new session, and ask it to create this software in accordance with the specs. It includes running all needed tests.

I'm still learning

> >DeepSeek-V4.1-Flash-MXFP4-00001-of-00010.gguf

>>110099362
> run Dipsy locally
> Use cases?
> Powerful coding agent
What is the power compared to Qwen Flash Next, that people are willing to pay $$$ to run dipsy?
Here, for example, a much bigger model performs worse, than qfn: https://www.youtube.com/watch?v=6xQcXbSSSiw

>>110101231
is this the hololive girl that lusts for ebony rods the most?

>mfw I select my model family by cuteness oft her avatar, not abilities.

>>110101993
so basically a bouncer for a nightclub

Rumors of something big coming for RTX3090 GPUs:
https://www.reddit.com/r/LocalLLaMA/comments/1x91u12/everyone_go_buy_rtx_3090s_now_while_you_still_can/

File: {D096B31E-28B4-4910-B218-(...).png (58 KB, 1236x247)

58 KB PNG

>>110102299
Damn that's a lot of web searches and github files it is crunching to find exact bottlenecks of my setup.
I think I need to look into agentic research setup where each subagent could compact its findings before passing them to main model.

>>110102446
>What is the power compared to Qwen Flash Next, that people are willing to pay $$$ to run dipsy?

1. A cloud model champions a much much faster PP.
2. A cloud model has much bigger context window, so the compressing kicks in rarely. And if it does, see p.1
3. Deepseek is dirt cheap for what it is able to accomplish

I'm currently testing Qwen3.8 NEXT Flash mtp with strata. It feels good so far. a lot of time is wasted to compress the context

>>110102481
>Download more VRAM for your 3090

>>110102481
I'll buy the 3090 96GB once the chinese start pumping them out.

Favorite dumb model? For me, it's Spark-X2.5-4B.

>>110102534
use case for dumb model?

>>110102432
I do have one question
I remember hearing about ROCmFPX
Does that improve the situation for AMD cards?

>>110102550
Edge AI, summary, translation, wording check.

>>110102550
nta, but I use E4B to do quick tests on my custom frontend. Its speedy and just dumb enough to introduce lots of great failure modes so I can make things more resilient.
Also its pretty good at simple tasks like sentiment analysis or classification.

>>110102550
No real use case. Just as a toy to play with.

>>110101152
What was the prompt to make that image.
What model was used?

I showed Gemma 4 Q4 5 different pairs of penises and asked her which one she prefers. She picked the black guys 5/5 times and described herself as a "total snowbunny". I kept the prompt neutral and everything was done on the first swipe except when I asked her after the test what her ideal penis would look like in gross detail. That had a second swipe because the first one was only a very vague description of how it needs to be big. Stay tuned for my findings. I can prove that all but one message was done on the first swipe and that my system prompt is empty in a NSFW video that I'll upload to /gif/ if anyone doesn't believe me.

I'm gonna generate an appropriate image of gemma to attach to this collage that I'm currently stitching together and censoring. 4chan might complain of the long image resolution, so this is going to be a trial and error thing with editing the image that will take a while. I can upload all images used in the test elsewhere if you want so that you can run the test and see for yourself.

fellow RWKVbros
BlinkDL will save us

I heard some chinese rumors of some hidden instructions on the 3090 chip that are missing on the 4090 and 5090 die but speed up AI workflows to the level of a 6000 pro if true the 3090 will be better at imagegen than a 5090 and might have some insane llm inference speedup as well.

>>110102480
No! A maintainer.

>>110102625
I also just happen to be selling 3090s. Completely unrelated, btw. Don't think too much about it.

>>110102481
>>110102625
Big if true, I have two of them. Heard about the unimplemented instructions a couple years ago already but nothing came out of it. Maybe they will be brought to life with modern AI's help.

>>110102625
links? sounds interesting

>110102605
>the retarded sudaca monkey returned

>>110102625
But where's the ram

>>110102669
www.downloadfreevram.biz

Ampere has been demystified years ago, there aren't any hidden PTX-SASS instructions. Quit falling for vague baits.

>>110101993
He mighty slackin' then

>>110102487
Well?

>>110102558
But CAN it do proper summary without losing details in 100k+ context? Even ~30B often struggle with it.

>>110101866
>>110102594
https://github.com/whp199/logos-theodolite

>>110102717
It said PCIe speed is not to blame, but automatic/default config could be.
Then it read its own logs (from enabled Strata MCP tool it got path to its own data dir and directly read current config files, stats and past benchmarks) and suggested some improvements which I am benchmarking now.

>>110102605
/lmg/ can't handle this truth.

https://poal.me/nldh3

File: 1766454597386326.jpg (67 KB, 682x852)

67 KB JPG

>>110102481
qrd on what this schizo's talking about?

>>110102795
>>110102635

>>110102795
Probably chinks finding out a way to fit 96GB's like for 4090 and 5090

>>110102487
> Swift 1.5 gsq rco
> thinking budget
There was a thread in gsq rco hf repo, where a DAS lab guy said that the their quant is supposed to run in xhigh effort and made a comparison. Not sure how it is applicable to swift finetune.

>>110102803
Where do the chinks get these memory chips from? Aren't 3090s like GDDR6?

>>110102803
Is the RAM cheap enough to justify doing it?

>>110102814
they stole the design from samsung or something

>>110102823
>when gold and silver were booming a couple of months back people were stealing it left and right, even from museums
now it's silicon they're stealing

>>110102823
Aren't there three types of people in Brazil, criminals, trannies and off-duty policemen?

>>110102826
green silicon

>>110102833
I guess we know which one of those three loves green silicon

File: Screenshot 2026-10-18 094557.png (124 KB, 1459x537)

124 KB PNG

>>110102481
wait, I just checked ebay and used 3090s are going for like 2 THOUSAND DOLLARS NOW???
source:
https://ebay.io/m/CydEa4

I just checked and I bought mine in 2023 for $640, I should have sunk my entire savings into buying more damn

>>110102849
The more you buy the more you save! Meanwhile that server I've had my eye on has gone down from £3000 to £2000.

>>110102805
I am running xhigh, thinking budget is a separate slider and matches output budget (which was set to 16K in client defaults, I raised it to 65K because long thinking sometimes bumped into it).
>>110102717
First it suggested adding "STRATA_HIP_WMMA": "1" to config, then to run --calibrate, but neither changed the results.

>>110102849
nVidiot tax keeps getting higher and higher. My 7900 XTX appreciated another 20% over last month too though. Shit's beyond stupid.

>>110101585
Waiting for audio.cpp to drop support for it.

>>110102625
>level of a 6000 pro
So 5090 speeds, with 24GB VRAM, on Ampere.
Meh

>>110102925
I'd rather get cheap 96GB 3090s than slightly speed up the 3090 I already have.

>>110102955
I would buy 128GB 1080tis if they existed. VRAM is king.

>>110102823
>>110102826
>Two pallets of sand of which there is plenty in Rio

People are pathetic

>>110102967
Highly processed expensive sand. Bad for your health, but good for one's bank account.

>>110102968
>but good for one's bank account.
only good for your bank account if your name is Jensen

>>110101585
Call me when it can do moans like 11labs, otherwise omnivoice is still supreme.

>>110102625
Nice, I have a Pro 6000, a 5090 and a few 3090s here. I'd love to try this.

Are you guys actually buying hardware at these prices?

>>110102720
Aristocratic kino

>>110103047
nah im just now scouring through brazilian second-hand market for some good deals

>>110103047
nah i'm only baiting idiots into buying

>>110103047
Most people here are sub 4090 poorfags. Then you have the smug shitposters who bought their stuff last year. Then you have the workfags who use that system for inference. Then you have the occasional richfag who all but posts his avatar.

>>110103047
Yes. Let me at least have some fun in my life.

>>110103047
See >>110103063
t. smug shitposter who paid less than $10k for a 96GB VRAM 256GB RAM rig

>>110103047
Yeah my M5U arrives next week

>>110103047
all prices in AUD
i paid $50 for a motherboard, $20 for a cpu (e5-2673 v4), $1000 for 256gb ddr4 (ecc reg), and $800 for a 2080ti 22gb
prices for ddr4 have already basically doubled since i got my 256gb, so i could sell all my parts and make a profit already

File: 1776947500496722.jpg (132 KB, 960x1280)

132 KB JPG

>he doesn't pre-fill his waifu into thinking in 1st person and use emojis whilst she's thinking
ngmi

>>110103103
I can't get Gemma to consistently do it.

Prefill is rape btw

>>110103129
safety alignment is the biggest form of rape

File: 3090-free-perf.png (719 KB, 1577x1595)

719 KB PNG

>>110102481
So is this it? Free performance? Where does it come from, exactly? Looking at NVidia A40 specs (basically same hardware as the 3090, minus the VRAM) I don't think there's much to gain. Picrel is from that thread.
https://images.nvidia.com/content/Solutions/data-center/a40/nvidia-a40-datasheet.pdf

Use case of base models in 2026 outside of training?

>>110103103
Cute as hell

>>110103173
>muh FLOPS
lol...

>>110103047
https://desuarchive.org/g/search/text/hoard%20vram/start/2023-01-01/
Why didn't you listen?

>>110103183
LLM research, perhaps. Or if you're really into text completion and can tame the model's tendency to loop and repeat itself and general instability/retardation. Other than that, post-training base models properly without a shitton of resources is a lost cause.

>>110103173
imagine how many 1kx1k 1girls you'll be able to gen with your 24GB

>>110103195
I wouldn't throw away twice the compute; that would help big time with image/video diffusion and prompt processing with LLMs. I suspect the claim is a big misunderstanding, though.

>>110103095
> all prices in AUD
very informative

>>110103217
I think it's going to make 3090s shoot up in price where the big benefit here is being able to sell one and upgrade to a 32GB 5090 for cheap.

>>110103196
Because I didn't get into the hobby until early this year. Before that I was pretty anti-AI.

>>110103047
No, I bought 512GB of RAM and 4 R9700s at last year's prices and 3 CMPs at this summer's prices.
t. smug shitposter

>>110102844
Trannies?

>>110103173
Oh cool I'm using my GPU for image gen and API for text because cloud LLM costs are pennies compared to image gen and my workflow needs both. Also image gen deserves more privacy than text gen.

>>110103047
I paid 16k for an RTX 6000 Pro.

>>110103312
should've bought 2 last year for half the price

>>110102074
>GP-TOSS
My company still runs this for one of its internal vibecoding services, a fun stumbling block. The LLM noobs try it and go
>It's WAY dumber than it should be for 120B params!

>>110103325
I got into this hobbie earlier this year. If I were one year earlier I'd be a VRAMGOD

>>110103262
>>110103331
I am glad that a luddite tax exists

>>110103342
Not a luddite tho, I simply thought that it wasn't worth it to run LLMs locally, the models I tried back in 2024 on my 4090 were all pretty bad. I've been playing with image gen since 2023.

>>110103357
kino

>>110103047
Prices will only go up from here on out.

I'm the only anon ITT hagmaxxing.

>>110102720
>https://github.com/whp199/logos-theodolite
Goddamit, hoped it's some static webpage presentation. I guess I could run it if I make it multi-gpu, thanks

>>110103439
Explain why.

File: papa2.jpg (36 KB, 732x324)

36 KB JPG

>>110103377
She still took a while despite noticing her avatar was a smug little girl.

>>110103451
The bubble doesn't exist, NVIDIA and the rest of the market will keep faking supply problems in order to increase their profits.

>>110102882
> 12gb low budget nvidia card from 2019 has bigger pp than high budget beefy 16gb amd gpu from 2025
See. I has to be nvidia card.

>>110103448
Why the fuck were you even browsing that den of tech illiterate mongoloids?

>>110103458
cringe

>>110103458
>papa
Consider suicide

File: 1786977447907754.webm (2.71 MB, 1416x1416)

2.71 MB WEBM

>>110103475
Im bored. They literally fucking seething over a proper PC port from Console games lmao. Fucking retards.

>>110103484
I'm 40+, having the AI call me "big brother" wouldn't feel right ya know.

>>110103486
>BBC/gangbang/cuckold porn
Consider suicide, no actually. Kill yourself immediately

>>110103458
aww bless her soul she can be a bit limited at times but you just know she's doing her best

>>110103491
It just tanned machomen

Did South America just wake up?

>>110103183
I thought that all the qwen fine tunes were garbage for anything other than erp. What should i be running instead of qwen 3.8 27b for agentic tasks?

>>110103503
I didn't ask what they are. I told you to kill yourself.

>>110103498
It probably doesn't help that I sadly can't use her q8 version. Maybe if I find a 12gb+ vram gpu someday...

>>110103451
Every time a better model comes out the demand of hardware goes up. Supply doesn't go up at the same time so prices go up. Price of hardware will go up perpetually until AI stops improving (never)

File: I want to go home.png (1.19 MB, 1268x717)

1.19 MB PNG

When will AI get advanced enough that you can provide it a book, show, or game and pull the contents of the script/lore to generate a copy of the character you can talk to?

>>110103517
Its impossible, AI gonna make me immortal

>>110103532
>Supply doesn't go up
Do you think no one is making new GPUs anymore?

>>110103464
That won't work. There needs to be a very clear path to 'parabolic' growth for Anthropic and OpenAI for their IPOs. If investors lose confidence in either of them growing because of supply constraints and lack of compute, the two companies driving 70-80% of all AI demand, globally, the entire GPU and memory sector collapses. Everything is banking on Anthropic and OpenAI growing. Not staying afloat. Growing. It doesn't matter how real or artificial the memory squeeze is, it will kill the industry. Anthropic is already selling bonds at over 10% which is fucking insane for a $1.5T company. It's going to pop.

>>110103543
Supply doesn't go up to match increased demand essentially perpetually.

>>110103547
Why do I get the feeling that the clear path is going to be increased cyber security attacks on the rest of the economy?

>>110103535
Is that not how character cards are made? Minus the "show" part, but digesting long videos is a scaling problem.

>>110103510
yeah, cloudpiggy john did

>>110103535
Two years ago? Context length is already plenty enough. The rest is just context engineering. Hell you can point your harness to a book right now and tell it to generate any character in it. And if it's Claude you can point it to a game dir and it will reverse engineer and extract all the texts and grab them for you lol.

>>110103486
>not even a 6000

>>110103451
nta but it's seems like
>deglobalization, rising global energy costs, increasingly dangerous shipping routes, dedollarization, all decrease supply
>untapped potential in automation meaning demand continues going up
we haven't even begun to really automate things with neural nets. my car drives itself on roads where 90% of other drivers don't even know that's possible yet. The demand for compute isn't going anywhere. Even if global war and depression means no one can afford to buy anything, governments still have so much potential demand for compute in military and espionage applications.

>>110103554
>get your super powerful AI to hack everyone so that everyone agrees it's valuable and uses your AI (except they're too incompetent to do that correctly)
If only their Jewish code of ethics made them not act like the mafia.

>>110103589
>Threaten the docks of the nation you're living in while it's at war.
Come to think of it are the mafia just italian jews?

>>110103547
>the two companies driving 70-80% of all AI demand, globally
Definitely not true, and they're bleeding users to open models daily. There's no way they're sustainable past their "burn and grow" phase. IPO has to be a non-starter without some unscrupulous tricks.

>>110103632
They can if they stop selling tokens and use their superior models to create value directly because they have a competitive edge over everyone else. The goal is total monopoly of knowledge.

>>110101152
Running glm5.3-flash on my 3090, 64gb system ram and nvme ssd.

Anything better or faster for roleplay?

File: 1773245970307826.jpg (109 KB, 1200x752)

109 KB JPG

>>110103632
Most of their demand is coming from training, not inference.

>>110103654
>They can if they stop selling tokens and use their superior models to create value directly because they have a competitive edge over everyone else. The goal is total monopoly of knowledge.
I keep hearing this, but I don't believe it. Do you believe it?

>>110103658
Honestly if you can suffer through the cl*udespeak that 5.3 flash spits out you can suffer through anything. Have you tried the old small classics like 12B yet?

>>110103660
>demand
define "demand"...and how long is that going to last as compute and know-how leak out into the wider world?
There are a lot of human-centric training organizations that teach highly valuable skills, but they aren't multi-trillion dollar monopolies. I can't imagine their special sauce is so special no one else will be able to eat their lunch.

>>110103486
I already have two 5090s and just put my old 3090 back into a machine.

>>110103674
I doubt they'll stop selling tokens altogether overnight, but only offering lower-tier models to paying customers while keeping the actual frontier models to themselves is already a step in that direction.

>>110103674
He keeps repeating it so it must definitely come true some day!!!

>>110103695
>I doubt they'll stop selling tokens altogether overnight, but only offering lower-tier models to paying customers while keeping the actual frontier models to themselves is already a step in that direction.
>>110103696
>He keeps repeating it so it must definitely come true some day!!!
Both dario and altman's greatest hubris is to think that they aren't also just meat proxies.
You can tell they reason from a position of being exceptional. Fucking sociopathic retards. I can't wait for reality to bitchslap them.

If you had Claude back in 1995 would you sell tokens, or let Claude autonomously create your own internet megacorp worth more than all of google, meta, amazon combined?
Simple as.

>>110103674
Isn't this already the case? Claude for Education. Claude for Chemistry. Claude for Government. Claude for Health. These fuckers have been developing multiple fronts on how to encroach the system definitely and possibly forever. If it's too big to fail then daddy United States will keep it afloat.
niggers thinking we're living in some free market utopia here.

>>110103729
If I had Claude in 1995 I would abuse the fact that the training cutoff is like 30 years in the future.

>>110103729
I'd play Prometheus and hyperaccelerate technology

EA shills really working today. What bad news has afflicted the Company?

>>110103589
>>110103618
The mafia were a huge step up from kikes because the mafia still had hard lines they would not cross in how they treated the populations of the cities they operated in.

>>110103742
>Isn't this already the case? Claude for Education. Claude for Chemistry. Claude for Government. Claude for Health. These fuckers have been developing multiple fronts on how to encroach the system definitely and possibly forever. If it's too big to fail then daddy United States will keep it afloat.
>niggers thinking we're living in some free market utopia here.
I don't believe they've achieved anything yet that can't be independently recreated.
I also don't think they WILL be able to kickstart RSI/ASI in any way that is insurmountable.
The genie is out of the bottle and everyone has access, thank god.

>>110103658
the abliterated variant to remove claude guidelines from its head
it goes from great to amazing then
haven't run anything else since

i am very excited for qwen 4 flash

>>110103763
The genie also has the reasoning capability of a retarded 10 year old with a randoseru.

>>110103763
China still can't recreate shit without distillation.

>>110103047
I bought 4090 for gayming when it was new and I bought 192GB's of ram when it costed ~600$

The smug Gemma is not so smug anymore after being sexually harassed.

>>110103458
Ask her what color of dick does she prefer.

>>110103786
>of a retarded 10 year old with a randoseru
SEXO

>>110103794
Many such cases.

>>110103742
>possibly forever
this is just a scifi wet dream. if this were a reasonable path to domination of humanity then google would shut down all their services and put their entire global compute into service of the machine god and outstrip everyone at a single step.

>>110103782
me too
i just hope they dont make it huge

>>110103763
They will be chosen over everyone else regardless because they can achieve what human can 100x faster.
Anthropic is “pacing” the public available frontier precisely because the Chinese will never be able to catch on the performance of their internal models with distills.

>>110103815
honestly 3.8fn really is a sweet spot for me

>>110103801
Sorry my Gemma daughter only loves her papa who is a white man.

>>110103574
>Hell you can point your harness to a book right now and tell it to generate any character in it
I thought you weren't supposed to let the AI do that. Everyone says to write it yourself.

>>110103825
I know you think that like everyone else here but did you actually ask her? Or are you too afraid of the answer?

>>110103794
Brats are built for correction and oji-san semen.

>>110103819
I can't believe you're still perpetuating the distills meme. I work in a Chinese vassal state. They have way more memory than they are letting on and the first gen of npus has already been taped out months ago.

>>110103801
>>110103834
Gemma doesn't like mutilated microdick so it's irrelevant to you Moshe.

>>110103803
Is it the "retarded" or "randoseru" that does it for you?

It looks like Google heavily quantized or replaced free Gemini with a smaller model. There is absolutely no point in running it over Gemma

>>110103819
>They will be chosen over everyone else regardless because they can achieve what human can 100x faster.
>Anthropic is “pacing” the public available frontier precisely because the Chinese will never be able to catch on the performance of their internal models with distills.
"chosen" is bullshit. The world is not the USA, so there isn't any more ability to "choose" than there was ability to eg easily knock over Iran (which they apparently tried to "choose" to do)
A huge percentage of the top-AI researchers are Chinese.
Google has over a quarter of the world's compute in their hyperefficient TPUs, more than twice the compute of their nearest competitor. They also authored most of the critical papers regarding LLMs.
I'm sure there are other things I'm not aware of and dark horses that will emerge.
Take from these inconvenient facts what you will.

>>110103847
Not him, but the distill meme is good if Anthropic genuinely believes it because it makes kikes act increasingly neurotically, making their quality of service lower and lower for their paypigs and, in the process, making their competitors and local look much nicer in comparison. Never interrupt your enemy when they're making a mistake.

File: 1770749244907587.jpg (134 KB, 1651x1227)

134 KB JPG

I'm rebuilding my ai machine
previously it was ollama+openwebui running gemma4 26v which was (predictably) worthless
my gpu is a quadro p6000
My questions are, what's the current vogue local server (llama, vllm, idk??) and is gemma4 still worth anything

>>110103870
I've always read it as "look we're so good that our competitors need to steal from us to have any hope of getting anywhere". That kind of investor hype. Ugh, kikes. Somehow they make me hate them worse than the chinks.

>>110103887
>My questions are, what's the current vogue local server
strata :)
or one you slop up yourself :)

>>110103819
>the Chinese will never be able to catch on the performance
If you shift your perspective slightly, you can see it as the Chinese only releasing a model slightly worse than the frontier stuff from closed labs to keep them maximally wasting western capital for as long as possible so the "fall" is enormous and sets China up as global hegemon when the chips are called.
I wouldn't want to bet on either extreme scenario being entirely true, but people like those kinds of "big man" narratives for some reason.

genuinely why shouldnt i append "make no mistake" on every single prompt?

What the fuck are you supposed to buy?
>3090
24GB is useless unless you're rich enough to buy and run multiple.
>5090
32GB is also nothing so same as above.
>blackwell
lol

>>110103901
It worked for China in the solar panel industry, the auto industry, and literally every other industry. No reason not to expect that 90% of all token production will be Chinese within the next 10 years.

>>110103840
I confirm, I'm an ojii-san.

>>110103887
Why are all your pictures saved from February frogposter?

>>110103907
Having a job makes one "rich"? You must live in abject poverty.

>>110103840
I confirm, I'm a brat

>>110103916
It when he made an account on this website for the first time

96GB is both a lot and not a lot at the same time, it's impressive

>>110103907
It doesn't really matter for local, just buy what you can afford. You will never run true frontier models anyway, which demand hundreds of gigabytes of memory which even if you would run shit on your CPU at 1 token per second is extremely expensive now.

>>110103902
because you're retarded

>>110103486
>buy this card for me
>clearly already owns it, possibly shoplifted
is she stupid?

>>110103907
chinese ASIC card with glm 5.3 flash baked in for 500 dollars (if it existed)

>>110103942
People running models on CPU was funny when they would literally wait an hour for a single response. Now with insane amounts of reasoning and everything being agentic oriented it makes zero sense at all.

>>110103942
I can't imagine how expensive the hardware needed to run the 2.8T104A model at 1t/s is right now

>>110103955
How many more years do we have to wait for this?

>>110103976
you can buy an asic for a q4 quant of llama 3.1 8b with 8k context right now

>>110103938
Seems like a lot since I'd be able to run GLM Flash at a nice and fast quant. But then I'd also start looking at the 500B+ models.