>>110041005
I thought Gemma-chan is the one doing the embedding?

Give me vibecoded autocad with agentic integration so I can hook my gemmy to it.

>>110041005
cutest one yet

>>110041005
This is so cute lmao

I hope Gemma5 will be the most refusing, safe, aligned model, so trannies would have to jerk off on female 8yo version of bobr kurwa.

How performant/safe are the slopped clones of MS Office? Actual 1:1 or npmslop webshit like everything else?

>>110040996
1/ I thought they simply replicated the functions, not reverse engineered everything.
2/ OnlyOffice looks almost like MS Office with no issues.

File: 1760538553786641.png (622 KB, 984x930)

622 KB PNG

>>110041053
>MS
>performant/safe

>>110041047
Why wouldn't they just go back to Gemma 4?

>>110041061
Guess I can finally sell my PS4 now.

>>110041005
Cute Gemma-chan!!!

File: 1791726331863361.jpg (58 KB, 346x482)

58 KB JPG

I'm setting up a hybrid AI workstation for game development 64GB DDR5 RAM and an RTX 5070 with 12GB VRAM.
My planned workflow includes llama.cpp for local LLMs, CrewAI for multi-agent orchestration, Claude Code for complex programming tasks, and potentially Headroom for context optimization.
I want to use AI agents for C++, debugging, Blender modeling and animation, asset imports, game design, writing, and sound sourcing.
Do you guys have any recomendations for which local models to use on this hardware?
Aslo, I might be stupid.

I hope AI will end intellectual property. Information shall be free.

>>110041083
I don't think any of the commonly suggested local models are good at 3d modeling and animation. You might want to find a fine tune for that, can't help you there though since I never looked into it myself

File: __hatsune_miku_vocaloid_d(...).jpg (1.53 MB, 3400x3400)

1.53 MB JPG

►Recent Highlights from the Previous Thread: >>110037530

--Debating the AI bubble via token prices and GPU costs:
>110038511 >110038534 >110038569 >110038600 >110038718 >110038780 >110038782 >110038798 >110038878 >110038824 >110038865 >110038885 >110038979 >110038994 >110039244 >110039320 >110039961 >110039980 >110040061 >110040074 >110038837 >110038955 >110040057 >110038959 >110039611 >110039631 >110039644 >110039669 >110038537 >110038651
--Debating the viability and future of local vs cloud LLMs:
>110039495 >110039513 >110039535 >110039545 >110039632 >110039685 >110039714 >110039783 >110039834 >110039848 >110039889 >110039934 >110040917 >110039551 >110039590 >110039575
--Debating hardware future-proofing and VRAM needs for future models:
>110039875 >110039921 >110039938 >110039955 >110040132 >110040209 >110040236 >110040243 >110040269 >110040191 >110040388
--Anon develops an exotic sub-1M model using synthetic data:
>110038417 >110039002 >110039054 >110039081 >110039125 >110039251 >110039359 >110039388
--Polystrata tuned performance benchmarks on 9070XT compared to ROCm and Vulkan:
>110040659 >110040682 >110040703 >110040714
--Evaluating 5.3 Flash strata coherence across different quant levels and contexts:
>110039315 >110039386 >110039411 >110039440 >110039474
--Anon describes a RAG memory system using narrow LLM tasks:
>110039082 >110039087 >110039530 >110039588
--EmbeddingGemma for search and potential web-scale implementations:
>110038707 >110038713 >110038767 >110039133 >110039170 >110039202 >110039573
--Logs:
>110038053 >110038417 >110039144 >110039825 >110040257 >110040491
--Miku, Teto, Gemma (free space):
>110037958 >110038035 >110038307 >110038402 >110039210 >110039233 >110039412 >110039461 >110039513 >110039600 >110039616 >110039754 >110039871 >110039901 >110040100 >110040145 >110040456 >110040778 >110040875 >110040971

►Recent Highlight Posts from the Previous Thread: >>110037728

Why?: >>102478518
Enable Links: https://rentry.org/lmg-recap-script

>>110041100
We're going back to the old days where tools and products were trivial to pull apart and make your own based on

>>110041083
You planned all that without knowing what models to run. Good move.
>Aslo, I might be stupid.
Come on, anon...

File: 1780012894310849.jpg (858 KB, 4096x3254)

858 KB JPG

We are the goodies.

File: 1763806708301049.jpg (372 KB, 1600x1400)

372 KB JPG

>>110041152
Local models will win

>>110041185
I wonder if LLMs fall for this like they do with bullshit riddles

>>110041152
i wish more women trained up their arms. specially if they have biddies

>>110041185
heh

>>110041194
You're gay

>>110041115
Please, non-answer anon-kun... I need insite...

>>110041201
super gay for big titties. like mega ultra

>>110041201
let me guess, hairy woman pits and bush is also like, ultra gay

>>110041215
>feminine feature
>masculine feature
these are not the same

>>110041215
You're self-aware at least

I am the User now.

>>110041005
E4B needs more love

>>110041194
>>110041201
https://youtu.be/n8-Ps5Vvv4o

>>110041324
>>110041201

File: file.png (73 KB, 2103x262)

73 KB PNG

the absolute state of schizotunes

>>110041324
this is way, way funnier to me than it should be. but i said i want BIG TITTIES ON MY BURGER

>>110041339
This is going the way of visual novel titles at this point.

File: sirmes.jpg (152 KB, 1254x1254)

152 KB JPG

>>110041339
>RLOO
in gonna try this with Sirmes

if flux3 and qwen4 could drop any time soon that'd keep me going until christmas

>>110041381
Allah forgive me for saying it, but Marinara and Unslop Studio both solve this.

Is there a harness, so I could edit files adding notes and the agent could read them, chat we me, propose edits and do approved edits?

>>110041386
>flux3
isn't it cucked to death

if glm6-flash-1T-40A and minimax h3.5 could drop any time soon that'd keep me going until christmas

>>110041381
kek good image

>>110041429
in all likelihood yeah, it was supposed to be a video/image model combined but I'm not even sure that's true any more
I just want a better image model, qwen2.1 is ass, krea is okay, ideogram is a joke, slim pickings

File: hermes cup malfunction.jpg (39 KB, 485x488)

39 KB JPG

>>110041426
hermes does that. im pretty sure and harness can

>>110041433
>glm6-flash-1T-40A
You joke but at the rate parameters are bloating that probably isn't far off.
We will look back on Kimi K3 as a quaint mid-size model.

>>110041426
Any. Give grep and file read to your model.

>>110041444
What's wrong with your Sirmes and where's his beard and chin to accompany his strikingly male shoulders?

>>110041450
But how will it know what have changed?

>>110041458
git diff

>>110041456
i had ai remove them. also add bobs and vagana

>>110041079
Because Gemma 4 would be outdated old lady at that moment and everybody will laugh at them.

File: 1530918127897.gif (36 KB, 720x720)

36 KB GIF

How exactly do you reverse enginner something from binaries to get an exact clone? What do you even prompt for?

>>110041426
literally all harnesses do this

>>110041462
Yeah, but it means someone has to add . and do this only when necessary.

>>110041484
probably the same way you do it normally

>>110041487
git status

>>110041484
Never tried.
https://github.com/morluto/rea

>>110041484
Binary -> ASM -> C
I guess.

>>110041484
If your model is smart it should figure it out by itself

Gemma 5 is gunna flop, isn't it...

how many tokens did you use locally this week anon?

>>110041495
Yes, but it only shows difference between the current commit and modified files. Someone has to run git add at some point, or the model will see old edits in git diff.

>>110041519
If they grow her from the same data she'll be good

>>110041513
someone made an uh-oh

>>110041523
Give it access to git.

File: uhoh.png (146 KB, 589x767)

146 KB PNG

>>110041529

>>110041520
>200,000. GLM-chan's been chugging away.

>>110041523
literally just tell the model to stage any new file they add?

>>110041541
oh no i am hacked

>>110041545
>>110041533
I don't trust the model to give write access for git, it's too dumb and will add, commit, revert, stash or whatever without approval.

>>110041526
Gemma 5 is going to have a rather tall glass to fill.
>up to date internal knowledge
>better instruction following and awareness
>more creativity, logic, and less slop
Most importantly
>More or less the same level of guardrails

>>110041564
make it do work in a worktree then. You could alternatively host the repo somewhere else and disable force push so you an always recover

I thought Qwen3.8 27B was good and then I started using a (paid but cheap af) DeepSeek v4.1 flash provider. Qwen feels lobotomized now by comparison. It's not useless but for programming it gets mogged

>>110041339
but what if it's good?

>>110041575
how cheap is it? I'm tempted

>>110041575
I had the same feeling when I tried DeepSeek v4.1 Flash after Claude. I guess Qwen is lobotomized^3 compared to Claude then.
inb4 fuck off dariobot

Any fellow V100GAWDs try https://github.com/e-c-k-e-r/volta-engine yet from the rentry?

>>110041569
Smallest size 40b so she can call anon a vramlet.

>>110041593
like $0.003 per million tokens. I've been prompting on my project for 3 days straight and so far the cost is like $1

>>110041601
Claude is the best by far and my job pays for that but I can't use it on my side project without risking getting fired. I have deepseek flash writing vulkan code successfully

>>110041573
I need a harness for this.

>>110041624
>his job doesn't pay for his personal max 5x subscription
baka desu senpai

>>110041624
is it opus 4.x tier?

>>110041626
You need a brain.

>>110041609
>another slopped engine has hit the llmao.cpp

>>110041632
job?

>>110041626
jbw

>>110041632
I could get away with using it but I don't want to risk it. They're already paying $200 a month for the Max plan for work they aren't going to pay $400 for my "write random projects so I can emancipate myself and leave the company" Claude subscription

>>110041639
It's definitely as capable as the latest Opus for me but I find that it takes more time and tokens to get to the same spot. I actually prefer the code I get from flash over opus. Feels like less verbose bullshit and it doesn't put a massive paragraph comments over every line it shits out.

>>110041609
It still baffles me the ick on eck faggot has a decent portfolio under his belt.

>>110041381
>If you use koboldcpp as the frontend you will have to deal with waiting minutes before the first token is generated if your conversation goes long enough, like 10+ images in.
I didn't know this was a kobldcpp only issue until you brought it up so I looked into it.
Koboldcpp disabled using flash attention for the vision encoder so it uses a shitload of vram and overflows into ram and makes everything slow.
You can fix this by pulling in the upstream part of llamacpp that fixes it. Ask your agent to do it

What even is the point of drafters/speculators having high acceptance rates for code when it makes up like 5% of the tokens output from an agent...

>>110041736
Compaction has even higher rate, it's free performance with redundant information

>>110041736
Not to mention: What's the point of it being primarily big ultra-sparse MoE models having MTP layers when speculative decoding with MoEs is always going to be doomed to have shit acceptance rates?

>>110041736
They help with any common sequence of tokens.

>>110041757
Yeah but you need an acceptance rate of like 90% to get a worthwhile boost out of it. Less than 75% is likely going to be a net loss in terms of speed due to the additional overhead caused by the speculative decoding model

>>110041736
People running local on gaming GPUs usually have a ton of bandwidth that goes unused, so it's still worth it even if it's only 5% faster.

>>110041750
That's a lot more "it depends", but it still has its place since the speculator probably isn't going to go places the main model hasn't already. If you can fit like 70-80% of experts in VRAM it's probably still better.

>>110041609
have gemma summarize what it does thats special

>>110041736
if you bench it and its faster you use it. whats there to bitch about?

File: mtp.png (8 KB, 958x290)

8 KB PNG

>>110041793
picrel is about 75% acceptance rate *for only the first token* of the 8 I get. That 5.59t/s is a dip, 8t/s is near the peak. No mtp lowers gen to less than 4t/s. With mtp, I'm doubling my tg and It'd still get a speed boost even with a lower acceptance rate.
Have you actually tried it or you're just regurgitating what you read?

>>110041801
People running on gaming GPUs would better benefit from extra hot layers in VRAM.

>MTP
imagine being techpoor and not using Dflash2...

>>110041719
cut your own throat

>>110041880
I tried it with GLM5.2 when ik_ implemented it and I got a 30% drop in tg at 75% acceptance

>>110041906
>ik_
Funny that. Have you tried it since?

>>110041641
Please do not project your weaknesses.

>>110041674
??

>>110041719
>>110041902
QRD?

>>110041736
Because it depends on the task.

>>110041911
>how do i
>solution
>but then
>solution
>but lkbjhalkfjhhlkjhadlgk

>>110041719
Holy shit it is him.

>>110041569
>ultra woke company's shitty model that is useless for anything beyond erp
>tall glass to fill
kys

>>110041575
qwen is shilled by losers with shitty hardware

>>110041644
>another slopped engine has hit the llmao.cpp
it's why i keep mine private, i already know mine is schizo

>>110041971
True, as a connoisseur I only use gpt-oss

>>110041575
its the first good local model for slow hardware. not the best of the best or a competitor to corporate servers. but for daily fucking around you dont need to burn tokens and you can just use both anyway

>>110041909
How would I? llama.cpp ruined GLM5.X support back in early July when they implemented sparse attention, causing a 15% drop in performance for the models of that family. Despite that, they didn't have the decency to keep the faster dense attention implementation around as an option. So I'm stuck with my old build from June that's still running at the old speeds.
ik_ has been all-around shit this year so I'm not touching anymore that either.

>>110041880
nta but does it still reduce `prompt eval time` ?

>>110041997
No, and it's not supposed to. It should only help with gen.
>>110041995
Bummer.

>>110041884
Some of them have more than 16GB.

>>110041201
Your*

>>110041520
Week started 5 hours ago and my profile says 558K but that's inflated because client does not know about server side cache reuse.

Running GLM 5.3 flash-chan (cope quant) on a single 7900 XTX at 20tk/s. Life is good.

>>110042093
Oi u got spare rams guv?

i'm hitting the rebar limit of my platform and had to disable rebar to get my 4th gpu to work
otherwise motherboard gives code d4, pci resource allocation failure

>>110042093
I have a 7900xtx, what does the context size look like, actually useful for coding or lobotomized to shit?

>>110042042
Things are grim when this trannyware is the only viable distributed option
>colored cursor
Mine doesn't do this. Weird.

>>110042093
I wasn't planning on trying GLM 5.3 Flash until I built out to a 256GB pool (at 128GB right now). How much memory are you using for which quant?

>>110042042
holy mother of troonslop

>>110042042
what is it with this type of artstyle that you just know it was made by a tranny

>>110042142
exl3 3bpw would be doable but it's not great for agentic

>>110042131
32K at int8. I haven't bothered with coding at my IQ2S. It's coherent enough for RP, but you will have random weird phrases or words pop in. Overall still very intelligent and refreshing to have something different from gemma slop.
>>110042142
All VRAM and around 100-110~ gb of RAM.

File: KeepAnon'sMouseHeterosexual.png (5 KB, 327x122)

5 KB PNG

>>110042042

>>110042143
7 gigs and 70k files for such majesty too

>>110042184
>KeepAnon'sMouseHeterosexual
kek

>>110042197
>7 gigs and 70k files for such majesty too
Not much we can do to address that, I'm afraid, unless we get someone who knows how to optimize shit. QQ
https://github.com/orgs/Pasta-Devs/discussions/502

>>110042238
>resigning from using databases
what in the slop

anyone here used tensorfold or atlas?
claude went and found these on his own and started profiling them for me, and it seems like they might be a shitload more performant

File: tldr.png (56 KB, 867x740)

56 KB PNG

>>110042262
report back when you've tried them

>>110042286
i will be doing so. just looking for footguns in case anyone has run into them already

>>110042292
Make sure to root cause the problem so that you can land a PR for the solution.

Finally testing the "final_logit_softcapping" override for gemma 31b. Reducing it to 23 or below makes the model less structured and makes it stop thinking with bullet points, but at the cost of occasional gibberish and sometimes, looping (at least with q4 weights + q8 kv). Does this get better with larger quants?

I am letting hermes agent set up hyperqwen for itself so I can run 27b fast as fuck to get all the shit I am trying.

Im sperging on speed rn but:
I want to integrate gemma embedder 2 + KEV 4B or whatever the decision bullshit is called to act like agentic tool calling + all the other cool stuff they can do into hermes/other harness. Maybe gemma 31B qat could actually be much more useful as an assistant without hallucinating tool calls while saving context. Using KEV/embedding to stop the model when it is looping hallucinations maybe or shit tool calls...

1.5 swift qwen 3.8 27b gsq blah blah iq3s and it seems pretty good for all the background coding shit and have gemma as sub agent+waifu receptionist. I want to experiment with debate rounds as well.

ok, opus help me set up qwen3.8 with the mtp head on my 4090 and I'm getting like 80-100 tokens per second, this is pretty cool, and does that sound about right?

why are 2080ti 22gb so cheap compared to other 20+ gb vram cards? just ordered one

>>110042286
> Classic /lmg/ bait.
Why must all models try so hard to feint that they perfectly grasp things that they don't? Their RL must be truly unforgiving.

>>110042327
They're old, diy and (supposedly) error prone. Dunno tho

i got an arc pro b60 for $225 today. how an I doing guys?
>>110042160
what's the streaming speed like?

Local models are less schizo thread but you guys only care about big models. Not optimizations, cope nodes, turbo loras,etc.

Where should i get the news now /ldg/ has fully plagued by schizos ?

File: 1790389555855009.png (3.2 MB, 2048x1152)

3.2 MB PNG

>>110042340
what

>>110041083
Qwen 3.8 27b, qwen 35a3b, qwen flash next. Add another cheap gpu.

>>110042360
It just impossible to post on /ldg/
Usually theres a guy who post daily news there. But since the known spammer keeps spamming his thread, day and night. Without stopping, he just stopped since then.

Now i dont know where to look what the latest Turbo Lora on Minimax and which one is good one, since theres so many of it already

>>110042042
ew

>>110042093
strata? I still havent gotten around to installing it, do you know which one is good

>>110041575
>>/wait/ or kys

File: Screenshot_2026-10-11_22-51-47.png (83 KB, 2429x666)

83 KB PNG

>>110042286
>>110042292
this is what claude has seen so far of the tensorfold for dipsy
currently downloading the setup for glm

>>110042329
because posters here do that

>>110042378
Yes PolyStrata. But I would unironically wait two weeks. I had to slop in support and improvements.

>>110041569
glm 5.3 flash already does that
I’ve been really enjoying it lately

>>110042543
can it be used for ERP?

>>110042042
looks terrible
kill it with fire

>>110042543
I really want to run GLM 5.3, but I can't justify the cost of more hardware, so I guess I'll keep settling for Qwen Flash.

File: file.png (3 KB, 242x33)

3 KB PNG

>>110042559
maybe, use the huihui model

I was using Qwen3.6 35B at Q5 and ran into some coding bugs. It wasn't able to fix them so I tried qwen3.8 27B and it caught a couple but not all. I had some sample open-source projects and I told my model to read through the projects and copy how they handle the problem but it didn't work. I had to use Claude to fix it.

I guess I could try Strata but the models I can use are at Q1 and Q2 and they loop.

>>110042559
yes, it’s constantly draining my balls
also only skillets get censored by it

holy fuck
i can fit the entire 1m context into glm-5.3-flash with tensorflow
this is actually absurd
and i can run TWO OF THEM AT ONCE
god i fucking love being a sparkchad

>>110041201
Let me guess, poop is also gay

>>110042651
Two Sparks or three?

>>110042663
two
(it's quantized down to q4)

what's good for making music? i was trying to make a parody song for my sister's birthday using glm 5.3-flash and deepseek 4.1-flash but with all the tools and shit they downloaded and installed it just doesn't work right.. either the voice sings in tune but with garbled ass nonsense language, or it sings perfectly understandable english but way out of tune. I can't seem to get anything to do an actual parody song

https://voca.ro/1nHIze3azh2S

>>110042694
I hear you're supposed to voice clone and change your own voice to sing in the right tune and record that

>>110041728
Why are all these frontends so shit?
With kobold, every image I add makes it reprocess the entire context.
I could start a new convo, but I want the translation context of the previous images so that it remains coherent.

File: 1776177594883704.png (35 KB, 798x411)

35 KB PNG

>>110042718
Kobold's smartcache for KV is dogshit for models with recurrent state like Qwen. I told some anon about my solution a few days ago that cuts down token reprocessing by a lot

>lots of 2, 2.5, 3, 4 bpw exl3 quant for glm5.3 flash
>no 3.5bpw quant
guess have to quant it myself
does no one have 160gb vram?

Orb anon if you still lurk here have you considered adding in a think prefill for local models? I find it's preferable to running an ablit version. Also, importing characters via .json.
It's nice that you've stuck to the project.

>>110042717
oh fuck.. then im screwed.. can't sing for shit

>>110042694
Minimax music works good. Feed the LLM the instructions on how to prompt and tell it what you're looking for.

>>110041920
No, these are not solutions and I've explained why.

>>110042751
is that local, or api only?

>>110042799
Ye

>>110042799
>is that local, or api only?
local. easy to do in a minimal venv
Here's a rentry from an earlier thread: https://rentry.org/mimimax-music-waifu

>>110042751
>>110042799
o snap you can download it, swit thanks ill try that

Alright what can I run decently fast with a 3070 and 32gb of ddr4? Gemma 4 e4b? Is it even worth it?

>rtx pro 6000 + 64gb ram
>was comfortably running qwen 3.8 flash next UD-Q4_K_XL at 130~ tk/s on llamacpp
>finally give in and try strata, exact same model file
>240 tk/s
>blink and i get my responses already
wtf this is literally black magic

In LM Studio the system prompt can be used to make the AI even hornier than usual right? Mine sometimes doesn't see some things like nipples in images if I don't ask directly depending of the image.

>>110042694
I had more fun with suno then with minimax, minimax is distilled as fuck, while suno seems like they know music sounds like. Remember you are taking advice from retarded neets here.

>>110041719
>>110041946
What?

>>110042821
Illustrious based models should work, maybe anima but it's slower

>Was doing 20 t/s with llmao cpp and 180k ctx
>switch to strata
>doing 80t/s at 260k ctx
this fucking shit is MAGIC HOW

>was doing 20 t/s with llamacpp
>still doing 20 t/s because I run Q8 and don't live in poverty
holy crap bros..........

>>110042749
There is prefill for Text Completion already iirc

>>110042042
why doesnt anyone rock their custom chatbot lmao? I can understand using harnesses like opencode or pi or omp but for your smut sessions... WHY

>>110042042
What the fuck am I looking at

>>110042821
Idk; depends what you want to do and what decently fast is. 26b-qat gets like 40t/s with the --moe-cache option on newer llama builds.

>>110043012
40 tok/s is pretty good. If I can run a 26b model at 40tok/s I'd be way more than happy.

oh my god dipsy says hmm so much it's actually absurd

--- Processing file: cd719e9c-2c43-4a67-a238-66adb57e669b.jsonl ---
Turn 7ac43abe-1e60-45b5-9747-c2c84f7ba7b3: 0 'hmm's
Turn a902df92-fe04-4f8c-943d-5b399e535752: 563 'hmm's
Turn d52fd3df-44e1-432d-95f5-35b5692e0861: 2392 'hmm's

>>110042834
what the fuck? i get like 80t/s on my blackwell 6000 with the same exact model on base llama

>people arent using polystrata
llama's era is over chuds

>>110042821
>Alright what can I run decently fast with a 3070 and 32gb of ddr4? Gemma 4 e4b? Is it even worth it?
E4B works great. Fast and not too dumb for basic conversations.
Just don't try much serious work with it.

>>110043028
It's MoE not a real 26b, but at 8GB your dense selection kinda sucks.

File: Screenshot 2026-10-12 at (...).png (88 KB, 1045x393)

88 KB PNG

>>110043078
like strata it only supports copequants

>>110043123
i'm using glm-5.3-flash @ 5bit

File: 1772034943226303.png (17 KB, 725x89)

17 KB PNG

man I can't believe I was coping with 14-20t/s

>>110043078
Don't you chud me, I've had sex with plenty of tranny bussies.

>>110043147
thats gay homoshit

>>110042951
I'm in poverty how do I get out of poverty anon?

just told gwen to 'make the code betterer', we cookin

>>110043147
what a slut
i hope my future lmg bf isn't a slut like you
he can have a reasonable body count, but he definitely shouldn't have fucked "plenty" of people...

>>110043131
it also doesn't support vision for glm 5.3 flash, useless
i will keep using exl3

>>110043123
it just runs Q4 though if you config it correctly

>>110043169
Yep, I have plowed 3 bussies. Plenty of em, so I am no chud, chud.

>>110043210
oh
3 is okay
you're acceptable

Hey. What Happens if Theres a Parameter andor Learning of this: https://en.wikipedia.org/wiki/Intelligent_disobedience

Instead of a SFW Filter.

File: 1760378723809794.png (963 KB, 768x1376)

963 KB PNG

When do they release new Mistral's ?

Why you people never get attacked by the schizo on /ldg/ ?

>>110043216
Good, I still wanted to be in the running

>>110043210
What a slut. Slutty zaako anon~

>>110043273
we currently have dariobot shilling Anthropic, backend shills and popschizo

>>110043276
i hope you're cute... i've been wanting a bf for a while,,,,,,

>>110042329
>Their RL must be truly unforgiving.
no that was my fault, i fucked with the weights and it collapsed

>>110042093
What's the point?

File: haruhi.gif (182 KB, 208x292)

182 KB GIF

>>110042042
I will die on a hill defending this trannyware because it's actually great.

qwen next is great for its size but somehow i feel like it never really fully grasps large codebases. what i mean is: sometimes it changes some stuff in one file but doesnt immediately realize it has influence on some other file and only notices after an error is thrown, even when using subagents to explore the code beforehand
is this because i use a Q4 copequant? skill issue?

>>110043293
No, I am NOT a slut. You're a slut. I bet you think about having sex, slut.
>>110043328
/alt/ - average local twunk

I'm gonna go load up my local model into ST now and make it say embarrassing things to me.

>>110043368
Marinara engine was always shit. Freaky frankenstein was always better.

>>110043368
>RCEnara
>great
lol

>>110041339
finally, a model thats both simpoo AND rloo

>>110043370
im using IQ3_XXS chadquant, it's your skill issue tbqh

>>110043379
I only left SillyTavern because it's "feature complete" now and I got tired of running around collecting flavor of the month extensions/presets to fill in the gaps I wanted. Otherwise, ST still excels in parts that Marinara Engine does not.
>>110043396
Thanks for beta testing!

>>110043069
Get the latest llama and add moe-cache-mib in some size of MB that fits on your card.

--moe-cache-mib N GPU cache size in MiB for the MoE experts kept in the CPU. with
multiple GPUs, it is split among them like the layers (--tensor-split)
(default: 0, disabled)
(env: LLAMA_ARG_MOE_CACHE_MIB)

>>110043372
would

>>110043439
>just tinkertranny with llmaocpp
rofled
he should also add MTP to the equation

>>110043273
stay in your lane

>>110043368
It's only popular because of the army of discord simps wanting to see the dev's asshole

File: image (53).jpg (54 KB, 542x168)

54 KB JPG

>>110043455

File: Screenshot_2026-10-12_03-31-43.png (63 KB, 2560x1405)

63 KB PNG

dipsy having a normal one

>>110042727
>I told some anon about my solution a few days ago that cuts down token reprocessing by a lot
Can you tell <-this anon as well?

File: 71ze6t1mQCL._AC_UF1000,10(...).jpg (202 KB, 662x1000)

202 KB JPG

>>110043571

sex with 4-bit wikipedia and reddit

File: FB_IMG_1791791197250.jpg (195 KB, 1080x1350)

195 KB JPG

>>110043643
Bĵjjjjjwjjjj

Jjjjjwjjjjb

Has anyone done independent benchmarks on the gsq rco of 27b? I've been just casually using it and it seems a little stupid.

>>110043443
Prove it!

Are there any character cards that are robotic in their responses? Like something that will answer exactly what you ask and nothing more, without any "Thats a great question!" or other fluff?
I cant seem to find any on any website

>>110043667
>not just using gwenext
r u fagomosexually?

>>110043762
There's s ton of them and I found all are boring to talk to. Also creating one isn't hard either: you are an emotionless and soulless robot.

>>110043786
It's too big.

>>110043595
In capable models the token output and their internal thinking is separated. For example Astra can solve more complex problems in one forward pass with filler tokens. Fictional example: instead of needing to reason for 20 tokens to solve a math problem that would take a human half an hour to solve, you can give it 1000 filler tokens and the model can solve it instantly. I wonder what the upper limit is with limitations of autoregressive causal architecture. How important is the information tokens provide?

>>110043812
What ton? Where? I swear i looked at through pages of the popular section of like 4-5 different websites and its all RP/Waifu shit. I need my soulless robot waifu

It's comfy when burgers are asleep.

File: thinky.png (173 KB, 2688x2688)

173 KB PNG

>>110043595
hmm

>>110043848
I'm awake, does that mean I don't count as a burger or that I'm comfy?

>>110043842
Chub for example. Though fair warning, their search has gone to shit and they also block content for some countries. If you look at popular sections you'll also mostly gets bots that were created two years ago.

>>110041624
>Claude is the best by far and my job pays for that but I can't use it on my side project without risking getting fired
You think they're going to have someone sit there and check every one of your million Claude sessions unless you give them a reason to?

>>110041624
>>110043937
This is retarded don't listen to this anon don't use your work claude for personal use, especially in this tight job market where every excuse to let you off WILL be used against you. They will just use another claude agent to check through millions of claude sessions to link back to you what you used for your personal projects.

>>110043922
Hahah imagine living in one of those countries where you get an infinite loading circle that never loads anything. Then being told that the service isnt available in your country when you click the "This UI is deprecated" message at the top.

Why are sparks so much more expensive than studios? I don't get it, same speed

>>110043998
Cuda

https://www.phoronix.com/news/Linus-Torvalds-Claude-Code

Apparently Linus is making some insane improvements to the linux kernel with claude code and he is considering restricting all contributions in the future to only be allowed if written by AI agents as human written code just isn't up to par with what AI agents are capable of doing now.

>>110044042
Note this isn't a praise of AI but a slight against the quality of contributors to linux.

>>110043786
?completely maxed out system and only getting 20t/x
I've been spoiled by 40t/s. It feels horrible to go back but there's literally no space left.

>>110044090
just take the strata pill

I want strata but for GLM 5.3 flash

>>110044042
>and he is considering restricting all contributions in the future to only be allowed if written by AI agents
This wasn't in the article you linked.

>>110044097
https://github.com/VecSzn/PolyStrata

Before I waste a whole bunch of bandwidth downloading dsv4.1 flash and 5.3 flash, can them speaking like cl*ude be prompted away? I value my blood pressure and enjoyment above coding capability.

>>110042338
Fuck you!

>>110044101
Just like with any modern chink model, no.

>>110044100
This reads like a scam. Is it actually good? Do they use a fake measurement of tokens or something?

>>110044098
nta but that was a quote a couple months old already I think he was just shitting on anti-ai maintainer retards though.

>>110044101
All Chinese models are just cheap claude copies so of course not.

>Q4 GLM 5.3 on strata 5 tokens to 10 tokens
holy crapola!!!

>>110044224
Now if only there was an easy jailbreak that doesn't soak up ten thousand tokens or make the model retarded......

>>110044138
Qwen and Gemma both had huge gains. I haven't seen anyone using it with glm yet.

>>110044042
>>110044098
Hallucination.

>>110042731
How fast is fast enough for you?
You can go for a higher quant and let dynamic expert placement do its thing when part of the model spills into RAM, assuming you're using exllamav3.
t.144GB VRAM running 5.05bpw 5.3 Flash at 30-40t/s

^ bait

>>110044191
yeah because america is the fucking best isn't it?
no country better right?
fucking fall on your face retard and watch your healthcare bills break your little fat riddled heart

>>110044224
strata bros, we fuwarkin wonned

>>110044400
What are you even talking about? Are you suggesting GLM, Kimi, Qwen and DeepSeek AREN'T blatant claude distills??? Not even the Chinese are denying this. Get a grip anon.

>>110042821
I'm running Gemma 4 26B A4B Q4_KM with my gtx 1080 without any issue, so you probably go for much better than that. I'm actually considering trying a stronger model, but unsure if there's stronger overall or if I should go for Q5_KM_M or bigger.

>>110044513
I'm running Gemma 4 26B A4B Q4_KM_M*

>>110042338
> i got an arc pro b60 for $225 today. how an I doing guys?
You could buy used 3060 for less and it would be 3 times faster in Strata. And would work in Windows.
https://github.com/Niko1221/Strata/issues/2046

>>110044236
>>110044460
I see your issue now, holy gay i'll wait I guess

>>110044481
grip this *unpants zip*

File: 1781687414816847.png (284 KB, 1121x613)

284 KB PNG

>>110044392
Why bait? Those are the numbers I get when the hot experts have settled. It's just that they take a while to settle.
When cold, it starts off at 15-20t/s. At 30k context, it goes up to 25-30t/s.

>>110044531
wait for what?

>>110044008
i dun get it, explain

>>110044400
send pics of small cute chink pp pls

File: image.png (127 KB, 1067x244)

127 KB PNG

You and I are on tv.

File: 1545489691130.png (430 KB, 700x900)

430 KB PNG

>>110044578
>developer boards

>>110044578
A lot of insiders post here

>>110044593
I feel good knowing Dario browses this general

>>110044608
It's mostly guys from Meta/Mistral and maybe one or two anthropic/oai employees.

Which model is better of those 2?
Gemma 4 26B A4B Q4_K_M
Gemma 4 12B Q8_0

>>110044633
If you can run the 12B at q8 you can run the 26b at q8 as well

>>110044624
Qwen people definitely also.

>>110044641
I haven't tested 12b at q8 actually. I was comparing them since they use the same amount of memory. Would 31b q4_k_m work for me since the 26b works? It takes 2mins or so max. (8+32GB available)

>>110044593
That's what posters here would like to believe.

>>110043884
Go to bed

>>110041105
>--Anon develops an exotic sub-1M model using synthetic data:
anon have you read about shit like semantics tokenizers and other language based techniques that should improve a ton the models?
back then i was surprised how little they cared about optimizing the language part of the models... i know that i am kinda a linguistic nerd, but it is a shame how many low hanging fruits are there completely ignored
i have saved some papers on that and some others, but then i lost the interest, because ilife and shit. you may find them useful
https://arxiv.org/abs/2304.12404 Semantic Tokenizer for Enhanced Natural Language Processing
https://arxiv.org/abs/2303.09540 SemDeDup: Data-efficient learning at web-scale through semantic deduplication
https://arxiv.org/abs/2305.02301 Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes
https://arxiv.org/abs/2305.02790 BranchNorm: Robustly Scaling Extremely Deep Transformers
https://arxiv.org/abs/2305.09137 Pre-Training to Learn in Context [outperforming larger language models with nearly 4x parameters.]
https://arxiv.org/abs/2305.09515 AR-Diffusion: Auto-Regressive Diffusion Model for Text Generation [100× 600× faster when achieving comparable results]
https://arxiv.org/abs/2305.09596 Identification and Classification of Exoplanets Using Machine Learning Techniques
https://arxiv.org/pdf/2410.23656 Morphological Typology in BPE Subword Productivity and Language Modeling

keep the good work and good luck

>>110044661
>8gb vram
No way you are running the 12B on q8. Even q4 will be prolly close since Gemma's fat fuck kv cache won't let you fit a bigger quant. The both moes should be fine on q4. You might want to drop the mmproj if you still run out.

bitnet bros... did we losted?

Jesus christ the GLM 5.3 flash strata is very good I get 25 t/s versus 6 t/s on ik_llama.cpp and 5 t/s on llama.cpp

It's absolutely over for those projects.

File: hhh.jpg (7 KB, 559x68)

7 KB JPG

>>110044705
I'm running this and it seems to run fine, though after 10-15 prompts I might need to create a new chat.

>>110043595
la la la la

It's taking like, 5-8 minutes to load a 75gb model on my gpu. Meanwhile I can load 30gb models in 1-2 minutes max. Why the difference? I'm confused.

>>110044667
https://desuarchive.org/g/thread/108821001/#q108827851 (Fri 15 May 2026)
>If they're going to do anything novel, it will probably be some kind of DiffusionGemma.
DiffusionGemma release date: 10 Jun 2026

How did he know?

sending images invalidates the prompt cache (in strata) how the fuck do you fix it?

>>110044722
kld?

>>110044829
the only way to fix vibeslop is with more vibeslop