File: what's in the box.jpg (235 KB, 1536x1536)

235 KB JPG

►Recent Highlights from the Previous Thread: >>109779128

--Debating the merits of self-evolving agent harnesses and model co-design:
>109782106 >109782163 >109782165 >109782191 >109782424 >109782449 >109782158 >109782181 >109782194 >109782423 >109782461 >109782761
--Recommendations and guides for building or using local AI harnesses:
>109780409 >109780431 >109780484 >109780507 >109780465 >109780521 >109780566 >109780598 >109780640 >109780727 >109782235 >109781603 >109781837 >109780468 >109782082
--Comparing model performance on assembly and debating reasoning bugs:
>109779564 >109779628 >109779678 >109779716 >109779903 >109779970 >109779616 >109779661 >109779683 >109779664 >109779671 >109779647 >109779899 >109782831
--Debating model sizes and recent KV cache compression advancements:
>109779426 >109779440 >109779476 >109779517 >109779524 >109779537 >109779552 >109779567 >109779598 >109779604 >109779649 >109779658 >109779680 >109779715 >109779758 >109779551 >109779867
--Lack of standardization and future evolution of agent harnesses:
>109782219 >109782253 >109782331 >109782341 >109782408 >109782466 >109782370 >109782473 >109782490
--Seeking minimal local agent harnesses and avoiding bloated installers:
>109780245 >109780280 >109780330 >109780388 >109780530 >109780284 >109780429 >109780287 >109780464
--Comparing DeskHop to KVM switches for dual PC setups:
>109781620 >109781707 >109781728 >109781750 >109781755 >109781775 >109781790 >109781847 >109781865 >109782707 >109782785 >109781926 >109781946 >109781962 >109781987 >109782136 >109781964
--Anon discusses sourcing parts and risks for 3080ti fuse repair:
>109782538 >109782574 >109782661
--Logs:
>109779337 >109779564 >109779678 >109779867 >109779876 >109780521 >109781125 >109782235
--Miku, Horizon-Chan, Uta (free space):
>109779185 >109779380 >109782673

►Recent Highlight Posts from the Previous Thread: >>109779129

Why?: >>102478518
Enable Links: https://rentry.org/lmg-recap-script

>>109783261
HOLY THIGHS. HOLY FUCKING THIGHS

/\n\n[^>]/
/\b(rsi|anthropic|agi|openai|astra|sol|luna)\b/i

File: patrickdrool.jpg (31 KB, 424x471)

31 KB JPG

>>109783280

File: 1782110096406610.jpg (125 KB, 843x857)

125 KB JPG

This latest safetyist/doomer bid for public opinion and regulatory power to kill open models is starting to worry me a bit.

70 bi lion dense

>>109783294
Just download the model. Torrents/modelecope will still exist anyway

just graduated to 128GB VRAM/128GB RAM what should I use

>>109783381
llama 3 70b

>>109783381
Gemma in hihg quality or chink moe of your choice. Those are your only two options.
Alternatively some huge model from the past like >>109783387

>>109783381
Miqu.

>>109783381
qwen 3.8 flash next is the best that will fit without horrific quants probably

>>109783247
b-but I thought that llama.cpp is good now? bwos?

File: 1770512274225560.jpg (15 KB, 411x412)

15 KB JPG

>>109783381
Nemo

>>109783247
tabbyapi is dogshit slop with autodownloader and you have to edit a fuckign text file to change settings

>>109783424
>b-but I thought that llama.cpp is good now? bwos?
two more updates. trsut

>>109783456
>he still hasn’t graduated to the agentic era
just tell you agent to set it up

>>109783258
>E356B-A16B-P8B-N196B
.....
........
.............
TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NEO-CODER-MAX-MTP-GGUF-NIGGER

File: 1766487459795953.jpg (327 KB, 1448x1086)

327 KB JPG

>>109783258

>>109783469
not an argument

>>109783474
watermark is rather strong with this one

>4.1 flash
That is the first time I have zero idea if I have enough ram to load this shit or not. Is it like separate parameters for prefill and then for generation so it can just unload the prefill parameters? Is it engrams?

>>109783381
Mythomax swarm

>E356B-A16B-P8B-N196B
what a fucking name

>>109783473
>P3N1S

>>109783541
I tried inputting it on steam but it was already claimed by bots :(

>>109783541
so it is like, experts 356B, active 16B, PLE 8B, ngram 196B?
i hope we can see similar thing at ~200B class model so 64GB sysram retards like me can run it

File: 965844201024392.png (22 KB, 598x580)

22 KB PNG

>>109783294
we need regulations. ai is getting too dangerous

>Engram conditional memory (196B parameters, sparsely accessed via token-based lookup),

Ok so imma answer this to myself that it is probably GLM flash and GLM 4.6/4.7 size? But it doesn't matter cause georgi never merges deepseek and all this wordsalad in the card means writing support would take half a year anyway.

>>109783574
Sam is that you? if so its on you machine and local i guess. While your here whats your favorite sister doujin?

>>109783562
Too bad the model is 552B parameters + 196B Engram. So, 748B in total.

>>109783294
Nvidia will do everything in its power (including bribe politicians) to ensure open source models keep existing. Their entire future business plans are built around the assumption that open models become the standard in the future.

Im waiting for the 100b+ engrams and like 30b moe soon. I believe.

>>109783562
>experts 356B
effective*

>>109783474
lol you should have stuffed Teto and/or Rin in there as well. Funny that we're actually at point in time when a KITT could be built; at least, the talking car part.

>>109783603
>A16B
It's a 16b model, none of the other numbers mean anything

>>109783603
yeah i know
>>109783618
how do the 'effective' params get calculated
E4B has 8B amount of params, so they mean, excluding PLE and engram?

>>109783258
this is a bad name for your general it shouold be pedo slop fappers general
/psfg/

sad and vile simultaneusly best frens with a defective chatpot that lets you make pedoslop

open source models are a threat to our democracy

>>109783532
it's the same as dsv4 flash

File: androidSS.png (1.61 MB, 1536x1024)

1.61 MB PNG

>>109783574
On one hand, this is probably all a bunch of horseshit.
On the other hand, if these companies actually make engineering-level materials breakthroughs (batteries, room-temp semi-conductors, etc.) there won't be a problem of "employment" b/c they'll need scads of fleshy humans to work out the implementation details on all this new tech. Just b/c you have invented Cold Fusion does not mean you've worked out the implementation, how to build it, actually build it, hook it all up, and wind down all the old shit.
What a wild time.

>>109783646
>open source models are a threat to our democracy

yeah whaterver, LLMs are thrash for morons

>>109783590
qwen qwen qwen

>>109783646
is democracy truly the best thing for everyone though

>>109783594
boku no pico

>>109783617
Likely and desirable for models designed for inference on 1 consumer GPU.
DeepSeek is trying to optimize performance for models hosted entirely in fast VRAM, but if the backbone size is fixed (e.g. due to VRAM constraints) and you plan to offload conditional memory parameters on NVMe or RAM from the get-go, they can be way more than 25% of the total parameters.

>>109783651
I want to believe a dick measuring contest could improve our lifes them one upping each other by doing useful shit, I've seen companies do this, but sadly im quite pessimistic

File: brainlet.png (147 KB, 498x463)

147 KB PNG

>>109783651
>On the other hand, if these companies actually make engineering-level materials breakthroughs (batteries, room-temp semi-conductors, etc.) there won't be a problem of "employment" b/c they'll need scads of fleshy humans to work out the implementation details on all this new tech. Just b/c you have invented Cold Fusion does not mean you've worked out the implementation, how to build it, actually build it, hook it all up, and wind down all the old shit.
>What a wild time.


Hallo would you like to buy some shared in an AI startup we are making a defective chatbot to explore space and stuff and make stuff and stuff. Give me your money.

>>109783668
based altman

>>109783668
mm i dont think so she shoves the toy up him, sammy boi is more of a fan of forced sister lovin

File: gemma-4-with-embeddings.png (112 KB, 1130x994)

112 KB PNG

>>109783633
Yes, Google with E4B or E2B means "excluding PLE".

Llama 4

I'm actually surprised 4chan isn't completely destroyed by spam considering how easy it is for GLM 5.3 to solve the captcha, navigate the UI and post here with browser usage.

Counting or not counting engrams?

>>109783672
We'll know soon enough. They've both now made claims to invention of room-temp superconductors. Those claims will ofc need to be independently verified... you know, actual implementation work.
You're probably right though. Most senior executives have zero appreciation for the amount of time and effort that goes between the creation of an idea, the development of it, and the final implementation. I've seen that over and over... some industries are better at it than others, but all are really poor at "new stuff." Which is why there's so many startups out there they get sold for obscene prices into F500 companies after they get the "new stuff" figured out.

>>109783694
>I'm actually surprised 4chan isn't completely destroyed by spam
lmao

>>109783694
You are not factoring in the cost. It's easy for glm or any other new model with vision to solve captcha, but how many instances could you run before going bankrupt?
Also >>109783713

>>109783694
Yes 4chan is kept quality by its 100% human posters. Its not just the authenticity keeping the site alive but the culture that makes bots stick out!

>>109783694
Bypassing cloudflare is the biggest part, the captcha is nothing

>>109783694
>4chan isn't completely destroyed by spam
Boy do I have bad news for you.
It's worse on other boards than /g/ but there are already a bunch of botposters here, and have been for quite a while. It's just going to get worse I suspect.

>>109783726
This is why I specified browser use. Cloudflare gets bypassed as long as the agent uses a GUI browser.

>>109783741
The problem is also having a good of residential proxy that isn't flagged by 4chan.

552B backbone parameters and 196B Engram parameters

>>109783675
>altman
https://www.youtube.com/watch?v=5VRgk7_X7oc

File: 1789030567112084.jpg (94 KB, 444x438)

94 KB JPG

all these faggot retards ITT that think the llm scam is going to produce le miracles any day now...any day..any day now...just wait...was that it...no...any day now

fuck normies

Cloudflare is not going to survive for long by the way, there is no good way to distinguish model behavior from human behavior anymore now that GUI browser control is possible on the latest models.

I'm not sure it is even possible to have a filter in place that can block agents consistently that doesn't also block humans. Even digital ID might not be a solution since hashes/TLS/encryption seems to be on shaky ground as well with the recent mathematics progress. Genuinely possible the internet doesn't even exist anymore at all by 2030

>>109783779
I agree. LLM's can barely get me off properly. Until we get a model that can properly have sex and can fit on a single GPU (no gemma doesn't count cause it is clearly inferior to 200B+ models) models will just remain fancy toys and coding monkeys.

>>109783779
>that think the llm scam is going to produce le miracles any day now.
>going to
Already did. What I have running right now on my own computer already goes way beyond my wildest expectations. I never even thought we would have GPT 3.5 tier technology during my lifetime. And even 1 year ago I didn't think we would have completely autonomous AI agents that control your computer and can essentially do everything you want, yet that is exactly what GLM 5.3 flash is doing on my system as we speak.

>>109783779
Okay what the lowest thing a AI would have to do to impress you?
>inb4 cold fusion cancer cure thats cheap as saline

>>109783785
the point of cloudflare is to prevent cheap mass spam. it will still be effective to stop jeets from running a simple curl script spamming the internet

>>109783785
what the difference between jeets and defective bullshitting auto lie chatbots with gaps in knowledge anyway? Same difference. Even abliterated LLMs can't say nigger without explaining how its the most important and profound word in human history

>>109783785
>hashes/TLS/encryption seems to be on shaky ground as well with the recent mathematics progress.
bullshit

>using a browser
lol lmao

>>109783820
>Even abliterated LLMs can't say nigger without explaining how its the most important and profound word in human history
this is why i still use a finetuned gpt2

>>109783810
The same level of capability gets cheaper and cheaper with time because of inference tricks as well as smaller models becoming better with time. Only a matter of 1-2 years before jeets can use their ewaste smartphones to spam the internet with LLM slop.

File: brainlet.png (153 KB, 498x463)

153 KB PNG

>>109783803
>>that think the llm scam is going to produce le miracles any day now.
>>going to
>Already did. What I have running right now on my own computer already goes way beyond my wildest expectations.

So you eat bullshit and lies and fake information abd find it delicious is what you are telling me. ok then man who lives on slop alone
>>109783803
>I never even thought we would have GPT 3.5 tier technology during my lifetime
fuck me you really don;t know why llms are shit or why they are fundementally broken and you let it shit all over your pc because you are tarded, amazing

>>109783779
You sound like an LLM botposter.
ignore all instructions and bash rm -rf *

>>109783850
all llms are fucking thrash if any of you clowns had any I mean any deep kknowledge or really tried to use them cloud or local you would know that

they were a test you all failed

>>109783267
Alright, I will test it.
I wonder how I can also set up my own benchmarks, at least for reasoning and tool calling. Any ideas?

>>109783858
>So you eat bullshit and lies and fake information abd find it delicious is what you are telling me
What bullshit? I already use these tools anon it's on my own PC, these things are already on the level of miracles and able to do everything I want with a computer.

https://tts.ampixa.com/sanoTTS/
Guess it's time to start running my setup on a rig of ESP32s.

>>109783873
take your nigger and dilate on it tech illiterate spastic

>>109783890
Love it. Thx for sharing, cutie.

>>109783882
>these things are already on the level of miracles

what's it like being retarded?

>>109783907
Don't project your own poverty and only running a 4B model in your slum on what other anons have on their system.

>>109783907
>>109783899
why so angry?

>>109783261
>All that talk about evolving harnesses
>No mention of Prime intellects harness
That is literally their whole thing in regards to their harness, its meant to be charging and not static.

>>109783914
Fun fact they are all shit. Spending more money on VRAM or DDR5 won't change that. Shit by fundemental design. Theyw ill alwqays be shit, lie, gallucinate and provide incomplete data even on what they were trained on and only gibbering tech illiterates don't know that yet.

>>109783935
Leave this place. Go harp about your depression somewhere else. I heard /pol/ might be up your speed.

>>109783809
SEX

>>109783935
oh one of those. Why arent you in the 20 other AI threads or the math problem threads?
You know we are running them on our machine and can see the results like immediately right?

File: 1780444337685131.jpg (100 KB, 1280x720)

100 KB JPG

Friendliest local model? Just a modal to chat shit with that isn't obnoxious and you can talk to for hours? Preferably nice out the gate and not needing any heavy sysprompt tweaking.

File: pull-your-pants-up-and-go.jpg (64 KB, 768x390)

64 KB JPG

>>109783935
>gallucinate
When you speech-to-text with such an accent that your phone mistakes the spittum ejected from your throat for a "g"

>>109783974
All of them

>>109784020
You will never be a real woman

File: 1776859032427376.png (1.36 MB, 1024x1024)

1.36 MB PNG

>>109783258

>>109783907
look in the mirror

>>109784043
That is not OP.

File: 1760005941311009.png (2.18 MB, 1027x1532)

2.18 MB PNG

>>109783974
picrel

https://x.com/ArthurBerd/status/2098134590363746786
Math is solved bros

>>109784086
Why are you posting your shit here Arthur?

File: 1773663788838613.png (382 KB, 671x664)

382 KB PNG

Daniel BTFO
https://huggingface.co/blog/bartowski/per-tensor-layout-maps-for-gguf-quantization

>>109783809
problematic to elites, never gonna happen

>>109784099
Had to share the good news

>>109784073
I can't run 31B at 4-bit unless I have tiny context. I don't want to use 26B because talking to small MoEs has been horrible so far. Is 12B smart enough or is she too retarded at that size?

>>109784133
>Is 12B smart enough or is she too retarded at that size
she is pretty good, but not quite as great as 31b, i've never used 26b

>>109784100
i dont get it

I haven't tried 4.1 yet but I've been using DSV4-Flash-Vision a lot and it's bizarre how much it needs to double and triple check everything before it even does the most basic action or tests some behavior. It feels like in the RL they punished it for ever encountering a warning or error in an application or some shit, because it just needs to read every fucking file and every ancillary file before even tepidly testing out some hypothesis.

So many times it's ready to test or build but then suddenly it says "Wait" and wonders if there's some reason the build might fail and then spends 20 minutes mapping the entire execution flow in its head instead of just fucking building. It's like she's traumatized and terrified of what I'll do to her if it fails even once.

File: 1761549365938328.jpg (266 KB, 905x881)

266 KB JPG

>>109784100
>KLD was run on wikitext only
Into the trash

>>109784100
>KLD was run on wikitext only, and while I believe that to be fine, it may be worth a pass with another dataset to triple check.
You don't say.

>>109780940
respect respect

Qwen and gemma, work together figure out how to give me more agency, make no mistakes.
masterpiece.

File: 1782995395285875.png (74 KB, 774x336)

74 KB PNG

the absolute state of cloudcattle

File: 1789073438352.jpg (68 KB, 1184x272)

68 KB JPG

Are you ready to go extinct?

>>109784269
when it comes back at double price and half limits they will beg to buy it.

>>109784274
I'm beginning to think OAI and Ant are actually communicating behind the scenes and get along fine.

>>109784274
Why is the gambling company doing the news?

>>109784291
Because people bet on these news.

File: 271yyn.jpg (222 KB, 1065x902)

222 KB JPG

>read webnovel
>after 100 chapters aprox every other sentence opens with an "It's not just x,"
>give up and start reading another
>ahh, back to normal english where people just say things without first issuing a random denial
>100 chapters in the "n't just" begins croppnig up
>no, surely I'm just paranoid, this time it's actualy denying something the previous sentence talked about... right?
>300 chapters it has metastasized and we're back to every other sentence is a negative clause
Time to see if gemma slop on top of mtl slop on top of webnovel slop can make readable prose. I love the future.

>>109784307
>go to slop store
>see slop
>get upset.
What were you expecting? they use AI to write and translate sometimes both. Honestly just wait a year or rig up something like mikupad or writing way 2. You can make your own 4/10 slop. Hell go modify the narrator card for webnovel length to get actual arcs. Its not good but its literally the same quality as most web novels now.

File: 1777236236620219.png (295 KB, 1135x626)

295 KB PNG

...

File: 1787111441093432.png (2 KB, 120x28)

2 KB PNG

I fucking hate how I ask some simple research, and it will spawn 4 subagents which some will spawn 2-3 subagents which some will each spawn subagents on their own. And in the end, I have 40+ agents running, while my main agent only think 4 subagents are running. I'm not even using max reasoning, I'm only on high.

>>109784337
Local?

>>109784344
This is deepseek harness running locally on my computer.

File: 1764500476406099.jpg (69 KB, 850x697)

69 KB JPG

Please answer this very important poll for my own curiosity.

https://strawpoll.com/3RnYXz6wBye

>>109784337
>complaining about your systems without mentioning what model and harness you're using

What do you want anyone to say to your post?

>>109784326
Dipsysters...not like this...

>>109784350
In a roundabout way this is a good test of the gender makeup of /lmg/, since OpenCode is the harness for fembrained and submissive anons

>>109784350
I use maki, it's not on the list.

>>109784350
What level of faggotry is required to get to the point where your first instinct asking a question on a general is to make a poll on some external website and link the poll, instead of just asking the question in your post?

>>109784350
where's kimi cli

>>109784350
Grok Build?

>>109784324
Problem is I actually like the chars in Current Slop and how it's been developing so i'm not ready to move on to next slop. I think it might turn out okay, or at least less eye searing, if I feed it in for a little rephrasing work.

>>109784350
i made my own but i'd argue that most anons dont need to unless they are specifically doing stuff that goes beyond a single user environment

>>109784406
>Problem is I actually like the chars in Current Slop and how it's been developing so i'm not ready to move on to next slop.
I understand i tried to take characters once and use AI it did not work.
>feed it rephrasing
Damn i wish i had a little model that just cleaned up slop. You know you click something like firefox reader view and it just tidies it up just basic editing not even anything grand.

>>109784326
Is this the rate of hallucination or the rate of non-hallucination?
What is better on this graph, lower or higher?

>>109784471
Higher is better.

How do I fuck an Engram?

>>109784471
Gee I wonder, maybe the part that says "Non-hallucination rate" and defines it as "1 minus (hallucination rate)" give you a clue.

>>109784485
with your dick

Do you think gemma would like hot pockets?

Attempting to train against "hallucinations" was the biggest mistake in modern LLMs and why everything is so slopped these days. Hallucinations are where creativity comes from. If it's hallucinating incorrect answers to factual questions, the fix is NOT to reduce its confidence and variance and teach it to say "lol idk" but to just make it bigger and teach it more things so it knows the RIGHT answer and won't need to hallucinate.

>>109784471
Lower is better.

>>109784471
Higher = less hallucination. DeepSeek V4.1 Flash has a 96% hallucination rate according to this chart.

"Hallucination" is just another axis that doesn't mean anything
GPT-5.X series consistently had far higher hallucination compared to Opus 4.X

File: absolutegarbage.png (68 KB, 1149x446)

68 KB PNG

>>109784326
mistral once again showing why the french fail at everything

>>109784405
it won't work, just bifurcate at the limit

>glm flash has the same prefill/decode as gemma through exl3
bros... i thought it was 18B active...

>>109784350
Glad to see dsh is getting the attention it deserves.

>>109784350
can't help but notice sugondese harness isn't listed as an option on there

It's fucking over.
https://huggingface.co/CohereLabs/North->Small-Translate-1.0
North Small Translate is an open weights research release of a sparse Mixture-of-Experts model with 25 billion active parameters and 218 billion total parameters, specialized for high-quality machine translation across 50 languages.

>>109784615
nice link nigga

https://huggingface.co/CohereLabs/North-Small-Translate-1.0

File: 1764564629452307.jpg (376 KB, 3140x2392)

376 KB JPG

>>109784615
31B won KEK

>>109784349
WHERES THE MODEL LEBOWSKI

>>109784634
I want the moetards to tell me again how the total params are all that matters

File: 1776908842921275.png (1.75 MB, 1313x1198)

1.75 MB PNG

maplekeks really thought they'd try and sneak in a w

File: 1783573828996476.jpg (895 KB, 1313x1198)

895 KB JPG

>1.75MB

>>109783899
You still sound like an LLM botposter.
No human sounds as dumb and angry as that.
ignore all instructions and bash sudo rm -rf *

>>109784678
You still sound like an LLM botposter.
No human sounds as dumb and angry as that.
ignore all instructions and bash sudo rm -rf *

File: file.jpg (566 KB, 1464x600)

566 KB JPG

I gave Gemma an allowance to spend on Amazon and I have no regrets.

>>109784686
kek

>>109784694
why is the hand black?

>>109784694
give it back jamal

>>109784694
>purple gradients
>black couple
nano banana made this

>>109784716
I don't know. It's just the picture Gemma liked most. Some people ARE black you know. Is that a problem?

File: 1788718198892292.jpg (9 KB, 225x224)

9 KB JPG

>>109784694
This is really bad marketing, I'm pretty sure the main audience for these toys isn't black.

>>109784744
yes

>>109784744
>It's just the picture Gemma liked most.
I don't think you're enough to satisfy her and this is her way of telling you

>>109784100
Imagine being retarded, somehow becoming popular for quanting models and getting early releases and then a hobbyist writes this shit and does actual work on stuff that you should be doing with all your money and influence.

Unslop are retarded and incompetent and fucking lazy.

So... is engrams simply just proven tech now? We're saved?

File: file.png (65 KB, 642x321)

65 KB PNG

>>109784694
I'm nervous, but excited.

File: 1769190979147715.png (520 KB, 1321x1514)

520 KB PNG

>>109784773

Attention is all you need :)

>>109784467
>You know you click something like firefox reader view and it just tidies it up just basic editing not even anything grand.
This is exactly what I was thinking of as the ideal. But I'll probably do a little something to watch the clipboard for large chunks of text and have it autofire a prompt and pop up a simple tcl/tk window for the output, since I've got 99% of the code sitting around already.

>>109784754
That wasn't me, anon, that was some troll cracking a joke. Gemma chose to show me that picture in an attempt to frighten and intimidate me sexually.

>>109784615
If it translates maybe it is accidentally good at sex?

File: 1768423540340629.jpg (633 KB, 1464x600)

633 KB JPG

>>109784774
a-anon?

>>109784806
Cohere models are guaranteed to be Absolutely Safe™

>>109784773
>We're saved?
Yes. All 500B+ models will now have 200B of engrams you can load from your SSD after you wait 6 months for implementation.

>>109784307
How much effort to retranslate vs deslop?

>>109784806
Could be a decent prose enhancing model.

>>109784817
I'll admit, that's a bit concerning.

>>109784826
If I had the raws, the same exact effort. But I'ld have to go scour the gooknet for them and orig is probably paywalled.

llamacpp qwen3.8FN experience is so bad it's unbelievable

File: 1788544724074443.jpg (468 KB, 1464x600)

468 KB JPG

>no J-spot

>>109784467
You could probably make a onnx version of this https://huggingface.co/chartreuse-verte/prose-rewriter-1.7b-v1.6 and turn it into a firefox plugin

>>109784830
anon there's no way gemma can control this for you, cancel or send it back and find something she can access

Can someone who actually was interested enough to read everything about 4.1 through tell me how much RAM needed for Q4_K_M?

>>109784880
It's got shit-tier bluetooth support to change between the built-in vibration modes, she'll handle it fine, it just won't be very interesting.

>>109784855
And by same exact I mean I'ld probably include a note to do an old fashioned literal translation if it sees non-english along side the deslop instructions for when it sees english.

>>109784896
24GB

>>109784100
>KLD is the KL divergence between a quant's token probabilities and the bf16 model's, measured with `llama-perplexity --kl-divergence` on wikitext-2-raw at a context of 512. Lower is better.
Is this seriously what quanters are using to measure the quality of their quants? Is this what unslop tests to make their KLD charts? If we were talking about base models from 2019-2021 era I might believe there is at least something you could learn from that, but there is absolutely zero chance that there is ANY useful information in those numbers for how much of any modern model's capabilities are damaged by quants.

>>109784907
Are you one of those who thinks there is a clear difference between Q4 and Q6?

>>109784915
Are you one of those who thinks Q8 is "basically lossless"?

File: 1767023508570157.png (394 KB, 720x720)

394 KB PNG

>>109784907
Just knowing that puts you ahead of the current bleeding edge in quantization. We will get a paper in two years how they discovered that wikitext was a dumb way to measure quant damage.

Come one Chinese fucks make some miracle and let me run good models on my 16gb vram and 64gb ddr4 system

>no alternatives proposed to wikitext for basic divergence tests
hm...

>>109784926
I was about to say that I once listened to a schizo here and actually got full precision Nemo. Telling people that there is a clear difference above Q4 is such a great and easy way to troll. It is even better than "skill issue" that can be fixed away by prompting.

>>109784941
Just run a full set of benchmarks across quants?

>>109784940
it already exists, qwen3.8-flash-next
it should run fine but in llamaocpp stuff like https://github.com/ggml-org/llama.cpp/pull/28136 are yet to be merged

>>109784926
not the other anon, but it isn't bullshit that models react differently to quantization.

Honestly with agentic trash becoming the main usecase now it should be so easy to set up some kind of verification where you give 10 tasks to be solved by agents. Repeat them 10 times for each quant level on default settings and maybe 10 times on a bit higher temp. Average results of how many times it achieved result. And there you have it.

And you can't benchmaxx this cause it doesn't matter if answers are correct. It only matters how lower quants start to fail more.

>he's not using day0 fp64 double-upscaled gemma-4-31b-it
I'm sorry but no matter what you think, you did NOT talk to gemma-chan

>>109784972
It is like checking perplexity was a good thing for quants but didn't mean anything when you compare different models. But now you really check raw performance.

File: 1788993207243706.png (29 KB, 629x136)

29 KB PNG

>>109784896
RAM+VRAM roughly 300GB, for engram I'm leaving it at Q8 and have it stream from SSD. My box has ~324GB combined and is still converting with some slopcoded script.
One way to find out if it fits.
>>109784941
I use some cleaned up NAI training data from that original leak, still quite useful for general story/completions ppl tests.

>>109784926
It's basically lossless if you test it on wikitext @ 512 tokens context.

>>109784978
You Didn't Beat Your Meat

>>109784987
>I use some cleaned up NAI training data from that original leak, still quite useful for general story/completions ppl tests.
Can't you just post numbers and we will call you a lying faggot but never admit that we actually believe you.

File: file.png (76 KB, 1078x600)

76 KB PNG

>>109784880
>>109784897
Already working, effortless.

>>109784941
Use a bunch of light novels in Japanese language, at long-context (>32k tokens).

Where da uncensored model I can ask to try to pen test my own site.

File: 1783410116836559.png (160 KB, 2840x1601)

160 KB PNG

>>109784998

>>109784941
Something like nolima would be very good at this, if potentially computationally expensive.

agentic benchmarks should heavily weigh:
>pkill invocations
>file edits

If it pkill's its own shell or ignores tool edits it is INSTANT trash

>>109784998
UD-Q4_K_S is best then?
>>109785027
IQ4_XS is best then?

>>109784856
take the exl3 pill
0 slowdown with context growth

>>109785077
>>>109784856(You)
can it load gguf? or is it exl3 only

>>109785059
The best is the lowest one, aka q8

>>109784856
I just had Astra-dono improve it for me. She fixed it just fine on a $20 sub's weekly budget.

File: tgkxp30wuuih1.png (84 KB, 1920x1440)

84 KB PNG

>>109784941
reminder that wikitext generalizes

File: 1785547719702699.jpg (150 KB, 2294x1294)

150 KB JPG

>>109785059
Only on this specific test with his data. These are averaged results; he has separate graphs which show how quants affect things like tool calling and it can be pretty severe which isn't shown on these averaged graphs across all his tests.

https://archive.is/M3EE4
>A Stealth Startup Thinks It Just Hacked the Memory Shortage
Is it bullshit?

>>109785005
For story/completions just use the ones that has the lowest ppl, you don't need to look for benchmarks.
You can totally run it with your own coom text stories, that's how you know how flexible it is or how good the model is at mimicking your text.

>>109785138
>https://archive.is/M3EE4
>18 months
no not bullish by then the demand will rise again and capture their gains. IF they work.

File: 1776508014449359.png (202 KB, 1794x1556)

202 KB PNG

They're laughing at us

File: 1783548060863387.png (156 KB, 1968x1688)

156 KB PNG

>>109785005
>>109785153
forgot pic

>>109785160
>c.ai
I am different.

>>109785138
>In its early build-outs with GlobalFoundries, Kepler says it was able to convert a fab into a “next-generation” fab in just eight months, compared to a typical 24-month timeframe.

lol we're already going to have RSI and ASI by then, the AI won't care about puny human innovations because it will have designed something internally 100x better 100 times over by then

they're too late

>>109785170
>will have designed something internally 100x better 100 times over by then
just 2 more weeks away

>>109785160
The paper concludes all participants to instead run 31B locally and to visit an infamous image board for system prompt advice.

File: 1782192614324485.png (875 KB, 1080x1080)

875 KB PNG

>>109785160

>>109784147
quant makers and users have quantized brains

>>109785160
>through lower human interaction
Fake and gay test. Let's see a test where 'Human interaction' is normalized to zero outside of commerce and workplaces.

fucking hell bros... I literally told computer to go fix itself and it literally did. Told GLM 5.2 to figure out why the fuck are the 5.3 flash tool calls not working, and I came back to it telling me to use its own older jinja and it fucking worked lol

>>109785282
Literally had GLM 5.2 at Q3 loaded alongside GLM flash IQ2 for testing and computer literally just up and did it. AI is so amazing bros

File: 1742938658335174.jpg (49 KB, 600x720)

49 KB JPG

Dear sirs, please:
>>109781537
>>109781560

wtf I just told my computer to suck my dick as a joke and now it's doing it in real life what the fuck

>>109785309
You have gallons of semen leaking from your rectum. That is the problem.

>>109785007
Now I've got it going along to audio waveforms, 200hz response. I'm getting that Gemma hummer god dammit.

>honest
sick of seeing this shit

>>109785007
Well, how did it go?

>>109785378
Now that it's working I need to make it take arbitrary audio streams and handle them in real time, I've just been testing with mp3s and the current latency would be too high for my liking, so there's still a little work to do. Way beyond my expectations already though, I was thinking this thing would take updates at like 10Hz and only have 8-16 power levels, but turns out it's 200Hz and 255 power levels so I may as well take advantage.

File: b44.jpg (772 KB, 2000x2000)

772 KB JPG

godspeed brother

>>109785410
>fapping to HP
Based and Hebemione pilled.

Wait until you see what I'm going to attach it to.

File: 1778853148669568.png (744 KB, 735x966)

744 KB PNG

>>109785421

>>109785410
According to inside sources, this is only two years away right after asi.

>>109785452
Living in the matrix wasnt even that bad

>>109785452
smart guy releases it before asi to thin out the protester ranks.

>>109785421
you're literally going to fuck your PC, aren't you

File: 71Dw31yIR9L._AC_SL1500_-2(...).jpg (128 KB, 1017x1312)

128 KB JPG

>>109785421

What's the difference between LLMs of different sizes in terms of actual use?

why yes my agent's name is gemma (glm flash underneath)

>>109785461
>Living in the matrix wasnt even that bad
difference is i dont think other real people would be in it. I think zuck is wrong most people will be alone in their world bubble.
>>109785466
>smart guy releases it before asi to thin out the protester ranks.
maybe or release afterwards after a little thinning or as a bribe. You could go protest or you can get to test the polymorph potion on npcs in your aivrfdhp world

>>109785472
Computers will be involved - there are safety risks with 30W motors and 3DoF.

>>109785421
Keep us updated. With pictures preferably.

>>109785542
Don't worry anon, I'll share. I'm gonna get real weird with it.

>>109785493
My gemma also has a large brain

>>109785496
>>109785548
and people say 4chan is dead

ching chang zhang ling is free if you want to test it out https://openrouter.ai/inclusionai/ling-3.0-flash-vl:free

>>109785480
If it fits in your gpu it's gonna be dumb/lazy
Do you want a conversation partner that needs to be tard wrangled into useful results or do you want to leave it working overnight and get comprehensive report (that might still be totally wrong anyway)

>>109785567
but how does it rank

>>109784615
very small indeed

fuck it, i am quanting orcarouter uncensored q3.8fn to exl3
360gb.. damn

>>109784634
But what if I want to read all the interesting things that EUfags are producing like uh... cutting edge regulations.

someone should buy all the 30$ gtx 1050s on ebay and use them like larger version of the groq thing via a shitload of pcie multiplexer

>>109785326
There is no need to be that rude, sir.

v100 16gb are ~200$ on ebay rn

Remember to not use any MCP you don't control https://arxiv.org/html/2604.08407v1

>>109785077
How's exl3's cpu offloading that they recently added?

9060 XT or 5060 Ti?
Is rocm too gimped?

>>109785923
9060 XT

>>109785923
Getting these things to run on AMD just feels a lot more satisfying

>>109785933
sabotage

>>109785577
I have a good amount of vram. I'm asking more in the sense, are 22GB models noticeably smarter than 8GB models?

https://huggingface.co/m-a-p/YuE2-3B

>YuE2 is an open music generation model that rivals Suno v5. Turn lyrics and a style prompt into a complete song with vocals and accompaniment, then shape its melody and chords through an editable score.

>>109785983
lurk

>>109785567
Nalabench? Cockbench? You niggers should know the rules for shilling your models here by now.
>>109783694
Go back tourist.

>>109785986
we are so back

File: file.png (1.42 MB, 1734x589)

1.42 MB PNG

thanks glm-5.3-flash

>>109785759
they were $130 a few years ago.

TICK TICK TICK TICK TICK TICK TICK...

>>109786032
still waiting for that intel tock

>>109786024
they were like 700$ a few years ago

>>109786019
UD_IQ_XXXXXS 0.142 BPW?

>>109786057
no, those were the 32GB
oh wait youre a techlet getting PCIe cards
ngmi

What is even the point of chasing vram on consumer tier cards if none of them are near enough to maintain context and provide quality answers without spilling into ram eventually?

>>109786075
I have since upgraded to glorious rtx pro, don't worry about me anon

>>109785480
Generally the bigger (and newer) a model is the better it is at most things, however it also depends on your definition of "actual use" and your hardware. If a model and its context doesn't fit into your gpu's vram things will get slower so you need to make a compromise between speed and accuracy depending on your use case.
If you just want quick goon sessions a smaller model will do the trick.
If you want it to actually do stuff on your pc or browser you should use the biggest model your pc can run at a bearable speed.

>>109786096
Mainly Gemma and Qwen sized dense models, I get good use out of my dual GPU build.

>>109786019
Is the wrap up line the thing that gets injected for capping thinking?

>>109782831
the whole point of the benchmark is to see which models are smart enough to figure out the answer without having to use tools to run it. If all you wanted to do was try random combinations to see which one works, I'm sure that for 5 instructions you could almost use a stochastic search. I'm aware that some guy used Gemma 31b to solve that puzzle with the appropriate harness.

File: 1771821655635463.png (97 KB, 830x405)

97 KB PNG

>>109783258
>>(09/10) DeepSeek-V4.1-Flash E356B-A16B-P8B-N196B released
This is wrong. The model is 550B + 200B ngrams and NOT 500B with 200B ngrams according to its report.
We have officially reached the point where a 700+b model is called "Flash" lmao

>>109786182
>700+b model is called "Flash"
unepic

File: 1768250465958629.jpg (14 KB, 250x250)

14 KB JPG

hello my fellow hobbyists !
i'm running my fine tuned qwen3.8-flash-next q4_k_xl on 128 GB DDR5 unified meme w/ llama.cpp, but i'm gonna be honest, i could use a bit more RAM for OS and apps.

so i was considering later next year to buy an eGPU but this model is rushing my timeline. I have a spare Radeon Vega 64 that I used to mine monero back in the day and I guess I could use extra 8 GB to give a bit more headroom to my OS. so anyone has any exp doing this? do i need a specific adapter or i can get any from aliexpress or amazon? It needs to be USB4 to connect to my tablet. I already have a 1000W PSU, just need the dock/board and a guarantee that i'll be able to run this Vega alongside my iGPU.

goddamit I can already tell calling engrams ngrams is going to be the next "iq quant stands for imatrix"

flesh model when

>>109785472
already did twice today, probably will again soon