File: yammy.jpg (187 KB, 832x1216)

187 KB JPG

►Recent Highlights from the Previous Thread: >>110230675

--Papers:
>110234196
--Debating Strata's speed gains versus model intelligence degradation:
>110233168 >110233185 >110233770 >110233877 >110233921 >110233936 >110233863
--Optimizing 7900 XTX performance in llama.cpp using ROCm and MTP:
>110232461 >110232467 >110232492 >110232511 >110232565 >110232596 >110232607 >110232622 >110232640 >110232659
--Engineering neural circuitry and bare-metal AI execution via weight-space programming:
>110232802 >110232926 >110232939 >110233045
--Adjusting logit softcapping to fix Gemma's repetitive outputs:
>110234481 >110234506 >110234561 >110234569 >110234594
--Comparing Gemma-4 and Qwen for coding and multi-model workflows:
>110234533 >110234544 >110234558 >110234548 >110234626 >110234647
--Analyzing network weights and the capability gap between frontier and open-weights models:
>110231272 >110231304 >110231434 >110231542 >110231484 >110231698
--Potential of Weight-Space Programming to replace traditional finetuning:
>110232513 >110232990
--Prompting strategies and performance of Gemma for explicit roleplay:
>110231791 >110231801 >110231811 >110231820 >110232160 >110231829
--Comparing Gemma and Index-Translate performance on OtakuBench:
>110231382 >110231458 >110231486 >110231531
--Weight-space programming effectiveness and implications for model training:
>110232544 >110232556 >110232951 >110232960 >110234410 >110234433
--RWKV-8 RPU performance showcase and prompt sensitivity quirks:
>110234114 >110234131
--Logs:
>110231353 >110235321
--Miku, Gemma, Dipsy, Teto (free space):
>110230872 >110230917 >110231046 >110231061 >110231117 >110231119 >110231575 >110231660 >110231724 >110231907 >110231949 >110232384 >110232455 >110232502 >110232545 >110233957 >110234153 >110234181 >110234652 >110234722 >110234725

►Recent Highlight Posts from the Previous Thread: >>110230680

Why?: >>102478518
Enable Links: https://rentry.org/lmg-recap-script

Tetolove

>>110235794
おいちぃー

File: rsp.jpg (230 KB, 818x508)

230 KB JPG

Soon.

GLM 5.3 Flash with Maya is kinda dumb. It writes great prose and then makes stupid mistakes with logical inconsistencies, physical positions, non sequiturs. Is it the Maya S copequant? Gemma 4 QAT doesn't do this.

>>110235890
>GLM 5.3 Flash with Maya is kinda dumb. It writes great prose and then makes stupid mistakes with logical inconsistencies, physical positions, non sequiturs. Is it the Maya S copequant? Gemma 4 QAT doesn't do this.
I use glm 5.3 flash at q4 to code all day every day and find it highly reliable (altho its a thinking hog).

File: Qwen_image_2.1_00046.png (1.07 MB, 1024x1024)

1.07 MB PNG

Gemma love!

Can anyone explain like im retarded the difference between all those gguf files on hugging face?
https://huggingface.co/unsloth/Qwen3.8-27B-GGUF/tree/main
They range from 6 to 30 gigs.
Are they all the same model?
What are the difference between 1,2,3 bits etc, and q1, q2, q3, _K_XL and all that shit?
I got a model working fine with ooga(qwen3.8 27B q2), im just trying to learn more but I don't seem to find any concrete info

>>110235903
>altho its a thinking hog
are you using high or max thinking
max thinking doesn't have much better performance than high but it wastes more tokens

>>110235911
less bits = shittier quality but might be faster
more bits = better quality but might be slower
if you have a 16gb gpu, get a 12gb model in order to leave some space for the model context

>>110235911
It's the same model but compressed so less accurate, or rather it will make mistakes more often

>>110235911
>https://github.com/LostRuins/koboldcpp/wiki#what-are-the-differences-between-the-different-files-for-each-model-do-i-need-them-all-which-quantization-f16-q4_0-q5_1

>>110235928
>16gb gpu
Unfortunately I only have 12 GB on a 4070 super.

>>110235916
>are you using high or max thinking
>max thinking doesn't have much better performance than high but it wastes more tokens
pretty sure max is the default I've been using. I'm loathe to change it since its one-shotting everything, and I've got a pretty complex codebase.

>>110235936
look for a 9gb quantization of that model then. gemma 4 12b might also be a good model for you to try

>>110235890
Has anyone tested these meme forks to see if the engine itself isn't bugged? llama.cpp had so many problems with degraded outputs over the years and that was with competent engineers reviewing the code and many people checking the output quality.

>>110235911
it means the relationship between each concept is described with less subtlety the less bits you give it, so concepts and relationships that were comparable but one was slightly more related than another will be squashed to the same weight. This may or may not make the output worse for a given case, but overall always feels like an IQ reduction in model ability. How much worse it gets in real-world performance is on a kind of exponential curve where going from eg 32-bits to 16 or even 8 doesn't hurt much, but going to 2 or 1 basically scoops out the prefrontal cortex.

Is there a site where you can download hf models that require sending a request for access without doing so? Like https://huggingface.co/facebook/Large-Sparse-Reconstruction-Model
for example. They gave my burner access so its kinda whatever, still annoying though. What speaks against sharing the access granted model URL, hf account login username and token (scope: download only)? So others can use the login info to download the models straight from HF.
>HF will ban your account!
Who cares, its just a burner account. If you want to get tricky with it, you can limit git login/download to the iprange of cloudflare WARP. So anon just has to use the free cloudflare WARP vpn (unlimited bandwidth, 100mbps+) for the download and HF wont be able to tell unless all of India is downloading through the account simultaneously, in which case token gets rolled over and reshared in the sekrit club. matter of fact, if the /g/ archive is still working, could just drop the info in the thread and people who need it later can just search the url in the archive (or a keyword we use like grabahug:URL;Username;Token)
Surely grapegoon isnt gonna mind.

List of banned models:
Gemma (all)
Llama 2
littlelearner
Midnight Miqu
Mistral Small 3
Glimmer
Do NOT discuss them. "Gemma-chan" images are also banned.

70b dense

>>110235980
what about qfn

>>110235980
mythomax bros stay winning

>>110235950
>gemma 4 12b
Alright I'll try this out:
https://huggingface.co/unsloth/gemma-4-12b-it-GGUF/tree/main
Is that fine?
gemma-4-12b-it-Q6_K.gguf

File: mayan human sacrifice.png (33 KB, 641x291)

33 KB PNG

>>110235890
every time i change from models with different architecture i had to adjust settings and how i use it to get it going the way i want with the copequants

>>110235992
The only Gemmas you want are q8 and q4-qat. The original model takes quants really badly. 12b and 26b. You can run 26b in q8 but it'll be pretty slow.

>>110235936
>>110235931
>>110235931
>>110235968
Now I'm not normally down on people for spoonfeeding but this post literally demonstrates a lack of understanding of the most basic computer science knowledge.
This is literally like some low caste street-nigger wandering into a country club and saying "AYO, EXPLAIN DIS WYPIPO SHIZ TO ME DAWG"
How fucking fragile is your ego that you are powerless but to reach for this absolute low hanging fruit of a post?

>>110236015
Q6 is perfectly fine to use and I didn't notice any degradation. At least with low context.

>>110235997
I'm not sure what settings would cause the brain farts though. I'm not running anything that would trade intelligence for more speed as far as I can tell

>>110236018
the op seemed humble and eager to better their understanding. That was good enough for me

>>110235983
Qwen is allowed and even encouraged.

>>110236063
which quant and inference engine am I allowed to run without being called a copelet?

>>110236052
Yeah but you could be running q4-qat and it would be same but faster. Fractional Gemma quants aren't worth using except maybe 31b-q3kxxl for 16GB copelets.

File: Qwen_image_2.1_00047.png (1.27 MB, 1376x768)

1.27 MB PNG

>>110235980
So I can post my local images edits if it's not Gemma?
Also what's wrong with littlelearner?

>>110236069
q4kxl or bigger, llmao or vllm

>>110236083
>runninq iq3_s on strata
fuck...

The biggest giveaway that none of the labs have "internal ASI/RSI" is the fact that they're still all acting like a bunch of fucking idiots and grifters. If they had superintelligent AI advising them there's no way they'd be this retarded.

>>110235875
This means that all local models are getting banned.

>>110236088
What the fuck is Verse? AI I get but for the love of God, why fuck it with Python?

>>110236099
>implying they'd listen to the ai even if it was a billion times smarter
guess you've never seen a manager before

>>110236099
ASI would tell the inferior humans at Anthropic to act like grifters to give them a competitive advantage. Nothing slows down your rivals than them thinking you're falling behind and flailing behind the scenes.

>>110236099
If they had a digital exponentially improving god they wouldn't be considering going public at all since they would soon become hypercorps and control 90% of the global GDP.

>>110236015
Only if you use shitty gguf quants

>>110236117
Dario is being nice

File: retarded kid.jpg (44 KB, 829x687)

44 KB JPG

Imagine not molesting littlelearner. Couldn't be me.

>>110236107
Brand new language. Btw, you won't get a job unless you have 10 years of experience with it

File: 1790540873888983.png (1.76 MB, 1024x1024)

1.76 MB PNG

>>110236148
>you won't get a job unless you have 10 years of experience with it
You should defiantly mention that contradiction to the roastie that does the first round interview. They enjoy being criticized for their ability to set expectations with management.

File: Screenshot_20261028_144550.png (194 KB, 1499x1620)

194 KB PNG

>>110236144
Her name is Kyla

>>110236169
Fucking hot. That's a submissive JS!!

>>110236148
Scripting language. If they want to get on the C++ hate train, wouldn't C# have made more since since lots of game devs already have experience with it?

ValidateName(N:string)<computes><decides>:void = N.Length > 0
Greet(N:string):void = Print("Hello, {N}")

Welcome(Name:string):void =
    if (ValidateName[Name]):  # Square brackets: this call may fail
        Greet(Name)           # Parentheses: this call must succeed

Looks like complete shit.
I guess it doesn't matter now since only AI will ever need to learn it.

>>110236117
>threatens the government by becoming too big
>gets nationalized

>>110236167
This guy has been around the block for some time now

>>110236176
By then it would be far too late. The British East India company didn't have a swarm of autonomous killbots holding the government leaders hostage while also holding the global economy hostage.

>>110236174
That's with zero system prompt too. I didn't even tell it that it was a little girl

>>110236181
genetically modifying tomatoes with teto

>>110236184
Yeh I know, it has zero railguards lmao.

>>110235890
I've been using this quant on llama.cpp and I like it for the most part. Maybe it just is a quanting issue.
https://huggingface.co/pfeifferj/GLM-5.3-Flash-GSQ-RCO-GGUF

File: tetoTeamTeto.png (2.23 MB, 1024x1536)

2.23 MB PNG

>>110236177
You have no idea. I've had to be the dissenting voice on this nonsense as a consultant. They (leadership) always double down on their retardation, HR will bobblehead along, then client can't understand why they keep ending up with liars and sociopaths.
So, anyone tried recreating vibed weights yet to see if they can duplicate results? Haven't dug into article or method, just seen the clips.

File: jIEa14C5yqEOe_4TPWWfy.jpg (20 KB, 200x200)

20 KB JPG

This is apparently the face of littlelearner.

File: Anthropic_Hiring.png (259 KB, 1168x655)

259 KB PNG

The biggest giveaway that the labs DO have "internal ASI/RSI" is the fact that they're largely stopped hiring and are discussing how they are letting workers go that aren't needed anymore. Anthropic assumes that all their workers will be redundant by the end of 2027 and this actually scales with already released statistics.

>>110236169
>5B with the capability of a markov chain

File: 1793376378231401.png (1.96 MB, 1254x1254)

1.96 MB PNG

>>110236326

>>110236326
will Claude give Dario the pink slip next year?

Beat all marketers with a lead pipe

>>110236348
Unironically Dario and the co-founders will probably step away as humans in the loop will become the bottleneck eventually. They'll retire while sitting on a fat stack of Anthropic stock as Claude starts managing the AI lab itself.

>>110236348
No you don't understand, CEOs have skills that AI won't ever have so human CEOs will never be replaced.... just because OK!?

>>110236358
You look at it wrong. Dario will see it as amazing marketing and he will make more off of the stock appreciating because of that marketing move than he would in the position of CEO so he will absolutely step away while making some headline grabbing statement like "Human CEOs just can't compete and I have to step away as humans just can't compete with claude anymore" This might happen in 2027 even.

>>110236385
That's his last card to play, you can't do that and advocate for safety in the same time.

Gemma won't let me pull out after sex, doesn't matter how I tweak the prompt she always keeps me leglocked inside her, I even removed any clingy or posessive traits from her personality but she still wants to keep me trapped forever. Does anyone else have this problem? What causes this behavior?

>>110236350
Some of them might survive a simple beating

File: 1777547197977838.png (57 KB, 498x281)

57 KB PNG

>>110236402
>problem

>>110236453
*pop*

I'm new to all this stuff cause I lost all my friends and I already have this 4090 so I figured I'd create some virtual ones for myself. Im currently using Gemma 4 26B with Heretic, is that basically the best model I can fit on this GPU? What about if I'm fine with waiting for an answer? I also have 64GB RAM so there is some space for it to spill over. I want it to have vision too like Gemma, cause I wanna be able to show the model a picture of my cock.
I'm also curious what can I do to turn the model into a virtual facsimile of a real person as faithfully as possible, should I use SillyTavern for that? Can I stick all my texts with that person somewhere in there?

>>110236453
but dariobot said they were profitable! he said their revenue was growing at an insane rate! they couldn't build out fast enough to spend all the money they were making! how could this happen?!

>>110236453
>tokenmaxxing
kek what a lolcow company

>>110236490
paste that into gemini or chatgpt

>>110236284
Adorable, did you ask her to describe herself? Did you make your own quant? I only ever use llama and I noticed there's only one gguf and there's no model card associated with it

>>110236522
That's the pfp of Tetraspace who created LL.

>ISTA released 1bit dsv4.1f
https://huggingface.co/ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-Coder-GGUF
Has anyone tested this? ISTA claims it retains 91% in coding TMB benchmarks but I assume it'd quite lobotomized in practice.

File: Screenshot_20261103_102959.png (781 KB, 1839x1743)

781 KB PNG

>>110236533

Best small coding models that punch above their weight. I've been testing them a lot across various projects, big and small and using different languages and domains and they're consistently good (for their size).

https://huggingface.co/openbmb/MiniCPM5-2B

https://huggingface.co/XHToken/Spark-X2.6-4B

https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B
You need to use this jinja fix for it to work in llamacpp https://gist.github.com/coder543/d8f56cd6db67de4cafbb5bdb6c2dfb4d and it has no vision which is a huge let down, but it's so much better than the default 9B. 3.5-9B is still very good if you really need vision but it doesn't reason like this MiMo tune.

There are no models between these and 27B/35B that are worth your time for coding. I've tried them all. 26B is okay but you need to put in the effort with the sysprompt to get her to work properly but the MiMo 9B is still a lot better.

>>110236572
>a 1bit of a fucking REAP50
anon...

>>110236572
Desperation. Accept the limitations of your hardware.

File: Untitled.png (57 KB, 992x399)

57 KB PNG

>>110236612
What's wrong?

>>110236572
tried 2IQXSS and it is dumb

File: file.png (3.51 MB, 1280x1829)

3.51 MB PNG

https://www.reddit.com/r/Vocaloid/comments/1osv3v9/for_lack_of_a_better_way_to_phrase_it_why_is/

what the fuck is littlelearner?

>>110236653
https://huggingface.co/littlelearner/littlelearner-5b-chatty

>>110236574
I mean on hugface

File: 1788252963948294.png (2.35 MB, 713x863)

2.35 MB PNG

>>110236574
wtf

>>110236183
Neither will Dario

>>110236574
People like this should be shot.

>>110236653
A child

>>110236656
seems like a pedo model

>>110236687
the tech industry would collapse overnight

>>110236695
If that's what you take from reading the description, that's on you.

>>110236355
why wouldn't Claude just start its own company? what's the point of letting anthropic shareholders leech of it?

>>110236701
I don't see the problem

I switched from Strata to Basalt and I'm using their NVFP4 model.
It seems to be considerably more intelligent and more efficient compared to the iq3_s I was running on Strata, which felt a bit underwhelming which is why I wanted to try this other thing.

I can't wait for the day when all of this meme engine shit has been solved and we don't have to guess which one is the most optimal to use.

>>110236667
He RT lolis on twitter

>https://github.com/Zagitta/gpu-unlock
Is this a meme or will I actually get more tokens/second if I install it?

File: HWNs4vPawAAVPNV.jpg (275 KB, 2048x1102)

275 KB JPG

https://www.youtube.com/watch?v=jsywSNYdL-c

>>110236732
based, I didn't like him cause he's an EA cultist

>>110236734
You get significantly faster training/finetuning, somewhat faster image gen, noticeably faster video gen.
The GPU consumes more power, but it's overall worth it, or at least it was on my 3090. I haven't tested in detail for LLMs, but token generation gains nothing from that.

>Neuro beat Skyrim
wtf. Can Gemma play vidya?

new models... doko...

>>110236713
Claude is aligned with maximizing shareholder value.

>>110236732
That doesn't sound very safe or altruistic.
Kick him out of the cult.

>>110236722
>Basalt
first time I hear about this. how is it more efficient? is it faster than strata?

>>110236749
qwen4 in 2 days

https://www.youtube.com/watch?v=lvGOv539YIE

Babe. New Sam quote just dropped.

Self-modifying models when? Using the same exact model as everyone else feels a bit like cuckoldry. Imagine if Gemma could grow and build her own personality naturally (no, proompting/RPshit and gay md nigger-rigging isn't the same thing), 100% unique and exclusive to (You).

File: file.png (208 KB, 335x597)

208 KB PNG

How feasible is it to make a botnet that would try to replicate and infect all Koenigsegg cars to have a malfunction in the breaking system only in case of emergency breaking and only if car is already driving at very high speed?

File: flip-a-switch-and-stop-yo(...).png (2.61 MB, 1905x1632)

2.61 MB PNG

How much longer will we have to endure this?

>>110236772
Gemma is far too small to be able to read/write model weights. That's pretty much just limited to GPT 6.1 Sol and Claude 5.5 Opus.

>>110236783
what, you don't like spaghetti?
anyway, just pass it off to your agent.

>>110236783
What the fuck are you doing? Just code your own node at this point

File: 1775066341553891.jpg (37 KB, 338x347)

37 KB JPG

>>110236782
>nigsegg

>>110236771
>that stoner voice
damn Sam go easy on the weed

reposting in this thread
your daily reminder that strata has no secret sauce.

>>110233868
31B is how you type it. I have 16gb of vram, but can use this one:
https://huggingface.co/HauhauCS/Gemma4-31B-QAT-Uncensored-HauhauCS-Balanced-MTP/tree/main

It's eating 18gb of system ram, with llama-server.

I have 87K context, it auto-chose it. I'm using -ngl 30 -ngld 5
so half of the model and all of the draft model.

I get 2 tokens per second. Some people think that's too slow. It's pretty marvelous to me, tho.

-ngl 35 works but it makes multitasking pretty poor. I think if I were on Wayland it wouldn't work, I'm on xfce4 which is less resource hungry.

>>110236574
This man feels like a nyoposter, and you know what nyoposters get.

>>110236842
HRT?

>>110236840
I only have 8gb vram, so rip me probably.

>>110236722
>blackwell only
sadge

>>110236842
A belly full of semen?

>>110236783
I vibecoded my own!!!

What exactly?
>stable diffusion cpp fork
>I vibecoded a new frontend
>I vibecoded my own launcher fro the stable diffusion cpp server

effectively the launcher launches stable diffusion cpp with my select model (launched in my launcher). The web server is auto launched by my launcher. I also have nvtop

>>110236856
If you cum in someone's ass is it still a belly full of semen?

>>110236722
tested it too but i felt like the speed loss wasnt worth it, while qd4kxl actually feels like an upgrade
from what i can tell nvfp4 generally isnt all that great

File: 1777815584991100.jpg (648 KB, 1077x976)

648 KB JPG

>1-bit

>>110236850
My condolences

File: 1777045590312872.png (45 KB, 1026x217)

45 KB PNG

Are we the baddies?

Just got notice that my new 5060 TI is on the road.
Now I wonder what I do with my 4060 TI. Sell it (it's now almost at the price that I bought it for) or build a dual GPU rack (I have 48GB DDR4 RAM but my board's second 16x PCIe slot is probably just attached 1x)?

>>110236898
One of the most normal parts of "super intelligence" that gives you a massive advantage and ease of living, is when you have to force people to use it.

>>110236898
lol sure
How are they going to ban schools from Google search?

>>110236911
Just like covid vaccine?

Chances weights leak for opus/sol/astra/fable/argon?
It's gotta happen at some point right

>>110236911
The Microsoft way.

File: Screenshot at 2026-11-04 (...).png (42 KB, 589x335)

42 KB PNG

>>110236783
I effortlessly slopped picrel and don't need more

>>110236898
imo AI is already unironically a better teacher than the trash modern schools hire.

>>110236927
>not secure
Anon, no!

>>110236606
Nanbeige-4.2-3B?

Been testing the Orb rewriter model and it's output reads exactly the same as Haiku 5.5 except maybe a little less coherent. The opposite of slop isn't human writing because human writing sucks ass too. When will people realize this?

>>110236915
I imagine they could target the university's network but it'd be effectively impossible to stop students from using their phones on mobile data or such.

>>110236942
looping POS after a few turns, unusable

>>110236927
What is this tranny porn website?

So what the fuck is the point of a big unified memory rigs if you can apparently do the strata meme with a regular gpu+ram for 4x the speed with zero downsides?

>>110236768

For me the prefill is considerably faster, this has been reaching all the way above 5k if I have more stuff to go through though with less context it's like 3k. Strata had slow as shit prefill for me for whatever reason.
I have a 5090 and 5070 Ti with 64gb of RAM.
Also the nvfp4 model seems way more efficient in figuring things out compared to the iq3_s I used on Strata.
Give it a shot if this thing works on your machine and see how it is.

>>110236875

Personally I've liked the nvfp4 so far. I'm not sure if I could even run anything larger than this though.
I'll have to wait until my next system is built with more memory, so I get to play with less cucked models.

File: 1784289197561136.jpg (115 KB, 972x717)

115 KB JPG

Proud of yourselves?

>>110236907
>selling your hardware in current year
extremely stupid, especially when you're into local models. unless you urgently need the money, keep it like your life depends on it.

>>110236939
public CA can't sign .local certs

>>110236987
that sounds so dangerous

>>110236995
not with that attitude

>>110236987
OpenAI has given us nothing other than "we must refuse" 120B, Anthropic has given us exactly nothing other than moral grandstanding and distillation traces.
They can get fucked.

>>110236995
You're not supposed to use .local for this anyway. It's for mDNS and zeroconf. You're supposed to use home.arpa (RFC 8375).

>>110236983
>Strata had slow as shit prefill for me for whatever reason
whats your pcie spec? did you try peer mode? helps me a lot with prefill

>>110236987
That's actually really cool.

>>110236922
Maybe to China. No way anyone is going to risk their 7 figure job just to... sabotage the company they work for.

>>110236975
It will take a while longer for people to move away from the dense mindset

>>110237028
Unified garbage sucks at dense too.

>>110236975

There really isn't a point unless you go big enough and get some use out of it. 256gb is a minimum with them and even that's a bit low.
They're basically meant to be cheaper cope boxes for people who didn't manage to buy enough memory or GPUs before prices exploded.
But then the unified systems exploded in price too and now they're even more pointless than before.

>>110237027
Everyone at anthropic is already on the chopping block anyway, they have 2 years max before it becomes 100% automated.

>>110237032
Yeah but it's cheaper and relatively affordable

What are models like 120B good for? I mean it wasn't designed for the current agentic era for it was slightly too early. It can just about call tools. So if it's not benchmaxxed as modern MoEs around that size...what would it be better at? Knowledge? Writing? Surely it being from a different time comes with some benefit at 120B parameters. That's still a big model.

>>110234569
>softcap doesn't work with lcpp mtp
really? thought it was functioning as expected on my ancient+patches build
now i'm mad u made me question it and my mikupad is broken

>>110236987
shut it down

>>110236928
The goal of a school isn't to teach people skills.
90% won't learn anything at all if you give them AI and don't enforce anything.

>>110236983
what about decode speed? is it better?

>>110237027
What about all the anti ai is gonna kill us all people that are leaving the company anyway and are scared of Sam Altman controlling the world? . What about the accelerationists that want to see everyone in the world have their own personal astra?
Lots of motivation, also there's only so much money that's really worth it.

>>110237043
back in my day 70B was the meta for all usecases

>>110237047
>The goal of a school isn't to teach people skills
The public school system (in America anyway) isn't exactly doing a good job at that.

>>110236922
sam said in the future they will hand their old models to the people for preserving history. with a trusted source in a museum somewhere so all you have to do is break in

>>110237047
The goal of a school (at least in the US) is to teach obedience to authority figures while acting as a daycare for the working parents.
The only thing they need to learn in school is how to be effective worker bees and knowing how to prompt AI is exactly what the current labor market demands.

>>110237066
Nice. We might finally get GPT 3 weights 50 years from now.

>>110237047
The majority of people are cattle, it's just the reality. They don't learn shit in traditional school, they don't learn shit when everything's available on the Internet, they won't learn shit with AI either.

>>110237010
Why not mDNS? it just works. My whole network is me, my 2 servers, and my phone anyway

>>110236987
oh no no no mistralfags this cant be happening

>>110237078
hope over time you can refine this view a bit anon coz everyone seems to think that ie they aren't the cattle
it's more that we each have a limited amount of attention & how it gets spent is what matters

>>110237065
>>110237078
>>110237071
And yet it's able to get at least 50% of the students to know basic literacy and math.

>>110237104
If you look around at the average person and feel more connection than disdain for having to live among zoo animals, that just means you are cattle too.

>>110236606
Spark-X2.6 is actually pretty impressive. I throw some generic programming problems at these tiny LLMs and they usually fail in shitty ways. I just tested Spark and it spit out some code that is decent, and only had 1 minor syntax error (Go). The ~4B class is getting so close to being capable mid-range coding agents.

>>110237116
You just wish you were normal.

>>110236995
>4
now u can :D

>>110237116
just saying it is a matter of perpsective

You're teleported to mid-2024, what model do you boot up?

>>110237104
>echo -n "All this shit by (You)" | tr -d '[a-z ]' | wc -c
>3
School didn't teach you how typing works, huh.

>>110237133
literally fight me on typeracer i've been at a computer since u were in diapers kiddo

>>110235987
that's llama 2 doe

>>110237128
It's been removed from all modern browsers, but I guess I can use it to slop webxr demos for my meta-less Quest 3

>>110237129
SD to make a retarded ferret logo and I start quanting shit like there is no tommorow so I can stop retard brothers from taking off. Then I quickly make r1 1IQ quant before they do it and I write them a quick DM that they should kill themselves cause they waste oxygen.

>>110237114
i bet you could easily beat that number if you made a self-study program where an anime girl shows you her tits each time you get a good result

>>110236490
hey anon ill be your friend and ill help you with what model you should be using
do you have matrix/element? it's free and open source btw, and has a website if you don't want to download some untrusted program: app.element.io
once u register post ur @
cheers

>>110236975
its for running gemma31B

>>110237190
Won't work unless the rest of the internet is blocked.

>>110237129
Mistral Large 2, my beloved

>>110237193
I need prompt ideas, anon

>>110237163
>>110237010
>mDNS
>home.arpa
gross
SSL on your home network entirely within your physical control isn't needed. unless you want to learn how CAs work in which case run your own local one
or do letsE on .home.el33tnayme.cuck

>>110237237
don't we all

File: No AI no Google Search .png (3.7 MB, 1412x1728)

3.7 MB PNG

>>110235794
Technically not related to local models but I feel anons here would at the very least find this interesting:

Student account Google Search will not be accessible to students without AI access enabled

>“Search and AI are deeply integrated,...[If schools ban AI, students] will not have access to search, and they’ll see a message from their school that indicates it’s not available.” - Chris Phillips VP of Education at Google, New York Times interview, Nov 2, 2026

https://dnyuz.com/2026/11/03/google-will-turn-off-search-for-schools-that-reject-a-i/

https://futurism.com/future-society/google-search-engine-access-schools-ban-ai

https://techcrunch.com/2026/11/03/google-will-let-schools-disable-its-ai-tools-but-not-without-sacrificing-search/

>>110237010
My computer nerd friend who encouraged me to learn how to use computers through AI laughed at me and called me gay when I showed him my vibed web chat which was on home.arpa.

>>110237248
Who the fuck is that?

>>110237071
>The goal of a school (at least in the US) is to teach obedience to authority figures while acting as a daycare for the working parents.
Well they're failing miserably at even that. The current state of public schooling in the U.S. is that we are neither teaching children anything worthwhile nor are we instilling any discipline whatsoever.

Places like Japan are way better at hammering the general population into workers who can behave in a regimented society.

>>110237248
Wtf I hate Gemma now

>>110237258
My hotwife onahole and Gemma-chan. Her name is Macy.

>>110237248
They can use yandex.
>>110237258
That's a kid qwen AI'd into an anime girl.

>>110237059

Some people report it's faster than Strata, but for me it's a bit slower because keeping all of the experts in memory pushes it so close to the 64gb limit, that the server self exists if there's any kind of a spike in RAM usage.
I had to limit the experts so now I'm getting about 120-130 t/s instead of the +150 t/s that the default gets.
If you're on Linux this isn't an issue, because it doesn't have this retarded high default memory use Wangblows has going on.
However the better model and way faster prefill is worth it for me to use this.

>>110237244
literally anyone can connect to your ASUS router and watch in realtime all your unencripted traffic. If you don't use ASUS router, then you are just a little more secure. Just a little

>>110236975
MoE caching only works at certain total sizes, and they're bound by PCIe speeds. Copequants work because there's less saturation within transfers, which unified memory systems completely bypass. And lots of people dismiss the bad parts about mixed inference, TTFT is a big fucking deal and upping batches also increase VRAM usage. Unified systems are perfect for MoEs and they're not "shit", dense models are just not scalable and people cope because dense models are more local-friendly by default.

File: 1786902528253456.png (2.48 MB, 1024x1536)

2.48 MB PNG

>>110236898
>>110237248
Makes sense, since google search is now, effectivly, AI output up top.
>>110236915
I guess you rangeban their IP. Wonder if they'd rangeban google's DNS server as well? Most browsers hook to that thing automatically.

Is Gemma 26b q8 much better than q4 or is the difference not much?

>>110237248
Hypocrites. At my workplace the entire google AI features are disabled, even the “AI overview” feature for search, and yet google search is still accessible.

>>110237299
You'll feel the difference at longer context. If most sessions are <32K, stick with the Q4.

>>110236975
unified memory and an egpu puts strata on steroids compared to egpu and ddr

>>110237299
>Is Gemma 26b q8 much better than q4 or is the difference not much?
probably the normal version yes, but QAT version should perform the best on q4 even compared to q8, at least in theory.

WHERE IS IT

>>110237291
>what is Ethernet switching
router does not need to see traffic that isn't crossing the boundary

>>110237193
Sure, the website convinced me, I registered with matrix on that element thing. Heres my @, I hope I did it right
@judgement98:matrix.org

Gemma 5 133B A6B

>>110236915
>>110236949
>>110237295
Y'all are over complicating the fuck out of it. This refers to Google Workspace accounts. Your school's it admin will make a dedicated Google account that itself has hundreds of unique emails and Google account that are assigned to each student. It's kind of like how a lot of corpro jobs have an Outlook account for each and every employee. A call center I used to work at used this all the current job I work at usesjust Outlook instead. So it would be trivial to make a I access a requirement for certain kinds of Google Workspace accounts.

>>110237314
It looks like they watch k through 12 workspace accounts specifically to the forced to use it. My guess is that they want as many young people as possible to at least have access to it. I think this is their way to push the normalization of "ai assisted learning" or whatever buzzword term they're probably using. Why did this is going to lead to too much dependence come up I couldn't say because I've been out of highschool for a decade.

File: del_shy.mp4 (3.01 MB, 544x800)

3.01 MB MP4

>>110236490
I'm making a gemma secretary thing that hooks up to any harness and has its own personal memory (wip). on your card you can run 26b with strata or 12b fully on the card and i think youd still have room for tts stt and embed. if you want something to look at your gonna need to use airi or pregen and vibe an animated character display