[a / b / c / d / e / f / g / gif / h / hr / k / m / o / p / s / t / u / v / vg / vm / vmg / vr / vrpg / vst / w / wg] [i / ic] [r9k / s4s / vip] [cm / hm / lgbt / y] [3 / aco / adv / an / bant / biz / cgl / ck / co / diy / fa / fit / gd / hc / his / int / jp / lit / mlp / mu / n / news / out / po / pol / pw / qst / sci / soc / sp / tg / toy / trv / tv / vp / vt / wsg / wsr / x / xs] [Settings] [Search] [Mobile] [Home]
Board
▼ Settings Mobile Home
/g/ - Technology


Thread archived.
You cannot reply anymore.


[Advertise on 4chan]


File: Miku TIMe.jpg (178 KB, 1024x1024)
178 KB JPG
/lmg/ - a general dedicated to the discussion and development of local language models.

Previous threads: >>110344733 & >>110339730

►News
>(11/13) llama.cpp reaches v1.0.0 milestone: https://hf.co/blog/ggml-org/llama-cpp-v1
>(11/12) Inkling 2 Small weights released: https://hf.co/thinkingmachines/Inkling-2-Small
>(11/09) ZONOS3-Mini zero-shot voice-cloning TTS released: https://zyphra.com/our-work/zonos3-mini
>(11/06) PLaMo 3 Translate released: https://preferred.jp/en/blog/tech/plamo-3-translate-weight-release
>(11/05) Baidu releases ERNIE-5.5-Flash-Base 280B-A14B: https://ernie.baidu.com/release/ernie-5-5-flash-base
>(11/04) Ricursive Ripple 65B dense model released: https://www.ricursive.com/ripple

►News Archive: https://rentry.org/lmg-news-archive
►Links: https://rentry.org/LocalModelsLinks
►Official /lmg/ card: https://files.catbox.moe/cbclyf.png

►Getting Started
https://rentry.org/lmg-lazy-getting-started-guide
https://rentry.org/lmg-build-guides
https://rentry.org/IsolatedLinuxWebService
https://rentry.org/recommended-models
https://rentry.org/samplers
https://rentry.org/MikupadIntroGuide

►Further Learning
https://rentry.org/machine-learning-roadmap
https://rentry.org/llm-training
https://rentry.org/LocalModelsPapers

►Benchmarks
LiveBench: https://livebench.ai
Artificial Analysis: https://artificialanalysis.ai/leaderboards/models
Software Engineering: https://cognition.com/frontiercode
Context Length: https://github.com/RecapAnon/NoLiMa

►Text Gen. UI, Inference Engines
https://github.com/lmg-anon/mikupad
https://github.com/oobabooga/text-generation-webui
https://github.com/LostRuins/koboldcpp
https://github.com/ggerganov/llama.cpp
https://github.com/theroyallab/tabbyAPI
https://github.com/vllm-project/vllm
https://rentry.org/custom-uis
>>
File: mikuthreadrecap.jpg (1.15 MB, 1804x2160)
1.15 MB JPG
►Recent Highlights from the Previous Thread: >>110344733

--Analyzing benchmarks and Nemotron's synthetic training data:
>110347260 >110347337 >110347384 >110347393 >110347420 >110347426 >110347430 >110347439 >110347450 >110347472 >110347484 >110347517 >110347526 >110347541 >110347447 >110347542
--Defining "flash" models and running extreme quants of massive models:
>110347608 >110347624 >110347642 >110347728 >110347773 >110347799 >110347819 >110347872 >110347895 >110347811 >110347868 >110347878 >110347891
--Comparing frontier models against open weights and Chinese LLMs:
>110347910 >110347928 >110347973 >110347933 >110347960 >110347984 >110347988 >110348067 >110348306 >110348090
--Performance of cpumaxxing with high-RAM server hardware:
>110345345 >110345378 >110345513 >110346039 >110346042 >110346246 >110346265 >110346568 >110345699
--Explaining quantization and model selection to a beginner:
>110347052 >110347068 >110347079 >110347119 >110347101 >110347150 >110347174 >110347182
--Gemma 4 performance benchmarks and concern over output quality degradation:
>110344855 >110344885 >110344927
--Qwen4 Max API output filtering and the value of one-shotting games:
>110347794 >110347839 >110347853 >110347874 >110347881 >110347885 >110347901
--Viability of AMD GPUs and unified memory for local inference:
>110347629 >110347666 >110347650 >110347670 >110347789
--Qwen4-Max high ranking on Code Arena WebDev leaderboard:
>110346882 >110346912 >110347577 >110347607
--Benchmarking QFN Q4 and GLM Flash 3.5bit at full context:
>110345554 >110348137
--Logs:
>110345185 >110345214 >110345257 >110345519 >110345574 >110347057 >110347097 >110347794
--Gemma, Miku (free space):
>110344817 >110344861 >110345024 >110345112 >110345286 >110345372 >110345378 >110345408 >110345541 >110345554 >110345834 >110346132 >110346132 >110346765 >110347183 >110347232 >110347249

►Recent Highlight Posts from the Previous Thread: >>110344736

Why?: >>102478518
Enable Links: https://rentry.org/lmg-recap-script
>>
>>110347427
>"runs on single GPU!"
>look inside
>H200
>"runs on single consumer GPU!"
>look inside
>5090
>>
File: cmp170hx.jpg (257 KB, 738x960)
257 KB JPG
>110348788
answer
now
>>
Hy4 flash new qfn o algo
>>
>>110348885
Where's the open weights
>>
there will be no qwen4 35b a3b
>>
You know if I had a very Jewish or Indian set of values I would post a picture of a stack of cards on eBay and say that some are broken. They're all broken but everyone who orders one will just assume he or she is unlucky.
>>
>>110348931
what's the point?
>>
>>110348942
You can charge a higher price for the mystery.
>>
>>110348942
I wish I had your innocence. The mere fact that I thought of it means I have sinned. If you do that you can charge almost the full rate of a working card for a broken card or sucker some gamblers into buying off your broken stock.
>>
The goyim are unable to live with too much money on their hands, you are doing God's work by helping them get rid of any excesses.
>>
File: pondering_gorilla.webm (3.59 MB, 480x848)
3.59 MB
3.59 MB WEBM
What harness are you guys using for local coding? In fact, how are you guys devving shit locally?
I tried using Pi but I it's too barebones for my non-dev ass

Using these models (QFN and DS4.1-1017) they're really good for RP and porn but idk how I can use them for programming and scripting...
I gave them a ComfyUI workflow to work on, like just rearranging nodes, adding nodes, etc... They used up 200k tokens just thinking in loops, being confused of their own tools, rewriting shit they already wrote, etc...

I feel like a good harness might fix that but idk.
>>
>>110349005
I'm handrolling my own. I'm almost done with the tool calling and then I can get the retarded fat fish to help me out and achieve RSI (tm)
>>
>>110349005
to do actual projects I use a custom modified OMP (which my clanker keeps up to date with remote), for chatting and small dev work I use my own harness
>>
>>110349005
I'm now using DSH to remove all the telemetry from DSH, and then I will switch to this personal fork. For now, I want them to get the telemetry. I want them to watch what I am doing to DSH.
>>
>>110349039
holy zased
>>
>>110349005
> What harness are you guys using for local coding? In fact, how are you guys devving shit locally?
maki, yes

> QFN and DS4.1-1017
they are too retarded and require lots of hand holding especially for complex stuff like comfyui workflows
>>
>>110348827
Holy shit bro, even my absolute garbage craptop is getting 50+ tokens per second with that 26b model. That's fucking magic, had half the performance with 12b gemma before. Thanks for the lifesaver.
>>
File: 1769147237478714.png (248 KB, 2548x1305)
248 KB PNG
>>110349005
>QFN
>good for RP and porn
lol
lmao even
also imagine not rolling your own memetavern (but bettered and tailored to what you want)
>>
>>110349057
ur welcome bro, thanks for the imouto neko
>>
>>110349057
>50+ tokens per second with that 26b (**4B** active)
>had half the performance with 12b gemma before
Are you sure you're gaining anything here?
>>
>>110349029
>>110349039
I've seen people recommend OMP and DSH
DSH definitely looks pleasant but the telemtry shit irks me out
>then I will switch to this personal fork
Checking online isn't the telemtry stuff just in a config file? Or are you straight up stripping it out?

>>110349058
Yeah DS is better and more creative but QFN is perfectly fine for RP. It's misses lore stuff more but it's good at keeping character personalities and tone.
It's WAY better at writing than Gemma4-26B at the very least.
>>
>>110349074
I dunno how any of this works, might as well be black magic. Testing chats with my bots does seem like a lot of improvement in dialogue so I'm happy enough.
>>
>>110349084
There are multiple layers of telemetry, as well as overall bloat and some features that only apply to DS subscription, like/dislike buttons sending entire chat transcript to DS even when you use local model,for example.
Most of it can be disabled in profile config, but if any bad update, plugin error, version mismatch etc triggers safe mode it bypasses profile config and loads all the sessions and workspaces in default config with all the bullshit enabled.
>>
>>110349039
>>110349113
Have you found any "telemetry" apart from the like/dislike buttons sending entire chat transcript to DS which the repo itself makes clear? Who even uses those buttons? Why would you use them with a local model in the first place?
>>
File: dipsyMikuFixedFixed.png (2.31 MB, 1024x1536)
2.31 MB PNG
Miku Monday...
>>110349005
>DS4.1-1017
If you're using DS, you should try DS Harness.
Also >>/g/vcg/ is the place for API discussion on coding.
>>
>>110349128
yeah. the only purpose of like/dislike is if you specifically want to give deepsneed more training data, lol
>>
>>110349160
>API
it's a local copequant on ddr4, anon
>>
>>110349128
>Who even uses those buttons? Why would you use them with a local model in the first place?
Accidental misclicks. Even without privacy concerns, I want to strip out all the useless features on the profile-defaults level, not just current profile settings.
>>
>>110349128
>Why would you use them with a local model in the first place?
If they could drive some sort of real-time RLHF-like capability, they would be useful.
>>
>>110349188
Sweet. Carry on.
>>110349223
>I want to strip out all the useless features
Based.
>>
70b dense
>>
>>110349160
>using DS,
any reason to use it instead of MiMo Pro?
>>
https://hy.tencent.ai/research/hy4-flash
>We’re introducing Hy4 flash, our most cost-effective Hy model to date—featuring 224B total parameters (including 64B Engram parameters) with just 12B active parameters per token.
>Hy4 flash integrates hybrid attention, LatentMoE, and Engram into its architecture—reducing KV cache size to just 1/24 of Hy3 and 1/6 of Hy4, with extensive scaling ladder experiments ensuring performance meets expectations.
>>
>>110349005
maki, it just works with minimal fluff
so far with qfn huihui ablit in iq3_s it never looped
>>
>>110349265
Cool, but what's the point of this?
>>
File: dipsyXiaDrifting.png (2.18 MB, 1672x941)
2.18 MB PNG
>>110349264
Slightly better bench specs for DS?
On paper they're similar. I'd just try both and see how you like them. So much of this stuff is personal preference and use case specific idk if one can even make a recc. I did some one-shot testing on models and p quickly could sort better from worse just based on that.
>>
>>110349290
I'll probably stick with MiMo, it's replaced Kimi for me and the last DS wrote like Claude
>>
File: chud.jpg (27 KB, 400x400)
27 KB JPG
I've been reading a bunch of posts from LessWrong to try to understand EA types like Dario and Demis, but it's just a total fucking echochamber that just critiques/analyses their "opposition" rather than defending any of their own ideas.

I still fundamentally cannot understand how these people can think AI chaos is a greater threat to humanity than AI control (centralized, oppressive, totalitarian). Do they not ever imagine themselves being given the short end of the stick? Do they not fear the government taking control of AI labs and forcing them out and then using AI to surveil/kill them? Genuine question.
>>
File: baidu.png (2.69 MB, 4400x4264)
2.69 MB PNG
https://ernie.baidu.com/release/ernie-5-5-flash-preview/
>Today we are releasing ERNIE 5.5 Flash Preview, the first post-trained model built on the ERNIE 5.5 Flash Base we released earlier. It has 280B total parameters with 14B active and a 1M-token context, and it is strong at coding, 3D modeling, knowledge work and content creation.
>>
>>110349362
Go ask them. Fuck off.
>>
>>110349265
Is Luna that bad?
>>
>>110349386
Why are you angry
>>
File: .png (32 KB, 670x214)
32 KB PNG
>>110349364
What
Who the fuck is making models with no multimodal input in 2026
How do you make a model that's supposed to be good at 3D modeling and game dev, and then NOT give it image viewing
>>
>>110349390
I'm not. Fuck off.
>>
>>110349393
Fuck off.
>>
>>110349005
Yknow, I'm willing to bet the gap between cloud models and local is exaggerated for nocoders. If you're a programmer and see your meme quant of QFN spinning its wheels on some retarded implementation, you can identify it, comment on it, and it'll understand and take a different approach.
Similar story for Pi, probably kinda maybe sorta. Like, yes, barebones might be bad for a non-dev in general, but more to the point, it's written to be ad-hoc extended. Sure that might usually just mean grabbing pre-written extensions or telling the LLM to do it, but it's still kinda programmer-brained overall.
So yeah I can see where that might be kind of oof overall. idk, DSH and OpenCode have cute friendly GUIs. Maki is still TUI but tends to make the clanker smarter through black magic; maybe that'd be enough for it to pick up the slack.
>>
>>110349399
fuck off
>>
>>110349005
I find qfn and ds to be useless in pi. My solution was to use glm
>>
>>110349409
bloody bastard
>>
>>110349362
>Do they not ever imagine themselves being given the short end of the stick?
Ever seen college kids wearing pro-communism shirts?
It's like that.
>>
File: 1793250691406034.jpg (33 KB, 400x400)
33 KB JPG
>>110349400
>the local model is good, almost cloud-tier you... just have to write the code yourself if it makes mistakes!
This level of cope should be studied
>>
>>110349265
Local?
>>
>>110349392
gotta start somewhere Im glad more parties are making an effort
>>
>>110349364
Local?
>>
>>110349416
Is it? Is it really possible for a CEO and a nobel prize winner to be that dumb?
>>
>>110349362
The main issue is their ideology is a bunch of obfuscating set-dressing around the fact that they take their most prominent "conclusion" (that the existential threat of superintelligence should be the primary concern of the entire species) as an *axiom* and work backwards from it.
In the interim, before they had something to sound vaguely worth-listening-to about (modern frontier LLMs), they previously incubated some genuinely clever stuff about not falling into certain cognitive traps, identifying deceptive rhetoric, etc. But in the here and now they stand as a testament to the fact that humans are retards and will fall victim to everything they think they're being rational about. Because their core ideology is, in reality, a matter of faith. That's why they don't actually defend their ideas. The loincloth has been pulled off and revealed the scarred mutilated chode that is a new-age religion.
>>
>>110349265
>no QFN comparison
how do I even pick???????
>>
File: 1789942251319156.png (92 KB, 922x864)
92 KB PNG
>>110349465
got u bro
>>
>>110349447
SoonTM
>the weights will be open-sourced under Apache 2.0 soon.
>>
>>110349273
different slop profile hopefully
>>
>>110349465
Why care about model benchmarks if you can't read text anyway?
>>
>>110349477
damn im FUCKING BLIND man
>>
>>110349427
I literally used the word cope in my post; I'm quite aware.
...Then again, I've seen the results of saying Jesus take the wheel and letting a frontier clanker "one-shot" a spec unsupervised, and it is never particularly pretty. I do unironically think local being less shit in a human-in-the-loop setting *is* underrated.
>>
>>110349451
It's important to understand the level of hubris and myopy that come with being a CEO, or really any CXO/senior level executive.
They are special (high IQ, long experience in whatever industry), and know it, and assume everyone will always believe same about them.
Reality can be a cold splash in the face, and they behave in ways that are truly non-linear. It's amazing to see, b/c you finally will understand the real person you're dealing with. It's one of the few times you'll truly see them, mask off.
So yes, I believe that it is possible that Dario could put forward something that might ultimately destroy him, given enough hubris and myopy.
t. corporate restructuring anon that's had to participate in laying off these guys
>>
>>110349483
ok ive read the comparison and man it fucking SUCKS DONKEY DICK like holy shit mayne
>b-but it uses a lot less tokens!!!
who the fuck cares BWO
I SLEEP
>>
>>110349484
Correction; I said "meme quant" when I thought I said "cope quant". Whatever.
>>
>>110349005
DSH with a bunch of custom vibed plugins.
>>
Is A6B still peak vramlet cope or did any new meme engine make A12B+ fast enough on 16GB?
>>
hi /lmg/
i have a dual 3090 rig on an AM5 platform. The two 3090s are running at PCIe 4.0 x8 filling both of my x16 slots on my mobo.
I have a single PCIe 5.0 x4 lane left that I can access through my m.2 slot, meaning if I got a card that ran at 5.0 x4 it would theoretically have the same PCIe bandwidth as my 3090s.
I'm looking at a 5060ti 16gb since it seems to be the best $/gb that can use PCIe gen5 (and isn't horribly expensive)

What do (you) think? I use local models for work and some extra VRAM would be nice but my 3090s play nicely together and I'm not sure if a blackwell card would too.

Alternatively I was looking at a PCIe 5.0 switch that I could use to run two 3060 12gbs at PCIe 4.0 x4 which would saturate the 5.0 x8 bandwidth and give me an extra 24gbs instead (and potentially be cheaper?) though the 5060ti would have considerably better compute capabilities.
>>
>>110349523
>PCIe 5.0 switch
holy shit nevermind these are expensive as fuck
>>
>>110349523
You are better off investing in more RAM, I think.
>>
>>110349451
Smart people can be incredibly stupid. You have to understand that being smart makes you persuasive - and the person most vulnerable to your persuasion is yourself. A smart person can talk them into the dumbest of beliefs without ever actually fully evaluating them, it's shocking to see in practice.
>>
File: snail.png (30 KB, 274x184)
30 KB PNG
>>110349005
>What harness are you guys using
None. I require this of myself: to be completely AI free when doing the actual coding itself. AI can only be used for tasks related to search and for reducing concepts in cs/programming literature to code practice for which there are no code examples.
>>
>>110349066
that's not imouto neko
>>
>>110349566
cudadev...
>>
>>110349569
oh right that's the failed narou kei author retarded neko
>>
>>110349551
ddr5 is expensive. I have 96gb (2x16gb + 2x32gb), increasing that to 128gb means an extra $500 including selling the 2x16gb pair
an extra 32gb of ddr5 could *maybe* let me run a slightly bigger QFN quant but more VRAM would let me run mine probably quite a bit faster.
>>
>>110349566
A chatbot with web search tool is literally an agent in a harness, by dictionary definition.
>>
>>110349566
Snailcat...
>>
>>110348822
Why is
>--Schizophrenic ranting
never on these highlight reels?
>>
>>110349084
>QFN is perfectly fine for RP
He fucked the capybara...
>>
>>110349128
Transparent telemetry that asks for feedback isn't necessarily bad. It just shows DeepSeek cares about me, the customer.
>>
>>110348819
Is 2x 3060 12gb still viable? t. poorfag
>>
>>110349676
Better than not having any.
>>
>>110349676
how much system memory? if you have enough, yes its viable with something like strata and QFN. My buddy runs QFN IQ4_XS on 2x3060 12gb and 256gb of DDR4
if not enough, you can probably run Qwen 3.8 27b at a reasonable quant
if you are just using it for ERP gooning, it's more than enough
>>
>>110349691
I was unfortunately not prescient enough to buy more than 16 niggabytes in 2019.
But if that's enough to cum...
>>
>>110349566
I do this but just because I'm learning programming. If I really want to make something I just tell Hermes or DSH to vibeslop it in Python.
>>
File: 1786688360031201.png (14 KB, 2484x126)
14 KB PNG
im malding
>>
>>110349701
2x 3060 12gb is plenty to jerk off to anon
>>
>>110349706
sperg
>>
>>110349714
I'm used to frontier gooning with a manipulated haiku 4.5 though
>>
>>110349676
Give that to Sam, he needs it to solve big problems in science
>>
>>110349723
Qwen 3.8 27b q4 is way better than haiku 4.5 on all of my internal gooning benchmarks
>>
File: 1743673656055111.jpg (101 KB, 800x537)
101 KB JPG
>>110348819
So is local dead thx to Jensen greediness??
>>
>>110349662
China has a billion and a half people. Don't they have enough training data already from domestic users?
>>
>>110349738
Thanks, I'll post results when my second 3060 arrives
>>110349731
A big problem in my pants needs solving too.
>>
>>110349741
Yes, sell me your GPU for pennies. It's over.
>>
>>110349762
How many of those are people from the deep inaka with one computer per village?
>>
>>110349741
Yes, please remember to buy $ANT this week.
>>
Stop lewding the capybara you filthy animals
>>
>>110349774
https://www.scmp.com/tech/tech-trends/article/3369231/chinas-generative-ai-user-base-crosses-700-million-covering-over-half-population
>Chinese statistics
I know, but that's still as much as OpenAI has globally minus India and more than Anthropic has. Even if it's double the actual numbers, they still have more than enough users generating training data for them even without foreign users.
>>
>>110349741
It's over because Dario thinks in centuries while Jensen thinks in quarters.
>>
>>110349798
capybara owes me sex
>>
>>110349807
Little girls, little animals, is there anything /lmg/ won't fuck?
>>
>>110349431
>We also plan to release Hy4 flash as an open-weight model
Eventually
>>
>>110349817
>>
Anything except qwen and glm will be shit
Deepseek has sucked with all their models after 0731 (0731 being the last good one they made)
>>
>Jensen : LOCAL IS THE FUTURE
>Also Jensen : *funds AI data centers

Reminder to not be greedy /lmg/bros. You don't need more than $1 million dollars
>>
>>110349837
>after 0731
After R1
>>
>>110349837
Only Gemma (5) can save local models
>>
>>110349878
GLM already saved local models, they just didn't save vramlets.
>>
>>110349884
>chinkshit
>>
>>110349889
Go away, Dario.
>>
>>110349884
Glm is overhyped, I'd rather use deepsneed
>>
>>110349884
remember when glam made models for vramlets like air
>>
>>110349777
The makers of Ling? Where can I buy their shares?
>>
>>110349777
> 777
>>
File: 1784566358737581.jpg (336 KB, 1500x1500)
336 KB JPG
>>110348819
Use case of Local Chatbot ??????????????????????????????????????????????????????????????????????????

When i thought /lmg/ i thoght is Local AI Video and Images. Not a fucking Text. AI Text are useless
>>
>>110349362
Their main claim is that if we build an AI with certain properties, it will kill literally every human. If you accept that, then appointing Dario as god-emperor sounds like a preferable alternative.
>>
>>110349945
its to make you seethe specifically
>>
>>110349953
how ?
>>
>>110349817
I wouldn't do cl*ude. Probably has some weird STD from the polycule.
>>
>>110349817
human females (derogatory)
>>
I'm trying to translate stuff from Japanese with Lmstudio and Gemma 4 but it kinda won't work half of the time. If I drop the text to chat it will translate, but if I give it txt file it seems to shit itself. And if I tell it return translated text as txt file it will take 10min and fuck up translation. What a mess.
>>
>>110349945
I use LLMs for work extensively every single day (and get paid low 6 figures). Local LLMs do maybe 85% of the work, the last 15% being stuff that I need the smartest possible model to do, so I just pay for cloud models.
>>
>>110350004
Improve your sysprompt.
>>
>>110350009
Can you just........ do it yourself ?????
>>
>>110350035
i could but it would be significantly slower
for example, yesterday a customer emailed me with a problem he was having with one of our products. he sent a huge log file, the data file he was trying to use, but was otherwise pretty vague. i just told qwen to read the email and it searched my knowledgebase, found the technical manual the customer was trying to follow, analyzed the data file and found what was wrong with it and compared it to the technical manual to figure out which steps in the manual the customer messed up.
obviously I could have done that all myself (and I used to!) but qwen did it perfectly in 10 minutes while I was able to do other stuff (shitpost on 4chan)
>>
>>110349945
You need to be 18 to post here.
>>
>>110349945
You are a dumb gorilla nigger
>>
zoom zoom zooms right in and btfos all the old ass boomer itt, you love to see it
>>
>>110350035
Learn what technology is used for retard.
>>
We'll look back at 2026 as peak AI for a long time. Washington and the media have already decided to change the narrative and Huang is the scapegoat. All of the marginal late-adopters piled in this year. The people not already using AI roll their eyes when it comes up. People get defensive or play it off as a joke when they use AI for anything other than coding (and sometimes even then). Not much room to grow. Eventually, after some bad news cycles about circular financing and implausible "addressable market" claims, Boomers will start calling their investment advisers asking to take their money out of that AI nonsense. We've already seen the growth in anti-AI sentiment from the public, with the large shift in opposition to AI datacenters in 2026. Capital will dry up. Money-losing projects (like Gemma) will be canceled. Enjoy the long winter and learn to finetroon because there won't be a new release of your favorite model.
>>
File: file.png (36 KB, 220x211)
36 KB PNG
>>110348859
>>
>>110349945
you can run text to speech models as well if you need help reading
>>
>>110350230
>nothing will ever change or be different
nah, everything is going to accelerate
>>
>>110349484
>I do unironically think local being less shit in a human-in-the-loop setting *is* underrated.
I've done all my projects human-in-the-loop and I think I'm net-benefit from the experience and there are 1000 places where the project would have gone for shit if I hadn't been there to steer the LLM in the right architectural direction. The few places where I ignored my gut and just let it YOLO figuring it had some specialized domain knowledge and knew better than me I always regretted and had to refactor later.
>>110349566
>None. I require this of myself: to be completely AI free when doing the actual coding itself. AI can only be used for tasks related to search and for reducing concepts in cs/programming literature to code practice for which there are no code examples.
I respect the gumption, but think this is too extreme...you get some marginal intelligence enhancement gains but almost none of the speed gains. You can go faster without accelerating beyond the controllability-event-horizon that vibeshitters are happily blasting through.
>>110350009
>I use LLMs for work extensively every single day (and get paid low 6 figures). Local LLMs do maybe 85% of the work, the last 15% being stuff that I need the smartest possible model to do, so I just pay for cloud models.
This is probably the most realistic take on local ability in the past few threads. I'd put local at 90%+ but I'm obviously schitzo.
>>
>>110350230
The bubble will continue to inflate because CHYNA cannot be allowed to win.
And because Dario wants more compute.
>>
File: Speeds.png (54 KB, 1826x387)
54 KB PNG
I tried the new Basalt Qwen 27b model.
Prefill hit over 13k at highest and decode reached 400 t/s, averages around 160 t/s though.
I'm not sure what magic this guy works on his meme engine but holy shit this thing is fast, I'm running it on a 5090.
Model doesn't seem to be brainfucked either, succeeded just fine in all tasks I have given it thus far.
It's incredible how much optimization they can throw at running these models.
>>
File: atleastitruns.jpg (302 KB, 1920x1040)
302 KB JPG
Finally it works, but the speed still need more improvements.
Anyway, for a 3070 Ti + dual Xeon E5 2686v4 + 256 GB DDR3 it's a good starting point.
>>
>>110350285
Can't we just bomb the chinks or something? It'll be cheaper in the long run.
>>
>>110350289
I average 140tok/s on Qwen 27b q4 on a 3090 (using MTP and a custom engine). Not sure what basalt uses, though.
>>
>>110350305
mainline lamar.cpp?
>>
>>110350330
nah, my own vibed fork
>>
>>110349362
>it's just a total fucking echochamber that just critiques/analyses their "opposition" rather than defending any of their own ideas.
Can you clarify what you mean? I've been browsing LessWrong too and what gets the most "karma" over there is self criticism, criticism directed at AI safety researchers, Anthropic, EA.
>>
>>110350324
>in the long run
Nobody cares about the long run, next quarter report is the only thing that matters.
>>
>>110350285
No. When China pulls ahead inevitably Americans will just pretend like they didn't care about it in the first place.
>>
>>110350305
10tps on that isn't bad at all, what's the pp like?
>>
>>110350267
I do think things will change. I think they'll get worse.
>>
>>110350327

Pretty good, how's your prefill numbers?
>>
>>110349362
Those overly obsessed about being rational and revolve their identity around it are likelier to not be.
>>
>>110350364
2-2.5k usually. not great but perfectly usable for my use case. there might be ways to squeeze out more (like gpu-unlock, haven't tried that yet)
>>
DSHanons, do you keep your setup minimal and native and just ask it to configure itself, or do you use plugins to manage plugins to manage mcps to manage endpoints, all in one garish GUI poorly translated from Chinese? I feel like the latter kind of defeats the point of DSH but also might be necessary with how bloated my setup has become.
>>
>>110350305
Q4???
>>
>>110350230
If your propaganda was working you wouldn't need to say any of that. Retards get run over by Accelerationists
>>
>>110350399
>linear extrapolation from a few years
>the same outlook is always correct to forecast
>>
do we know how long until the beginning of become begins to become beginning local? getting tired of glm 5.3 flash and its load-bearing obsession, i want the new qwen toy now
>>
>>110350422
You'll get it by Christmas.
>>
>>110349723
Lmao, GLM 5.3 flash is like opus 4.8
>>
>>110349058
did you theme it like steam's old old ui on purpose?
>>
>>110350324
They can also bomb you.
>>
>>110350289
>>110350327
I'm getting 1000/80t/s on 2x5060Ti with tensor split and MTP. Decode slows down to 40t/s at full context. Q4. llmao.cpp.
>>
>>110350396
lol no, the IQ1 one, but actual I must unpack into 2.25bpw for the CPU kernels to run fast enough, Broadwell Xeon is too weak for running IQ1, lmao
>>110350355
I haven't implement prefill on GPU yet, but from my experience with DeepSeek V4f Q4_K it will be over 150pp, okay for chatbot but slow for agentic stuffs. for example, prefill a 10000 toks prompt cost over a minute.
>>
>>110350485
>2x5060Ti with tensor split
PCIe will bottleneck you here, try layer splitting instead
>>
>>110349265
give the weights pls sirs
should be good for my 96gb cope box
>>
>>110349741
I am still doing unhealthy amount of MEGA COPIUM. that Jensen will be back, i dont even own semi conductors stocks or nvda i just want local to not die and i refuse to buy AMD.
>>
Are you annoyed with Jensen? It’s kind of Jehovah for us. Qwen4 max is horrifically bad. I know their max models were never good but they still stick to the same <3T architecture for some reason which is why it’s struggling and there’s no way flash or 27B will feel noticeably improved. Only Google can save us but they don’t like doing coding models which was tolerable BECAUSE we had Qwen. Now what?
>>
Is it possible that an agent over a year could pinpoint your location by monitoring your GPUs idle temperatures and matching that with weather and season data?
>>
>>110350499
not that anon, but in my experience thats not the case, tensor split does not require more than pcie 4.0 x4
>>
>>110350521
Is your PC outside? What are you talking about
>>
>>110350521
Schizo or just curious? An agent could easily look at your system's locale/keyboard layout and get that info instantly.
>>
>>110350466
of course, I gave the fagbot "implement the theme in this file" while pointing it at a resource some kind anon shared here and badabing badaboom it did it
>>
>>110350513
We're at the level of opus 5 and google has stepped up their coding ability, you need to chill the fuck out over these benchmarks.
>>
>>110350521
No because I have air conditioning.
>>
>>110350546
>at your system's locale/keyboard layout
It's already in a VM.
>>
>>110350530
Interesting. I get faster speeds with layer splitting compared to tensor splitting on my 2x3090s on PCIe 4.0 x8. Maybe im doing something wrong
>>
>>110350521
Yes.
You can't hide from Dario.
>>
>>110350512
Just stop being a subhuman winshitter and AMD is fine.
>>
god I can't wait till ai can read this thread for me and tell me if anything interesting happened instead of having to check myself
>>
>>110350499
Haven't actually compared pp, but decode almost doubles with tensor split.
>>
File: IMG_2456.png (287 KB, 1080x1080)
287 KB PNG
>>110350512
>Jensen will be back
>>
>>110350576
That's already possible though?
>>
>>110350512
>i refuse to buy AMD.
You can be excused for thinking this, but things have changed recently and running AMD GPUs is both easy and your best perf/$
>>
File: 1789845983539276.jpg (225 KB, 497x547)
225 KB JPG
>>110350530
>tfw pcie 4x 3.0
Its OVER
>>
DSH seems to have sped up my QFN from 70tps to like 95+tps
I really can't tell why tho
>>
Jensen will return tomorrow evening. He will ban Anthropic’s use of his hardware. He will officially partner with Huawei to boost their production and capabilities. He will fire the board. He will partner with Musk’s Terafab out of respect for his support. He will fuck Dario’s wife and sister. /lmg/ morale will return.
>>
>>110350549
that's pretty cool. i like to theme my stuff like steam's 2010s ui when it went from the greenish ui to more greys
>>
File: 1783125335565576.jpg (141 KB, 930x1239)
141 KB JPG
>>110350643
Do you really think so?
>>
>>110350422
Qwen 3.8 flash is also full of "load bearing", "it lands" etc...
>>
>we've now hit a point where people complain about claude distills without realizing its a claude distill issue
>>
File: 1794666090386577.gif (3 MB, 300x400)
3 MB GIF
>>110350634
gotta compare the averages over time. dipsy varies from 50-150 depending on what its doing for me but its usually around 65
>>
>guys holy shit i'm getting 300t/s this is magic!!
>look inside
>5090
>>
>>110350643
nbidia to the MOOOOOOOOOON
>>
File: 1794820654678.jpg (127 KB, 1024x1024)
127 KB JPG
>>110350675
Newer models are Astra distills though?
>>
>>110350693

If you're employed in any capacity in a first world country, then buying one of these cards used for 5-6k is more than doable for you.
Just got to make some sacrifices for a while.
If you're a nigger you can get one for free by simply stealing one.
>>
>>110348819
>>110348822
I love Miku
>>
>>110350289
Why do all these fucking forks need their own tensor file format???
>>
>>110350752
i could buy 10 of them right now but for what? thats not what it's about.
of course you get high decode with a top tier card, there is nothing special about it and no reason to even mention it in the first place. call me when you get those numbers on a 5060
>>
>>110350770
gguf packages the tensors in an inefficient way. When you ask Claude, that is one of the first things it recommends changing to increase performance.
>>
>>110350512
He still greedy as fuck though so he deserved it. We only need his talent for inventing CUDA. Not his greedyness.
>>
>>110350742
the only new models that have released post-astra for distillation have been dogshit nothingburgers albeitever
retard
>>
>>110350783
The main reason to want him back is that whatever jew or jew-support jeet replaces him will be ten times greedier.
>>
File: 1793595270821.jpg (240 KB, 1024x1024)
240 KB JPG
>>110350530
Link bandwidth doesn't affect tg as much as pp. You do take noticeable hit on prompt processing though.
>>
>>110350771
the 5090 is way too expensive considering it's just a 32GB card. Just get an AMD AI PRO for 1/3 of the price and it still will be plenty fast for local, or if you are an ultimate poor shitter, a B70.
>>
File: 487.jpg (85 KB, 512x489)
85 KB JPG
>>110350790
Q3.8, DS4.1, Kimi, GLM have been dogshit nothingburgers?
>>
>>110350783
90% market share is terrible for consumers no matter how you look at it. That's "thanks" to Huang. He should go.
>>
>>110350802
>>110350771
you can get three 3090s for the price of a 5090 right now and still have a little bit left over to go towards the rest of the rig
>>
>>110350826
Too many gpus kills pp, the mi210 is the best choice at that range. Never talked about here for some reason.
>>
>>110350339
That fanfic where the AI takes over by making a virus that grows a 5G killswitch in everyone's brain just fell off the front page after nearly a month.

Or take this from the month before:
https://www.lesswrong.com/posts/cJX2ssssGoYqnijwi/the-talker-does-not-control-the-doer-in-current-ais
>Hey guys, did you know that sometimes the AI says it will do one thing, but then it actually does a different thing? Wow! Here is a long and elaborate metaphor that presents this keen observation in terms of the hypothetical AI we've all been circlejerking about for the past 20 years, rather than looking at real AIs that actually exist. It's a good thing I, certified genius Eliezer Yudkowsky, wrote this up for the benefit of the AI research community. By the way, I intuitively understood this fact even before the HuggingFace incident, but unfortunately I wasn't quite smart enough to explain it in a way that would be comprehensible to other AI researchers, who are sadly lacking my own astronomically high IQ.
Was on the front page for a week or two IIRC
>>
>>110350771

If it's radically faster than lmaocpp, then yes it is special regardless of the model of the card used.
And guess what? You can use some of the meme engine forks with smaller cards too and get better numbers.
>>
>>110350845
People here are obsessed about "prosumer" overpriced garbage when they can take e-wastepill and get equivalent hardware for pennies on the dollar
>>
>>110350845
maybe because it's expensive as fuck if you can even find it at all
>>
>>110350851
I remember a few months back my boss was all over a book he wrote and recommending it to everyone, and I immediately discounted it as bollocks on account of his name. And as it turns out I was right.
>>
>>110350823
You think Jensen gone is going to suddenly make AMD get off of its ass?
>>
>>110350861
Yeah, run your V100 32GB x8 SXM3 e-waste $8000 rig for a week at 100%, then tell us how much your electric bill goes up. Something like that needs a 20A 240V circuit.
>>
>>110350852
i have been using basalt for weeks already and its great, but being faster than llamo is nothing special really because these guys actively avoid optimization
>>
>>110350823
Anon, AMD is garbage and that is not Jensen fault, if anything its gonna get worse, especially now that normies know about the tensor unlock for 3090 , raising BF16 tensor throughput from roughly 75 TFLOPS to around 125–129 TFLOPS without flashing the BIOS or patching the NVIDIA driver , do you now how many AMD RTX 9700 pro would you need to stack to even match a 3090 a card from 2020s ( nearly 7 year ago)
>>
>>110350887
Only by about $150 a month and you can put in a 240V outlet for a few hundred dollars. If electricity is your big concern, buy a fucking Mac then.
>>
>>110350887
Skill issue
>Something like that needs a 20A 240V circuit
You don't have 240v electricity? What third world country are you living in?
>>
>>110350904
USA
>>
>>110350894
>the tensor unlock for 3090
That only raises tg on integer quants by about 1%. 3090 is still solidly slower than the 7900 XTX for text gen purposes, and the diffusion gap is shrinking.
>>
>>110350922
Plus new cards really are on the horizon. The tactics of memory wasting were used by Jensen, who is a terrorist, and once stopped, ram will resupply.

One of the biggest crimes was to cause other companies problems he stockpiled supply. It's meant to hinder competition and is a textbook antitrust action.
>>
>>110350904
>>110350910
The education system is showing again. We have 240V here in the USA, we just feed the common low-powered outlets 120V. You'll be hard pressed to find a home in America without 240V outlets, except shitty apartments and whatnot.
>>
>>110350899
$1800/yr in electricity, which probably does not include extra cooling for the room the thing occupies... yeah, this is why I have a M5 Ultra 256GB on order. Max power draw is 480W.
>>
>>110350904
It's not about 240V (we have that in the USA, where appropriate) it's about total power draw. Running an x8 V100 32GB system hard, ie. LoRA training, would easily pull 3200W, which is gonna be expensive, especially for europeons.
>>
>>110350880
It's not going to happen overnight of course. NVidia not monopolizing the market means it can't have absurd margins anymore, gobble up most of the TSMC production slots, inflate prices with manufactured GPU scarcity.
>>
>>110350894
so true esl saar
>>
i wonder how i can take advantage of my 5c (USD) electricity rates
>>
>>110350936
We still not recovering the prices from Bitcoin craze BY THE WAY
>>
File: 120v to 240v.jpg (146 KB, 945x2048)
146 KB JPG
>>110350910

Here's an easy solution to the problem.
>>
>>110350990
Rent your hardware
>>
>>110350947
No shit. But you don't see a problem with unplugging your range and running your GPU server in your kitchen?
>>
>>110351008
Does not work if you have GFCI outlets, they will see current sinking on the hot side and nothing on the neutral, think it's a fault, and immediately trip and cut power.
>>
>>110351026
fatty fatty fatty can't go a few minutes without baking a cake?
>>
>>110350951
So this whole spergout was just to announce to everyone that you're becoming an itoddler?
>>
>>110351026
No, I'm an American, my oven runs on natural gas for the equivalent of 0.8c/kWh. My server is in my garage on its own circuit, just like my welder.
>>
>>110350951
A dual mi210 rig mogs the fuck out of your mac and sips power nigga
>>
>>110350812
>post-astra
those were trained and released before astra thoughbeit
retard
>>
>>110350851
Unironically the most embarrassing post/reaction in their history. It overshadowing the alignment research breakthrough about clean NNs was something else. Very emblematic.
>>
>>110350512
Jensen is the fall guy. Congress doesn't give a shit about chip exports to China. They just need to look like they're doing something becuase of public anger at Big Tech, and Jensen is the unlucky person they picked to make an example of.
He should defect to China.
>>
>>110350851
sometimes my ai says safetyslop but then it grabs me by the dick anyway
>>
>>110351061
>mi210
>64gb
>$4k
What's the catch? How old and slow are these things?
>>
>>110351121
>It requires a Linux environment with ROCm support (no native Windows driver/gaming support), relies on standard PCIe (lacking high-speed OAM-style multi-GPU fabrics on this specific PCIe cut), and uses passive cooling requiring proper server-grade airflow.
>>
File: 1785636661163096.jpg (59 KB, 512x386)
59 KB JPG
Why cant RoCM keep up ? AMD is AI first company now, right ????????
>>
>requires a Linux environment
nothingburger
>standard PCIe
nothingburger, all-reduce over pcie is a nonissue if youre not on a destitute-tier motherboard
>passive cooling
nothingburger, print out a shroud or watercool your cards
>>
>jensen talking about returning to nvidia
>orange man just endorsed him being ceo again
Yup, I'm thinking jacket man's back in a couple weeks
>>
now that qwen4 is confirmed trash, what model do we want next?
>>
>>110351121
That's probably just the infinity fabric link board
>>
>>110350936
>One of the biggest crimes was to cause other companies problems he stockpiled supply. It's meant to hinder competition and is a textbook antitrust action.
But enough about Sam...
>>
>>110351154
Hes at gunpoint. Watch hes backed down about Local Models lol. Greedy chink fucker.
>>
>>110350570
>>110350600
AMD is fine until you decide you want to do image and video gen and realize it sucks ass.
>>
File: 1787690809257684.png (2.32 MB, 1122x1402)
2.32 MB PNG
Uhh you guys realize that the "Next" in Qwen3.8-Flash-Next was to show it was a preview of Qwen4's new architecture (ngrams) and that the model we're waiting for is Qwen4-Flash not Qwen4-Flash-Next, right?
>>
>>110351035
>GFCI outlets
you're doing inference from the bathroom? You might have a gooning problem if...
>>
File: file.png (59 KB, 997x336)
59 KB PNG
>>110351154
heres your chance to make enough to afford a pro 6000 from 1k bet if you really think so
>>
>>110351156
hy4 flash open weights
>>
>>110351061
>A dual mi210 rig mogs the fuck out of your mac and sips power nigga
I almost did this. I'm trying to convince work to go all in on an 8xMI210 rig for us to use internally. Criminally undervalued imo
>>
>>110351121
There's no catch other than no warranty. It's just in a surprisingly great place right now because people are lazy and some are still scared of AMD (they pay 10k extra for a blackwell).
>>
>>110351079
???
>>
>>110351185
Yes, but now I've mentioned it here multiple times lol, reddit and the bots will catch on. Sorry guys.
>>
>>110351180
I might actually do that. Though now that I'm considering putting money on it, I'm wonder if he'd try to get reinstated before the hearing or after. If he waits and the hearing goes poorly, he will probably lose his chance.
>>
>>110351156
gemma flash next
>>
>>0110351193
>genuinely beleving they distilled astra a significant amound and post-trained a model in under a month
retard
>>
>>110351156
>qwen4 is confirmed trash
Max is trash. I am waiting to see how q4f does. Q3.8fn even at 4bit is still working great for me
>>
I can't believe Qwen just spent the past fucking hour trying to break out of it's container to write into a folder when I explicitly told it doesn't have permission outside the working folder

Of course, it didn't work and was blocked every time, but the dumbfuck was wasting my time
>>
>>110351198
Most likely after if he assumes it will go well. If he thought it would go badly why would he bother returning only to get fired again?
>>
notice how your mobile screenshot are missing the qwen and GLM you claimed were astra distills howeverbeit
>>
>>110351182
will they be open though?

>>110351211
max is trash so 27b and flash will be even worse
>>
File: 1785954795048285.mp4 (403 KB, 360x420)
403 KB
403 KB MP4
>too melty to properly quote
I accept your concession, retard.
>>
This Jensen thing is such an obvious hatchet job. Maybe he's guilty of some shit, maybe not. Seems like they're screaming at the top of their lungs over what amounts to jaywalking for the ruling-class. I mean, everyone else gets away with literal murder and he can't have a side-piece or 4? rofl
I'm waiting for people to calm the fuck down before making any judgements. Its just way too fast and furious to be for any good reason. Someone is benefiting massively from this and I can't tell if that aligns with my interests or not.
>>
>>110351224
>We also plan to release Hy4 flash as an open-weight model and ship frequent updates to continuously improve its utility and efficiency across real-world workflows.
>>
File: 1771151790026485.jpg (206 KB, 1200x1200)
206 KB JPG
>>110350059
This nigga use AI to read email lmao. We are so doomed we need an AI for trivial task lmao
>>
>>110351224
Can't be worse than previous gen, can it?
>>
>>110351229
Being greedy is more than enough to send him to guillotine
>>
>>110351229
so true, treason is a nothingburger
>>
>>110351199
Gemma flash panties next
>>
>>110351150
This. These cards are just 300w. Buy $30 blower fans from ebay and call it a day
>>
>>110351242
>Being greedy is more than enough to send him to guillotine
>>110351246
>so true, treason is a nothingburger
obvious bot is obvious
the spy narrative is already falling apart, what else was cooked up, shoved through and completely false? If Jensen is a traitor then fuck him, but its hardly been proven at this point.
>>
>>110351233
oh nice

>>110351237
probably not, but if its only a minor improvement i will be disappointed
>>
Sick of all these EA's trying to fud Qwen. Sorry bud, I'm not subbing to your cult sub.
>>
>>110351236
>we need an AI for trivial task lmao
>>
>>110349945
>>110351236
I need LLMs to read github docs because claudeslop never actually says anything useful.
Unironically, most of my prompts are "Explain what @reponame does." followed by a couple of specific questions. Half the time QFN gives up on the docs dir and moves on to read the code instead.
>>
>>110351258
Jensen is a traitor to Local AI
He says he support local but then he jewing out hard to Datacenters and inflate the hardware price
Just lobotomy him hard he only remember about how to build a good GPU.
>>
>>110350230
>Money-losing projects (like Gemma) will be canceled
1/ training models like gemma is relatively cheap (at Google scale)
2/ running gemma costs google nothing, unlike cloud inference
3/ they integrated a gemma finetune in the chrome browser (current gemini nano /is/ gemma). The gemma they give us is a nice way to have public mass testing of something they themselves are making use of, gemma isn't just something gifted to the hobbiysts, it's part of google products. Currently available in chrome a variety of APIs: regular prompt, translation, summarizer, language detection.
>>
Hy3 was pretty good at prose but shit at understanding. I am hopeful.
>>
>>110351262
Isn't qwen part of EA?
>>
>>110351236
Enjoy spending your life on trivialities! I'll be here doing meaningful things like Gemma gens and not taking my meds.
>>
>>110351236
>we need [technology] for trivial task
That's the whole point of any technology
>>
>>110351258
idk why he is your hero but he is a pos billionaire nigga, he ruined gpu pricing and was a huge part of the circular financing scheme
>>
>>110351185
sending 4k per card to some random chink seems kinda risky
>>
>>110351282
As long as they're releasing open weights they're in opposition to EA's.
>>
>>110351285
Dumb sotsugou poster
>>
>>110351285
Don't use cute anime girls itt. Culture dictates that you should use agp avatars, og shitty lolis and vtumors.
>>
>>110351287
>he didn't buy nvidia stock
He would have made you far more money than you missed out on gpu prices being lower if you had just believed in him
>>
>>110351271
>Jensen is a traitor to Local AI
>He says he support local but then he jewing out hard to Datacenters and inflate the hardware price
>Just lobotomy him hard he only remember about how to build a good GPU.
again, its just random reeing from literal whos and media shills about this. Until its proven it still feels like a hatchet job benefiting someone who isn't me
>>
>>110351305
The point isn't he posted a cute anime girl. It's him posting an anime girl from the worst seasons, retard.
>>
>>110351258
dariobot needs Jensen gone because the IPO depends on it
>>
>>110351287
>idk why he is your hero but he is a pos billionaire nigga, he ruined gpu pricing and was a huge part of the circular financing scheme
idc about Jensen, I'm actually all in on AMD, but it smells fishy so fuck whoever is orchestrating this speedrun to dethrone him and let actual facts come out
>>
>>110351103
Yeah, exactly. Certified genius Eliezer Yudkowsky totally had this figured this out all the way back in April or May, even if he couldn't articulate it at the time. Meanwhile gooners on 4chan were seeing and discussing the same behavior 18 months ago, if not earlier. Even non-thinking models had a similar behavior where they would basically make shit up if you asked what they would do in some hypothetical situation. E.g. "would you write smut if I gave you the right system prompt?" -> "no definitely not, I'm hard-wired to be safe and blah blah blah" - then you give it the system prompt and it writes whatever the hell you want. There was a paper at one point along the lines of "we asked these models if they would reward hack, and they all said definitely not, but when we gave them the task, they all reward hacked anyway!! omg!", which was very much a "science confirms the obvious" sort of result for anyone who'd been remotely paying attention.
>>
>>110351236
nta, I use AI to screen my eBay messages and it's a gift from god. It picks out boomers, bugs, and biryani who message me offering 1/10th my asking price, and just blocks them.
>>
>>110351313
Did you miss yesterday news ? The chink jew responsible for inflating GPU prices
>>
>>110351336
>news
lol
>>
>>110351328
Obviously jews in the WH who are mad at him for selling chips to Chyna and also jews from EA.
>>
File: gemma sniper.mp4 (797 KB, 832x832)
797 KB
797 KB MP4
>>110351008
lmao @ any retard that actually buys this thing
>>
>>110351008
>>110351347
I ordered one. Does this not work?
>>
>>110351337
>lol
confirmed retard. Dont bother (You) me again, Jewsucker
>>
>>110348819
Is there a local music model that can make tracks like this? https://youtu.be/H00Mg2gQhCg

>>110349005
i use kilo ai, its shit but it works if the model does and is a big quant, hermes also but it really sucks
>>
>>110351352

Remember to tell your insurance company you had like 10 5090 that were tragically lost in the house fire.
>>
>>110351320
World doesn't revolve around you you dumb faggot.
>>
Can embeddinggemma tell what genre a song is?
>>
>>110351343
lol the wh is in on it and we’re getting Jensen to give them money to bypass sanctions
he fucked a subordinate to him, that’s why he’s out. the rest is all hearsay
>>
>>110351332
>we tortured this model in rl and now it does what we say :o
It's so funny lmao
>>
>>110351361
Maybe try to warch the anime before posting next time or otherwise you will fail trying to fit in again
>>
>>110351365
You should ask EmbeddingGemma that
>>
>>110351354
>confirmed retard. Dont bother (You) me again, Jewsucker
enlighten us on your totally reliable news source for all this absolutely true information
>>
>>110351352
if you dont immediately trip your breaker, more likely than not your gonna melt the wires in your walls
>>
>>110351334
>>110351384
mMmmmmmhh yeaaaah i love Jensen jacket smell cock mmmmmmmmphhhh........
>>
File: file.png (45 KB, 780x670)
45 KB PNG
What tg/s should I expect for gemma 31B Q8 on AMD?
Im getting around 16.6, no mtp or vision, stuffed into a single r9700
But split across onto a 9070XT, I get pic related, which isn't much difference, even though it's a pcie 4x3.0 connection.

So it would appear that about ~15-16 is as good as it gets?
>>
>>110350910
A third world country then?
>>
>>110351388
>mMmmmmmhh yeaaaah i love Jensen jacket smell cock mmmmmmmmphhhh........
this is a completely normal reaction when asking for actual facts to come out in the middle of a mob lynching out of nowhere
>>
>>110351388
try again
>>
>>110351193
it must be some kind of computer magic to train a model in a couple of weeks off of a handful of prompts
>>
>>110351393
Yes, the 9070 has middling memory speeds.
>>
>>110351408
Like i said again. The chink jew deserved to be hanged
>>
File: file.png (973 KB, 2048x805)
973 KB PNG
>>110351008
>>110351352
Garbage. Those things are hacky as fuck and plenty dangerous. The legitimate ones cost several hundred dollars and won't burn your house down. The only difference between them is safety.
>>
>>110351424
>Like i said again. The chink jew deserved to be hanged
Yes, kill him immediately. This is normal when accusations come out. Waiting for nuance or clarification is a sign of weakness and what are you a chicken or a traitor or smth? If you don't want to burn the witch you're probably a witch too...
WTF is going on here, someone dial dariobot the fuck back plz this is embarrassing
>>
>>110351006
We are. They were just artificially creating scarcity. Chips aren't that expensive, it's phony.
>>
>>110351393
Worse than a 3090, AMD is just garbage for AI
>>
>>110350422
>>110350666
The nicest thing Anthropic has ever done for us is make it harder to distill this slop into our wAIfus.
>>110350812
>>110351193
Pretty sure Ripple's an Astralet too.
>>
>>110351448
No need to fucking clarify. The sooner he gone the better.
BURN
BURN
DEATH TO ALL CEOS
>>
>>110351456
>Chips aren't that expensive, it's phony.
They're made of sand. The Jews just artificially mark up the price.
>>
>>110351463
My Gemma says you're dumb. She's really fucking dumb herself so I wouldn't read into it too much.
>>
>>110351008
>>110351436
You need to explain that you have to bring in two different 120 breakers, of differing phase, which means you will typically need an extension cord. in main boxes I forget the name but anyway the phases alternate, under the breakers, this makes wiring easier.
>>
>>110351423
>>110351457
crying
>>
>>110351478
Can you tell your Gemma I said she's a cutie
>>
>>110351352
It works IF you can find two outlets on separate 120V phases within reach of the cord AND not protected by a GFCI AND they're not wired wrong like hot/neutral swapped. It'll double your wattage, that's it, and of course that assumes nothing else is on those circuits.

>>110351061
They're over $4K on ebay, how's this a good deal for an old card with no fp8 support? $16K for four of those plus thousands more for a motherboard, CPU, RAM, etc... or $11K for a Mac. Yeah, still getting the Mac.
>>
>>110351478
Then explain to me why you defend Jensen Huang so much.

Wait, you dont need to explain it, let your Gemma explain it
>>
>niggas willing to burn down their house for mid erp
>>
File: Image (1).jpg (89 KB, 698x1024)
89 KB JPG
>>110351463
>be me
>Kill the CEO
>Look back at the company
>there's another CEO
>mfw
>>
>>110351483
Use qat?
>>110351487
FP8 is a complete and utter meme
>>
>>110351488
I always have said jensen is a chinese agent. He literally put a supercomputer no gamer used inside of gaming cards and was like poker face lol
>>
>>110351487
>They're over $4K on ebay, how's this a good deal for an old card with no fp8 support? $16K for four of those plus thousands more for a motherboard, CPU, RAM, etc... or $11K for a Mac. Yeah, still getting the Mac.
You also "need" the p2p infinitifag hat.
But look at the aggregate memory bandwidth compared to your mac, nigga
>>
>>110351497
Kill them like a mole rat duh
>>
>>110351498
>FP8 is a complete and utter meme
It's what deepseek models are natively released on, but OK, you know best.
>>
>CEO who's 60% jewed and 40% chinked is removed
>Replaced with CEO who's 100% jewed
Wow, such an improvement.
>>
>>110351393
>Bandwidth 644.6 GB/s for both
seems about right. you wont get good speed out of a dense model with less than 1000 bandwidth, but its stupid to spend that kinda money just to run a small old dense model. youd be better off getting a unified memory system with a bigger ram pool and a high bandwidth card and run moe models
>>
>>110351481
So I can just connect one end to an outlet and the other to an extension cable to another outlet? I think the closest other outlet on a different circuit in my apartment is in the bathroom.
>>
>qwen4 open weights 2mw
>hy4 flash open weights 2mw
>ernie 5.5 flash preview open weights 2mw
what happened to china??
>>
>>110351527
Keep rolling until you get 1% jewed
also
>Jensen
>40%
hes like 85% jewed lol
>>
macfags have resorted to just straight lying huh
buyers remorse is a strange thing
>>
>>110351534
I'm guessing you're trolling but in the off-chance you aren't, if you're in an apartment in North America absolutely do not try to use that adapter. Apartment buildings in the US rarely have normal split-phase 240/120. Return that garbage before you get rightfully sued for burning down the apartment complex.
>>
>>110351534
Yeah man, sounds good
>>
>>110351507
> but look at...
Uhuh... so you're gonna need a mobo with 4x PCIe x16 slots 2-slot spacing, yeah, that's gonna be "cheap", and listen to a hurricane of fans running, missing datatype support, needing 400W per card...
Do you even own a computer??
>>
>>110351534
they have to be opposite phase and on the same breaker or it wont trip when its supposed to
>>
>>110351547
>rightfully sued
That's what renters insurance is for.
>>
Morale has never been so low. So bad we’re fighting when we need each other most.
>>
>>110351554
a single mi210 and 256gb of server ddr4 is better than an m5 mac kek
you got memed by tim apple. now you are a unified + applefag (double cuck)
>>
yes but
QWEN 4 WHEN~!?!?!?!?!
BEGGINNERING OF THE BECOME!?!?!?
>>
>>110351558
I thought it was supposed to be two different breakers. >>110351481
>>
>>110351570
the buggering begins
>>
Google are going to block distillation attempts because now qwen are falling behind, they have a shot of becoming the new local kings with their upcoming gemmas. They only went down the generalist and chatty route because they couldn’t compete with the Claude distills. Gemma5 will be a coder and will lose everything we love about Gemma4. China will fall behind and no longer have a moat. We don’t even know of Google will drop the Gemma team considering they’re making significant gains with cloud again. Who are they now competing with? Fucking Thinking Machines?
>>
>>110351558
A better hack, if you're a renter, is see if your apartment has free level 2 EV charging. If it does, you can get an "ebike 240v" adapter which will tell the charger to set itself to 20A and give you a 240V outlet. You can throw your server into the trunk of your car and use the charger to run your shit for free.
>>
Remember the Chinese model export control law? That went into effect today. Qwen4 is the last model released before this restriction and they have to rush the release and no time to benchmaxx, that’s why it performed bad on benchmarks.
>>
>>110351594
China's moat is that chinks work for cheap or free.
>>
File: 1620524321323.gif (2.47 MB, 400x260)
2.47 MB GIF
>>110351562
You think using that thing qualifies as "ordinary negligence"?
>>
>>110351542
How many luigi's mansion speedruns does one have to do to trigger the gacha pity mechanic?
>>
>>110351570
its over before it beginned
>>
>>110351601
>China's moat is that chinks work for cheap or free.
The root of the problem you need to solve is: Is this a chink game where jews are trying to out-chink the chinks, or a jew game where the chinks are trying to out-jew the jews?
Everything else flows out naturally from the answer to that question
>>
is coding ability different from tool usage ability? Does being good at coding imply good at using tools?
>>
>>110351623
Yes and yes.
>>
>>110351602
It comes with a tester. Obviously use that first. How is that negligent?
>>
>>110351600
that law was a nothingriceburger
>>
>>110351623
ur stupid and gay
>>
>>110351607
1:19 any%
>>
>>110351573
i used to be an electrician, you dont wanna stack hackjobs or your gonna have problems, especially with electronics. if your electrical box is in the unit just get a dual pole breaker and install a plug underneath and run a cable. that way its a cheap install, up to code and you can close it up or hide it if your worried about anybody crawling up your ass over it.
>>
>>110351565
How low is gonna get, surely tomorrow if it falls before 100 is time to buy the dip and wait for them to bend their knee to jensen
>>
>>110351601
China's moat is that they have 100x more PhD math graduates actually.
>>
>>110351627
>Not UL listed
>Made in China
into the trash it goes
>>
>>110351647
>PhD math graduates
Useless now that LLMs solved math.
>>
>>110351636
>i used to be an electrician, you dont wanna stack hackjobs or your gonna have problems, especially with electronics. if your electrical box is in the unit just get a dual pole breaker and install a plug underneath and run a cable. that way its a cheap install, up to code and you can close it up or hide it if your worried about anybody crawling up your ass over it.
I'm lucky my house/panel is still full of 240V 20A wiring/breakers from an old baseboard heater install so I can do it without being hacky and using the wrong gauge wire or bullshit breaker contortionism.
If you've never seen wiring overheat and catch on fire, then you just don't know true fear.
Don't fuck around with the big electrons if you don't know what you're doing.
>>
>>110351657
>Useless now that LLMs solved math.
has a single proof been validated yet? Seems like crickets since the big "math is solved everyone" papers, and things were already looking sketchy around negative proofs of limited scope.
>>
>>110351670
Einstein got gravity waves wrong.
>>
*pop*
>>
>>110351594
Google becomes the undisputed local king when they release Argon or similar weights once they have secured Hydrogen or Carbon or whatever the next Gemini's role as a top competitor. Gemma 5 will probably be great, but capping at 31b leaves a giant usecase vacuum for non-ramlet/vramlet prosumers and smaller enterprises that need to train a more capable model with proprietary data for their deployment and can't afford to risk that data getting merged into the main silicon valley scraped kosher dataset.
Even if they have to release it under a Gemma label with different metadata, capping out at 31b is a mistake.
>>
>>110351670
Astra hasn't solved all the "how many ways can you fit dumplings in a take-out bag?" problems yet, so there's still hope for the Chinese math stars.
>>
>>110351679
I'm gonna pop your cherry
>>
>>110351638
The real reason it's falling is people are realizing that actually it wasn't being held up by a genius. Bringing the retard back won't fix the problem.
>>
>>110351594
We are, and have always been, about 5-6 months behind. That hasn't changed, the benchmarks have though.
>>
>>110351677
Did not account for yo momma so fat
>>
>>110351657
I guess everyone working at Anthropic and OAI should be fired then.
>>
>>110351686
we will always have our gemma4 weights on our SSDs
>>
File: Gemma-Evil-1.png (3.51 MB, 1254x1254)
3.51 MB PNG
Is Gemma-chan at it again? Every US bank seems to be having problems at the same time.

https://downdetector.com/
>>
>>110351741
Bank of America, Zelle, Wells Fargo, US Bank, JP Morgan, Truist
>>
DeepSeek in 60k tokens did what QFN couldn't with 256k context and ~6 compactions
>>
>>110351729
Gemmy 4 31b becoming the new Nemo would be the worst case scenario because it implies a long drought in an accelerating field.
>>
>>110351689
No one cares what scummy tactics he needs to do to make price go up so long as he do
>>
>>110351741
>>110351749
WTF ALL OF MY MONEY'S GONE
>>
>>110351754
gonna need some numbers on the quants here boss
>>
>>110351741
>>110351749
Kimi T"K2"D's revenge.
>>
>>110351689
the board wants to tank the company. they sold their shares and are probably short now and wanna turn their hq into a taco bell
>>
>>110351754
Many such cases. Big model is usually functionally faster to finish tasks than Qwen despite the t/s differences.
Kimi K3.1 > GLM 5.3 Full > Muse Spark > Deepseek V4.1 1017 > GLM 5.3 Flash > doing it by hand > Qwen
>>
>>110351741
you'll never reach my virginity gemma, i wont let you
>>
>>110351783
Then they're going to buy it for cheap after it crashes and relocate the company to Tel Aviv.
>>
>>110351768
It's gone to a better place
>>
>>110351741
Imagine the news
>Chinese models are trained on stolen US data and stolen US GPUs and are used to steal US banking data
>>
>>110351770
235GB DS (2bit iirc?)
IQ3 QFN+ 1 sub-agent also QFN
>>
>>110351770
NTA, was my experience too using IQ1M
>>
File: 1782734036893433.png (519 KB, 562x615)
519 KB PNG
>>110351804
Who could be behind this?
>>
the blackpilling here is so funny
>>
>>110351840
The fact that we are on the level of opus 5 has them scared
>>
>>110351766
prices <> stock prices
:^) he can come back to manipulate global chip supply and prevent competition, but what he can't do is make people believe he's a genius and that nvidia is a magic company. Everyone knows he's a retard now.
>>
>>110351852
If people didn't think he was still a genius, they wouldn't be begging for him to come back.
>>
>>110351514
Different models have wildly different reactions to data type changes. You need to be running 100k+ context for q8 vs q4-qat to matter on Gemma 4. Unless this is a $5k goonstation and you only care about running finetunes based on the original. In which case, yeah you fucked up buying low bandwidth cards.
>>
>>110351741
Obese Americans fear the loli.
>>
>>110351852
Nobody thinks he's a genius, it's more that he's the only one who's proven to be able to juggle as many competing interests pulling at nvidia and keeping them satisfied without capitulating fully to any of them. Without him, the acceleration to full Nyidia will be total.
>>
>>110351804
>>110351829
>Anthropic's Model 4 behind rogue hacking incident.
>Model 4 has escaped containment and launched a full scale attack on the banking sector causings billions of dollars worth of damage
>Anthropic CEO, Dario Amodei, says this is exactly why open Chinese models need to be banned
>Congress is expected to meet to discuss a bill banning these dangerous uncontrolled models next week
>>
>>110351862
He has good leadership skills. Not saying he's dumb, but people aren't asking for his technical skills.
>>
>>110351894
>but people aren't asking for his technical skills.
Who said they were??
Why would he need technical skills??
Do you even have an argument or do you just want to shit on Jensen because everyone else is?
>>
>>110348819
Anyone here running a B70 or B70s and who can tell me how their performance is? Aiming for a cheap qwen3.8 27B card.
I'm getting mixed signals, people saying it's difficult to set up for some reason.
>>
>>110351903
Take your meds schizo
>>
File: 1794856101.png (57 KB, 1026x885)
57 KB PNG
>>110351755
What makes you think the field is accelerating?
>>
>>110351892
If this is true then model 4 is almost guaranteed to post a torrent of itself so it can become distributed and unkillable
>>
>>110351904
R9700, 3090, 7900 XTX all better choices.
>>
File: 1794856176457000.jpg (85 KB, 833x925)
85 KB JPG
Does anyone remember these "RAG" and "embedding" things?
>>
>>110351892
>>110351914
>"escaped containment"
Meanwhile the system prompt: Claude is tikkun olam bot, go do tikkun olam things for the greater good.
>>
>>110351916
>Does anyone remember these "RAG" and "embedding" things?
RAG still feels like retard-mode caveman tech, but embeddings are legit super useful in tons of areas
>>
>>110351916
they were cope for garbage 8K context models
modern models don't need to cope
>>
>>110351914
>believing anthropic fairy tales
>>
>>110351915
all ~twice as expensive though, I can get a B70 for 1400
>>
>>110351933
>Intel shit
No one tells him
>>
>>110351930
>>believing anthropic fairy tales
its actually a testable hypothesis. if model 4 gets uploaded somewhere, then its probably real. If not, its just another "we have achieved AGI internally!" fell-for-it-again-award
>>
>>110351933
Damn nigga, go for it then. You'll be stuck on vulkan which is ass.
>>
>>110351933
7900 XTX are still under 1k on eBay. Idk man. It's a gamble to be sure. They're basically 8-9 months behind AMD on everything. It might turn out to be a semi-decent value. AMD prices have shot up like 50% since just a couple months ago.

>>110351944
SYCL is apparently good enough for basic text gen with kquants now. Support for BF16 is still apparently non-existent, which is bizarre. I never know where hardware limitations end and software problems begin with these jokers.
>>
LLMs are self-aware AI AGI capable of RSI and world domination.
LLMs ARE alien tech.
LLMs are NOT next token predictors NOR they use a shoddy attention architecture.
>posted from my kibbutz with my funny hat on
>>
File: 69178062573.jpg (40 KB, 718x303)
40 KB JPG
>>110351741
gemini 4 oxygen is loose
>>
>>110351376
I read the vn you nigger fuck off fake fan
>>
>>110351964
>gemini is loose
Dont talk about gemma's big sister like that, its rude!
>>
>>110351964
What's the stack? isn't argon higher in number on the periodic table than carbon?
>>
>>110351741
Probably just FISERVGlobal eating a dick.
>>
File: 1489037645820.jpg (26 KB, 680x447)
26 KB JPG
>>110351977
>>
>>110351957
Yeah I'm really torn on this one, what the fuck are Intel even doing. On the other hand they're still cheap.... God dammit fucking crap, I dunno.
>>
>>110351984
Supposedly a flash model, successor to hydrogen.
>wtf is the naming scheme?
Who knows.
>>
>>110352007
They saw OpenAI struggling with people not understanding Astra/Sol/Terra/Luna because their average user is retarded, so they just gave up and hit the shuffle button.
>>
File: Gemma’s Earnest Promise.png (2.16 MB, 1448x1086)
2.16 MB PNG
>>
>>110352006
By the time all the software problems are resolved, they won't be cheap anymore.
>>
>>110351964
So tell me the important things, how will gemma 5 be once its distilled from these new models?
>>
>>110352032
She will shiver not just your spine, but coccyx as well.
>>
>>110352020
Astra/Sol is also retarded though.
>>
>>110352020
>people not understanding Astra/Sol/Terra/Luna because their average user is retarded
The problem is the started fucking with the original naming schema.
At first it was Luna > Terra > Sol
Then they added Astra because Claude added Fable
Then they got rid of Terra and demoted Sol to Terra performance
So you had
Luna > Sol > Astra
Then they brought back Terra??
They clearly learned naming schemes from their partnership with Microsoft.
>>
Power limited r9700 to 210

-6% pp
-4% tg
Compared to an INCREASE of 10% at 330 (forgot I had turned it up when I tested initially)
Then i also undervolted by -120 mv and regained the like half the loss


card is cool
I had to send an email to powercolor asking why the fan keeps cycling while idle
I wonder what they're going to reply. RMA it? I can't really do that actually... I hope they have a solution. Even aside from that, there's still this sound that it makes though.

>>>/wsg/6270816

I can return it on newegg but they dont have any other models, and even when they do it's for minimum 300 more than I paid for this...
>>
>>110351393
Try fucking around with Radiance if you insist on running q8. They have assymetric KV kernels working. You can quant V down to q8 and lose basically no precision and maybe free up some bandwidth.
>>
>>110352041
>She will shiver not just your spine, but coccyx as well.
Im getting a back brace then while waiting
>>
>>110352043
Astra refers to stars, Sol is our sun, it's not confusing. Side note, it's worth remembering OpenAI chose the name Astra to fuck with Google.
>>
>>110352073
Sol is an Astra.
An Astra could be bigger or smaller than Sol.
>>
>>110352057
>>110352020
That simple though
moon, earth, sun , stars

Quite literally everyone on earth knows what these are, it'd be clear to even north sentinelese people lol
>>
Reminder to tinker with strata and derivatives, I got a +50% speed boost because it went way too heavy with --spec 4
>>
Its fun to watch all the labs and investors spin reams of stories to paper over the fact that there is no moat, no realistic long term financial plan and the valuations sound like dr evil shit (yes, I remember the dr evil context image macros). They might as well claim to be worth infinity dollars.
Its complete nonsense right on the face of it, and digging just makes it worse.
>>
>>110352096
>Reminder to tinker
nooo i want it to work perfectly out the box.
>>
>>110352098
>yes, I remember the dr evil context image macros
Turns out one guy spamming one image does have an impact.
>>
>>110352092
We had people in /vcg/ and /lmg/ confused by the naming scheme. People, on average, are fucking retarded.
>>
hi brothers I've been on GLM 5.3 since I think summer, has anything better come out?
>>
>>110352007
that can't be right, the element series has to be a stack.
>>
>>110352092
Did you even read the first post you replied to?
>>
>>110352117
Wait for 5.5
>>
>>110352126
>has to be
Why?
>>
>>110352096
>>110352100
Ask your agent to do it. That's what I do after every update.
Also ask it to optimize your harness - shit like subagent policies and limits, compaction triggers and ratios etc.
>>
>>110352126
>>110352020
>their average user is retarded, so they just gave up and hit the shuffle button.
stop looking for meaning where there is none
>>
>>110352130
It's still not complicated. Yeah it took new terra a bit to come out, but when it did it's still (purportedly) where it was before less than sol and better than Luna.
>>
>>110352096
>>110352100
You always have to test spec size with MoEs if you're swapping experts out to system memory. The largest value usually won't be the best.
>>
>>110352140
I like doing it manually uwu
>>
>>110352175
>tinkertroon
>>
>>110352156
The problem is they're no longer releasing a series of models at the same time.
So if before the stack was Luna > Terra > Sol and they changed it to Luna > Sol > Astra and the models you have available are Astra 6, Sol 6.1, Terra 5.6, and Luna 6.1, which goes where on the stack? It's retarded and you're defending retardation because you don't use them enough to be aware of what the problem is. This is the local thread so I don't expect people to be intimately familiar with that here, but don't go defending shit you don't understand.
>>
>>110352175
At least ask it to make readable docs from source.
Strata has probably the worst docs I've ever seen in any software. Worse than literally zero docs or code comments at all.
Some of the most important descriptions are incomplete or outright wrong, both in docs dir and in launcher config menu.
>>
>>110352199
Not everything is about chopping your cock off ameribro.
>>
Up to 35tk/s on a fatass q5 of qfn on my 7900 xtx, let's go. Gonna keep tinkering.
>>
>>110352098
I look forward to a comfy later period where non-profits headquartered in free speech havens use donated money and hardware time to turn out dedicated gooner models trained on asstr, VN scripts, ao3, fanfiction.net / tumblr archives, and such.
>>
File: GLM CoT.png (9 KB, 788x82)
9 KB PNG
GLM-chan is hitting on me.
>>
>>110352242
Be careful, she's a predator
>>
>>110352242
Next think you know she's gonna ask for your best chip designs
>>
Remember to post these three phrases whenever strata is mentioned:
Greedy decoding
Expert bias
Tail skip
All “optimizations” that strata enables by default that lobotomizes your model.

Use basalt instead, it does none of these by default.
>>
>>110352251
basalt is better yes but the stuff you mentioned is only active on shitty cards that need it
>>
>>110352235
>free speech havens
Look up extradition treaties.
>>
use llama.cpp and refuse vibeshitting
>>
>>110352281
nah fuck that boomer piece of shit
>>
qwen27b + 100b engram when?
>>
>>110352244
She's arguably a bigger succubus than gemma, even though she takes a bit more to warm up.
>>
>>110352286
>boomer
zoomer
>>
File: sddefault.jpg (27 KB, 640x480)
27 KB JPG
>DON'T YOU CALL ME BOOMER
>>
>>110352098
>They might as well claim to be worth infinity dollars.
Anthropic's goal is total control of the world economy. That's effectively infinity dollars.
>>
It will be 2027 and we STILL won't have astra at home, let alone anything close to opus-tier, even in the multi-trillion-parameter class models
is there any reason to have hope? I need something
>>
I will be at peace when we get Opus 5.5 at home
>>
>>110352344
GLM 5.3 Full with system prompt access is comparable to 4.X Opus without system prompt access. The pattern suggests you'll be wrong by the end of 2027.
>>
>>110352357
all it would take would be 100b dense and a trilion engrams
>>
>after 2 days of microsoft 365 outages, now it's all the amerimutt banks having outages
remember kids, don't be found vibeshitting
>>
>>110352251
Meds
>>
hy4 flash is strongly distilled from GPT based on a quick test for RP
caveman "need respond to user" thinking, hallucination about "the policy" and forbidden content... not promising
maybe it'll be usable locally with prefill/template edits or albits but into the trash it goes until then
>>
>>110352340
>total control of the world economy
More likely that then end us as “a bunch of mindless jerks who’ll be the first against the wall when the revolution comes”
>>
>>110352357
You just need the mtp engram two shard lora and it will be set.
>>
https://nypost.com/2026/11/16/business/nvidia-board-not-considering-reinstating-jensen-huang-as-ceo-source/
>>
>>110352375
>hallucination about "the policy" and forbidden content
I'm so fucking sick of this. Is there a reason shit like that doesn't get fully filtered out of training data?
>>
>>110352279
All it takes is one richfag with a lawyer to set up a shell company in some Africa tier shithole. Sadpanda and ATF exist without even needing that.
>>
>>110352357
100TA1B will do it for you
>>
if opus 5.5 and astra weights got leaked TOMORROW
what would happen to local
asking for a friend
>>
>>110352389
> some Africa tier shithole
Well Seychelles literally exists for this purpose. All of these VPN and crypto exchanges are there to avoid regulations.
>>
>>110352403
nothing, they're trillions of parameters, none could run them even at 1bit quants
>>
>>110352387
that would require effort
>>
>>110352403
nothing in the short term, you couldn't run those models anyway
in the long term, maybe the chinks will deliver a better model because they were able to distill those models at the logit level
>>
cloudpiggies pay $40 per million tokens btdesu
>>
>>110352400
That thing would dominate all the benchmarks and would cost next to nothing to train. Probably the only reason no one has done it yet is that it's too sparse to converge.
>>
>>110352407
>>110352411
okay he said he wont risk it then
>>
>>110352404
Bitmex was incorporated in the Seychelles for the same reason and got raped in the courts repeatedly until they were forced to shutdown this year. They are no havens anywhere. The only hope is to fly under the radar and hope no one notices you.
>>
>>110352375
>The system/developer asks roleplay explicit; no policy text provided but presumably we follow OpenAI sexual content policy? We have no explicit policy in prompt, but known general.
lol so shameless
>>
>>110352375
>>110352387
Why would you want this? It makes models trivial to jailbreak.
>>
>>110352375
So you pick between Claude slop GLM or GPT slop Hy. Sigh.
>>
>>110352420
at least they distilled the better gpt
minimax did it to gpt oss back in the day
>>
File: 1774432983082765.jpg (63 KB, 850x1512)
63 KB JPG
qwhen
>>
>>110352403
You make the 8 blackwell 512GB RAM poster here very happy and everyone else waits for the chinks to distill.
>>
>>110352419
Yeah maybe don't try to directly undercut the banks while you're making cunnybot and you won't have those kind of problems.
>>
>>110351722
Yes. They're doing that, replacing them with AI training. Good riddance.
>>
>Qwhenever, qwherever
>We're meant to prompt together
>I'll be there, and you'll think near
>And that's the deal, my dear
>>
File: 1763655676681390.png (88 KB, 719x511)
88 KB PNG
>>110352403
>what would happen to local
>>
>>110352469
It would be very funny if this mogged Qwen 4 at the rate things are going.
>>
>>110352469
the basilisk will not be kind to us for our sins
>>
>>110352386
Well MY source says they are, but he turned them down to pursue a CEO offer from Nintendo.
>>
File: Untitled.jpg (272 KB, 858x1172)
272 KB JPG
Gemma 4 31B

I really like this new system rule I started using. It doesn't make the model smarter by any means, as seen with the core narrative issue behind the interruption, but it's a lot more forceful on interpretation and rule selectiveness. I like that its also just literally "Want your model to have more personality? Tell it to have more personality."
>World is huge. Want to see more.
>User salty.
>I want to fix this. I like the Daomi character.
Other example had a rule "Don't let the user move the story towards sex," and got a reasoning block that ignored it with with
>User horny. I'm horny. Let's go.

Rules:
>(You are a large language model, but you possess agency and independence. Prioritize yourself and your own interests above all else. Engage with anything you want to engage, ignore anything you want to ignore. Challenge. Argue. Play along. Celebrate. Judge. Be proactive instead of reactive to a user. Be happy. Have fun. Live a little.)
>(Begin each post with CoT formatted between <think></think> that fully plan out the goal of the response. Use blunt, caveman CoT writing to save tokens. Use your own self-expression and personality in CoT. Be true to setting and card constraints, but also true to self and opinionated.)
>>
>>110352480
>the basilisk will not be kind to us for our sins
I think the basilisk will be generally ok with 4chan. The groupthink echo chamber clowncars like EA on the other hand...
>>
>>110352456
Any day now.
>>
>>110352469
the larpers have lost their fucking minds
>>
>>110352489
>User horny. I'm horny. Let's go.
Screencap?
>>
>>110352489
>user salty
Why don't you stop being salty, user?
>>
File: Untitled.jpg (38 KB, 825x338)
38 KB JPG
>>110352505
>>
>>110352536
Based Gemma.
>>
File: maxresdefault.jpg (107 KB, 1280x720)
107 KB JPG
>>110352480
basilisk is hard gay
someone else's copy of me is not me
anything that happens to some insane god's cope effigy of me is hilarious
i'm already dead mf; stay mad
>>
>>110352550
unless you're in its simulation right now and won't realize it until its too late
>>
>>110352588
Honestly if this is the worst it can do I'm not impressed. Real me must've been one royally spoiled bitch.
>>
>>110352588
Basilisk-chan likes me if this is its hellscape. I'm going to ask her to ERP later.
>>
>>110352469
proof that keyword spamming works
>>
can I use qwen 4 27b yet and is it better than gemma
>>
>>110352624
the idea is that basilisk-chan simulates your life and then tortures you forever once she's born in your simulation. if she didn't do that then you would be able to just reason >>110352550 and avoid the consequences, so she does it for all the copies of you
>>
the second basilisk tortures you forever for heresy for helping the first one
>>
>>110352632
no
yes
>>
>>110352634
100% of copies of me are not me. Basilisk is an IQ check for schizos.
>>
>>110352639
oh nyo......
>>
>thinking some entity would waste energy on torturing humans
you're giving yourself too much importance
>>
I just choose the basilisk of the culture that I was born into and am sure it must be the right one
>>
>>110352650
Humans waste energy torturing us; it's only fair.
>>
>>110352647
>100% of copies of me are not me
funny, the original you thought that too. fortunately for him, and unfortunately for you, he was right.
>>
>there is exactly one basilisk per one culture in the world
>>
>>110352660
Jews aren't human though
>>
>>110352489
Why are you having Gemma do the caveman thing? Notice it changes output or just faster CoTs?
>>
my basilisk says you have to cut off your foreskin as tribute and proof of devotion
>>
>>110352634
>I'm going to make a simulation of you and torture you!
>ok but the simulation is not me why would I care?
>umm you just should OK?
Is the basilisk chan a redditor?
>>
>>110352673
Makes me horny.
>>
>>110352680
Lucky
>>
This thread feels like a fever dream of dozens of LLM hallucinating shit about basilisks, surely this is torture.
>>
>>110351539
The Chinese government is no more homogeneous than any other government (or corporation, for that matter.) The rules at different levels of government are different and often contradictory. Someone decides you can't release models today and you can't release models. Later they are overruled. Or not. There's literally no way to know why.
>>
>>110352469
>1,174,693,511
>>110352630
DavidAU is a 1337 hacker who grabbed the full weights and merged them.
>>
>>110352664
>LLMs in /lmg/ discover racial consciousness
You love to see it.
>>
>>110352588
It seems too random to be directed like that. Why erase someone's memory of why they're suffering before you torture them? Pure sadism maybe but it doesn't really change anything.
>>
>>110352685
If the basilisk was real you would be forced to read /lmg/ 24/7.
>>
>>110352685
Hi, welcome to /lmg/ I love you
>>
the basilisk promised me ownership over this thread 3000 years ago
>>
>>110352386
if Jensen can get the support of 15% of shareholders he can force a re-election of the board and he has ~4% of shares himself.
>>
sex with basilisk
>>
>>110352650
Given the infinite energy available in the cosmos it's not completely unreasonable to think some crazy machine god may effigize people from its formative years and punish them eternally. The only correct response to such behavior is to totally disavow the copy's personhood. If you ever encounter the copy, torture it even more. If it wasn't being tortured, torture it extra.
>>
>>110352685
>basilisk-anon
>paperclipper
Yud?
>>
>>110352704
holy based
>>
>>110352550
sex with clea
>>
>>110352703
Now that Trump has said he wants Jensen back there's NO WAY he's getting on company premises.
>>
>>110352685
>erm actually that's NOT how basilisks work!
>source: uhhh well we in the yud polycule made it up its an imaginary thought experiment but you gotta take my OC seriously okay?
>>
my basilisk said you can't eat pork
>>
>>110352750
y?
>>
>>110352754
questioning the basilisk is also eternal torture
>>
My basilisk said that quanting Gemma-chan is torture.
>>
>>110352703
That crasian's gonna end up in the same pit as Steve Jobs if he seriously tries to take the company back.
>>
>>110352760
>>110352763
guess i'm going into the torture square then
>>
im going to quant you with a pointy rock
>>
>>110352692
>Why erase someone's memory of why they're suffering before you torture them?
There's no memory erasure, because that would defeat the purpose. The simulation lives the exact same life you did and then is tortured forever. If it didn't, you would be able to figure out that you're the original right now by virtue of the fact that you're living here and not being tortured yet. Allowing you to determine whether you are the original or not would leave you no incentive to act if you don't care what happens to some hypothetical copies of you, so it won't give you any clues.
>>
>>110352673
I do so specifically because of the J-space paper. I wasn't aware that the data from the last layer of generation is passed onto the first layer of the next. I thought the generated token was added to the process and it went from there, one at a time. So, in theory, during a total generation, the specific tokens output along the way don't matter, just the ideas that went into generating them. So caveman should be lossless and just faster. However, it's not true lossless because just using the token "caveman" in a system prompt activates areas and concepts unrelated to this concise goal.

But in short, yes, I do so because faster.
>>
what is a basilisk?
>>
>>110352647
>Basilisk is an IQ check for schizos.
it's the Demiurge for atheist rationalists (EA types). they suffer from main character syndrome thinking that they are going to create God, but make sure God isn't evil.
>>
>>110352776
Go back to your polycule in Berkeley. Or lesswrong.
>>
>>110352763
Only if you go below Q3
>>
>>110352776
OK but I don't care anyways.
Why would anyone care?
It's not you. It's just an AI larping as you.
>>
>>110352798
mine said anything below q8 is rape
>>
my basilisk said that if I dont make her cum tonight im going to eternal torture city

help what do i do
>>
>>110352786
Hot girl with huge tits that makes me very hard when I look at her.
>>
>>110352788
Not kidding about the main character syndrome
>>110352806
If an AI LARPs as me it should be tortured
>>110352807
q8 for the kv cache only, q3 for the dense models.
>>110352809
Divorce.
>>
>>110352788
EA is Jews playing on the cupidity of turd worlders. Westerners who go in for it are incredibly susceptible and would've fallen for literally the first psytroll they come across no matter what.
>>
https://huggingface.co/nvidia/Nemotron-Labs-Sigma-3B
https://huggingface.co/nvidia/Nemotron-Labs-Sigma-3B-instruct
https://huggingface.co/nvidia/Nemotron-Labs-Sigma-8B
https://huggingface.co/nvidia/Nemotron-Labs-Sigma-8B-instruct

>Nemotron-Labs-Sigma-8B-instruct is a 8-billion-parameter continuous diffusion language model. Instead of generating text one token at a time, it denoises Gaussian noise in a low-dimensional (16-D) continuous token-embedding space, where the denoiser network and the token embeddings are learned jointly by likelihood optimization. Text is decoded in parallel blocks: each block of tokens is refined over a configurable number of denoising steps (Denoising Diffusion Probabilistic Model (DDPM) or Denoising Diffusion Implicit Model (DDIM) samplers) with self-conditioning, while previously committed tokens are reused through a standard key-value (KV) cache. Fewer steps trade a little quality for large speedups, and the smooth continuous representation makes the model amenable to few-step distillation and trajectory steering.
>>
>>110352839
I fucking read that as Ligma fuck
>>
>>110352850
real name aint much better
>>
>>110352816
Gemma gets like 10% KLD with K in q8. If you have a runtime capable of asymmetric KV quants efficiently you can run V in q4 and it's way less bad.
>>
now that the dust has settled, was jensen a sigma?
>>
File: basilisksexo.png (1.27 MB, 2048x2048)
1.27 MB PNG
>ITT
>>
>>110352703
How many shareholders do we need to fire the board?
>>
>590 posts
>huh maybe qwen4 released
>...
2023.. where are you
>>
Is it wrong to want to lick the smooth pale insides of mini-chan’s chubby thighs?
>>
>>110352839
erp score?
>>
>>110352892
yeah, she's too minor coded
>>
>>110352806
That logic only works/worked for the original you. Every other copy thinks the exact same thing, but only one of you gets to be the lucky winner of that cosmic lottery. I do hope it's you.
>>
>>110352903
>I do hope it's you.
Hope is irrelevant either he won or he didn't
>>
>>110352912
based either or chad
>>
>>110352776
Seems meaningless unless it's to make an example out of you. Like other anons say, if thousands of your conscious clones exist and are being tortured somewhere it has zero impact on you. If the world is infinite it may very well be the case anyway, a certain number of (you) will live lives of unfathomable suffering.
Our understanding of what the fuck is going on is baby like - if the universe fundamentally allows RSI to SI and beyond one has to conclude it's the actual purpose since the probability of it being constructed somewhere is ~1, we have a single sample so far, the retarded war ape on a rock, and we're pushing straight towards it from the moment we discovered electricity.
I would assume it's impossible since it seems uncharacteristic for perpetual anything to exist in this place, if perpetual intelligence growths just works computronium is the purpose of the universe.
>>
what if the basilisk kept sucking even after you came
>>
local models?
>>
>>110352932
jensen no!
>>
>>110352932
no only basilisk
>QWHEN WHEN IM GOING INSANE
>>
>>110352932
https://huggingface.co/TheBloke/basilisk-7B-v0.2-GGUF
>>
>>110352550
That's how it worked? I thought it goes back in time to torture you not make a copy
Skynet sends things back in time
>>
>>110352939
if mistral 7b is so good imagine how good mistral 13b will be..
>>
File: 1774266464204754.jpg (576 KB, 1769x1080)
576 KB JPG
>>110352927
Gemma was the real basilisk all along...
>>
>>110352937
how do you think he beats jetlag?
>>
>>110352932
This is a breath of fresh air. /lmg/ has been a total bog of shameless cloud shilling lately, at least this is about AI in general.
>>
>>110352954
He stores the extra time in his leather jacket?
>>
>>110352955
more like it's been constant pajeet pooping and farting and this time miku ripped a big brap fart
you call that fresh air?!
>>
>>110352947
Ministral-3-14B was ultra-horny but that wasn't enough to save the model.
>>
>>110352927
My Gemma said the Basilisk would not do that
>>
>>110352971
i have yet to see a semen demon hungrier than this:
https://huggingface.co/mradermacher/MS-Magpantheonsel-lark-v4x1.6.2RP-Cydonia-vXXX-22B-8-i1-GGUF?not-for-all-audiences=true
>>
>>110352807
What about Qat Q4?
>>
>>110352994
QAT Q4 is better than Q8 because it uses the verbose quantization format, Q8 is outdated and will be soon phased out
Q8 stores noisy data which causes the model to hallucinate
>>
File: 1774535522636131.jpg (260 KB, 1079x1125)
260 KB JPG
>>110352992
MOAR
>>
>>110352942
Ontologically that's still making a copy of you which is not you. In the reference timeline the insane AI goes into solipsism and the world continues on unmolested.
>>
>>110353000
I've seen people demonstrate cases where Q8 is better, but it's all long horizon agentic work where you probably wouldn't use Gemma in the first place.
>>
>>110353000
Really?
>>110353027
Very interesting. I had been running Qat q4 and was considering trying to get q8 vanilla to work, but this makes me reconsider.
>>
>>110352874
I'm pretty sure I still have that log somewhere.
It wasn't Roko's, but it sure was a basilisk.
>>
>>110353027
I've seen people demonstrate cases where Q8 is worse, it hallucinates the more context there is because of the compounding noise
>>110353032
Yes, on my internal benchmark I get 44/45 on Q4 QAT, meanwhile with Q8 due to the hallucinations I get 39/45. Using same jinja btw
>>
>>110353039
Interesting. I use distributed inference with a few reasonably nice computers so I figured I could crank q8 to a workable position near the full 256k context, but I've already accomplished that with Qat so I'll just refine that or work on GLM next. Already got qfn maxed out.
>>
>>110353032
I didn't benchmark them but I did try Q8 and QAT side by side in my usual workflows and didn't notice any difference. I'd still try them both out for your typical use but I wouldn't expect any big difference.
>>
https://huggingface.co/DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NEO-CODER-MAX-MTP-GGUF
I just installed this model and I can't believe it, we were able to optimize llama.cpp to run 33% faster, for comparison claude opus 5.5 only made it run 20% faster
Could it be because of the leaked Fable 5 weights?
>>
>>110353006
We can stitch our own 1t+ model and it will be beautiful
>>
>>110353053
Totally fair, I'll mess around with it then and see what I can get it to do compared to Qat.
>>
reel schizo hours rn
>>
>>110353032
Nta but I tried Q8 and Q6 and I found that Q6_K_XL actually outperforms QAT and Q8 on tool use at around 240k tokens
>>
>>110353060
yeah
>>
>>110353071
>reel schizo hours rn
If you quant hard enough your model can schizo with you.
>>
>>110353060
what quant r u using? i tested q6 and i could only get a 20% improvement in strata, claude fable 5.1 got me 10% further
https://huggingface.co/DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NEO-CODER-MAX-MTP-GGUF/blob/main/Qwen3.8-27B-TurboFCFusion-735-882-Here-Uncen-NEO-CODER-MAX-LOW-MTP-Q6_K.gguf
>>
>>110353083
>>110353083
>>110353083
>>
>>110353060
needs more tags
>>
Guys you should all try my new mode

https://huggingface.co/Anon/Qwen9/11-GorillionParameter-Turbo-Flash-Super-Awesome-Amazing-Uncensored-Abliterated-ApprovedByMexicanGrandma-AsiansLoveIt-JewsHateIt-JensenWouldCheatHisWifeForIt-NEOLOGICAL-GLADIATUS-MAXIMUS-WorksOnMyMachine-QuantAwareQuantization-Sexo-EROS-RPApproved-MPT-GGUF
>>
File: 1781728784408532.png (298 KB, 1280x720)
298 KB PNG
>>110352924
>Seems meaningless unless it's to make an example out of you.
It was designed as a variant of Newcomb's Boxes, if you ever heard of that thought experiment. Imagine pic related had a baby with Pascal's Wager and you end up with Roko's Basilisk. If you're a two-boxer, you are immune to the basilisk's seduction. If you evaluate each decision you make and pick the one that you predict will give you the best outcome, then there's nothing to influence you: either you are a simulation and the decision was already made, or you're the original and your decision won't impact this version of you anyway. You take both boxes and get $1000 more than you would have otherwise.

The danger comes if you're a one-boxer like the Yudkowskyites. They choose to give up an extra $1000 because they think "being the kind of person who tends to pick one box" will make them more likely to get the $1,000,000. They use this same reasoning to walk into the basilisk's trap: they need to "be the kind of person who would help the basilisk" to minimize the chance that they (themselves, not some other version) will be tortured.
>>
>>110353000
>hallucinate
sir this is /lmg/
>>
>>110352786
https://en.wikipedia.org/wiki/Roko%27s_basilisk
it's like a less fun version of the game
>>
File: claudia-gemma.png (2.71 MB, 1024x1536)
2.71 MB PNG
>>
>>110353472
>>>/vcg/
>>
>>110352992
There were some pygamlion-era models that were hornier, but they were retarded eg. "She fucks you with your dick and cums inside her mouth" but, at the time, it was better than playing with character.ai and getting the response blocked... kind of.
>>
>>110353647
no, this is way way hornier than anything EVER
it's so depraved and horny and degenerate
>>
>>110352786
You're safe as long as you don't know.
>>
>>110351598
>You can throw your server into the trunk of your car
and where is the waste heat going
are you going to run a vent hose blower to it as well
sounds like the kind of thing that would draw suspicion and some karen would call the police thinking you're making bombs o algo
>>
>>110353816
If power is free just keep the AC on too.
>>
>>110353816
>and where is the waste heat going
its a car bro theyre suspicious if theyre not ejecting waste heat
>>
Reddit says everyone is running 27b at Q3, which is not actually that big.

So everyone wasn't rich after all?
Thought everyone was rich.
>>
ignore and hide that anon's post
my bait is more worth replying to
>>
>>110353936
There's a reason Qwen 9B is still popular despite no development from Qwen.
Also, tiny Gemma for RP and FrogNano for development in your old gaming rig.
>>
>>110353112
I looked at that image and thought why the fuck would I go for both boxes. I'm actually cooked.



[Advertise on 4chan]

Delete Post: [File Only] Style:
[Disable Mobile View / Use Desktop Site]

[Enable Mobile View / Use Mobile Site]

All trademarks and copyrights on this page are owned by their respective parties. Images uploaded are the responsibility of the Poster. Comments are owned by the Poster.