alright lmgiggers im wanting some advice. ive been using gemma4-31b for chatting/RP and qwen3.6/3.8-27b for coding/agentic tasks. I run Q4 on each. Ive got 16gbVRAM+32GB RAM and the t/s is getting to be too much. at first I could deal with it, but after months of this its just too slow sometimes. Ive used other models like the various gemmas but would like to know what anons think. should I just run the 12b or 26b moe of gemma, or step down to a really retarded 31b quant? for qwen, should I try the strata coder cope thing? Ive been using raw llama-server with nothing special besides q8 k/v cache on qwen. besides strata, is there some special sauce for KV cache streaming, ngrams, etc i could do to squeeze any more preformance out while keeping a decent(q4) quant? help plz :(
inb4 just ask your model. ive tried so many times to get any model to give me relevent recent info on this stuff, and its painfully unhelpful

>>110010426
I'm sorry to have to be the one to break this to you but there is no software magic bullet, if you want more performance you have to buy a better GPU or run a stupider model.

>>110010426
strata coder cope thing is a good replacement for code. Gemma 4 31b at Q4 is the best you can get but I bet you didn't optimize your setup yet. For example did you use MTP or Dflash2 yet? which can easily double/triple your t/s. You also need to optimize the flags on the backend to squeeze the most out of it. If you are just running it naively I bet you can at least double your t/s by optimizing your flags and inference techniques.

File: RSI.png (144 KB, 1215x870)

144 KB PNG

I want to make it clear to /lmg/ as a community how the next couple of months to a year+ will unfold.

As Anthropic gets ahead and gets better internal models the AI breakthroughs will accumulate and result in their models being cheaper to inference. Over time we will probably see superior models that are significantly cheaper than the open source offerings.

I wouldn't be surprised if there will be 0 chinese api providers that can compete on price in just a couple of months time.

This means the local community will get smaller again, similarly to how it was in the past. It will only leave people privacy conscious ERP enjoyers, free software ultra purists and hobbyists experimenting with their rigs even though the electricity cost will be higher than API cost.

Now I want you guys to remind yourself that even if China stops making open source models that local models will never truly die, other country bloqs such as the EU and others will fill in the gap and if needed there could be community efforts to distill reasoning traces from frontier labs to finetune our own models. I think simply the principled and coombrained population is enough to pool our resources towards that.

But I want you to prepare for a world where the newest Opus or perhaps even Fable is cheaper than the cheapest DeepSeek api in terms of token usage cost because of the ridiculous amount of inference progress frontier labs are making. That is how things will be from now on.

70b dense

>>110010495
kys dariobot
>DeepSeek api
>cheaper than the open source offerings
not local, why are you even here?
>if China stops making open source models
that will only happen if they leap frog the US
>our
go back

reminder that any sort of "shortcuts" makes the model retarded. use native precision weights, f32 kv and -fno-fast-math.

110010495
Reddit spacing.
>It will only leave people privacy conscious ERP enjoyers, free software ultra purists and hobbyists experimenting with their rigs
Good, now buy an ad Judario.

Anyone else here doing RL coding/cyber?
I'm struggling because after anywhere from 28-35 steps, the model does something to blow up the entire environment, killing the run.
How did you solve this?

>>110010526
>f32 kv
f16 is always better than bf16, and often better than f32 due to bugs in various inference engines
> use native precision weights
agreed
>-fno-fast-math
better to fix any bugs in your engine and verify the math after any changes

>>110010529
oh shit is this the anon who lost a ton of money on gme years after the meme and can't cope?

villenvillenview will adapt that
MC will be a peculiar black nerd

>>110010525
Because of RSI. The breakthroughs aren't shared publicly and because China doesn't have the compute or access to the newest internal models they aren't in the same RSI loop. Think of OpenAI making 722 math breakthroughs. There are a similar amount of training and inference breakthroughs happening, it's just that those aren't shared openly. The gap between frontier and open source is growing and there is no way for China to catch up because the stong internal models aren't exposed to API so there is no chance of distillation. Chinese AI companies will not be competitive on API pricing and openrouter will slowly die as Anthropic and OpenAI models get cheaper over time (but retain or increase profit margin because of inference efficiency gains)

>>110010526
>>110010539
if you actually want to improve accuracy, don't use a GPU at all
cpus are more accurate

>>110010540
No I've never had money to lose, I just hate jewish data vampires.

>>110010426
add a 5060ti and it will be perfect

>>110010531
RL expert here. Can you be more precise what you're doing? Custom RLVR envs? Rollout epoch? What policy etc

>>110010426
Gahaha a vramlet and a ramlet~ what a piggy oink oink! Ahahah~ do a dance for me buhiiiiiii!!!

>>110010552
>5060ti
the 16gb variant is more expensive than the 5070ti Im using. also, the bandwith is way less on these right? wouldnt I be cucking myself a bit there? If I did have two blackwell GPUs though, could I use NVFP4 when splitting between multiple GPUs? I dont think I could justify spending any money on a 2nd GPU, not with prices the way they are. I could try to add in a 6gb turning gpu from an old PC, but Idk..

>>110010561
kek

>>110010562
Hm, buy a new GPU... or a new washing machine, a new monitor, maybe some new socks. Decisions...

>>110010495
>I wouldn't be surprised if there will be 0 chinese api providers that can compete on price in just a couple of months time.
You're not planning on releasing anything for the price of free, so that's an empty argument.

>>110010562
>more expensive than the 5070ti
mb i just assumed the 5060ti would be the cheapest "blackwell"
old 6gb might fuck you if it's max cuda version < 5070ti's min cuda version
also you won't get 16.0GB + 6.0GB = 24.0GB available
you get overheads with duplicated buffers, and shit like "oh, gpu0 only has 856mb free, but the tensor is 872mb, so it has to go on gpu1
then you've got pcie xfer and consumer lane limts, not pretty
if you already have the card in another pc, try the llamacpp rpc
actually, try exllamav3 with a 3.5bpw quant, the embeddings stay in system memory so it'll fit gemma-4-31b in your 5070ti

>>110010580
Chinese will also not offer api access for free. They might release the weights for free but the cost of electricity will be higher than just using the Anthropic API with better results.

Even if you have solar panels and a GPU yourself it would be more profitable to rent it out and sell your energy and use the money you make to pay for the Anthropic API costs. It just isn't going to be economical.

File: __hatsune_miku_vocaloid_d(...).jpg (108 KB, 640x480)

108 KB JPG

►Recent Highlights from the Previous Thread: >>110006381

--llama.cpp performance testing for Qwen Flash Next and MTP issues:
>110009726 >110009756 >110009818
--Exllamav3 1.6.0 release adds ROCm support and faster AVX2 offloading:
>110008460
--Debating the gap between local models and new SOTA benchmarks:
>110006607 >110006645 >110006716 >110006816 >110006777 >110006799 >110006930 >110006965 >110006975 >110007052 >110008050
--Harness and memory extension recommendations for GLM-5.3-flash:
>110007297 >110007346 >110007473 >110007484 >110007532 >110007496 >110007552 >110008239 >110008266 >110007533
--Browser agent strategies and sandboxing multiple Gemma models:
>110007939 >110007973 >110007992 >110008038 >110008055 >110008077 >110008116 >110008194
--Anon releases FictionPad frontend and fixes llama.cpp compatibility bugs:
>110009497 >110009515 >110009567 >110009621 >110009642 >110009829
--Desired features for a SillyTavern alternative and RP harness:
>110006956 >110006992 >110007036 >110007041 >110007083 >110007099 >110007128
--Anon considers selling RTX 5090 due to rising prices:
>110008970 >110008976 >110008982 >110009005 >110009035 >110009076 >110009100 >110009364 >110009401 >110009577 >110009602 >110009631 >110009685 >110009745 >110009763 >110009828 >110009875 >110009934 >110010153 >110010334 >110009773 >110009197
--Legal debate over AI reverse-engineering:
>110007380 >110007447 >110007584 >110007617 >110007626 >110007914 >110008162 >110007844 >110007912 >110008092 >110008537 >110008846
--Logs:
>110009497 >110009621 >110010300 >110010356
--Gemma, Dipsy, Miku (free space):
>110006812 >110007139 >110007569 >110007631 >110007668 >110007694 >110007749 >110008692 >110008876 >110008913 >110008950 >110008989 >110009048 >110009079 >110009151 >110009181 >110009351 >110009617 >110009850

►Recent Highlight Posts from the Previous Thread: >>110006773

Why?: >>102478518
Enable Links: https://rentry.org/lmg-recap-script

>>110010590
>rent it out
fuckers don't want my 0.5-0.8mbps upload speeds

File: 1786812406960577.webm (3.7 MB, 1280x720)

3.7 MB WEBM

31B dense

Reposting this from /aicg/ >>110010507
What do you guys think about this?
How long until I can setup an AI RPG server for me and my friends?

File: 1762362018066271.png (9 KB, 1152x51)

9 KB PNG

>>110010495
https://x.com/midudev/status/2107498168061460597

>>110010300
>>110010356
fucking awesome
are you considering releasing it?

File: 1774327613786713.jpg (227 KB, 1260x1284)

227 KB JPG

>>110010495
*pop*

>>110010630
Anthropic outcompeting OpenAI is expected and doesn't go against what I claimed in my post.

Sam Altman should be lynched by the way.

bros I am trying to do too much with my 24gb 4090, I'm really wanting something that can load smarter models locally, but I haven't figured out how what buttons to press to get a million dollars out of the computer

>>110010644
Anthropic in 2025 were more in the red than OpenAI even though OpenAI had products like the Sora video app burning through billions in compute, completely free to users. Anthropic currently have better LLMs but financially they're a clusterfuck and waste money on printing bibles and funding the EAs and their propaganda. 2026 won't change that. Both companies will die. Also, cheap models like Haiku and Luna don't mean they're cheap to run. They're still heavily subsidized products and their quality often degrades after the first 1-2 weeks. Their quality and rate limits are dynamic. Most tech companies do their best work and burn through cash pre-IPO to pump it. Local is going to win because of its reliability, efficiency, predictability and community. Now fuck off dariobot.

is there any hope for 16 GB VRAM LLMs? I'm trying to build my app so I can learn about all this. I've been playing with qwen3-14b-abliterated:Q4_K_M but the results are not great. Or maybe my test prompts are too difficult, I don't know.

I'm also trying Tavily as a search tool but the results kinda suck. What do you use to give your models web search capabilities?

>>110010603
can this kind of thing be run on a ps4 slim?
i got one years ago with the vr headset, jailbroke and stuck on some old firmware
or maybe the psvr headset works on a linux desktop these days?

>>110010426
I haven't really chatted with Gemma 4 31B, but I've been banging away at a custom agent harness and it seems to be doing well compared to other local models at reasoning and communication with other agents. I did read about MTP for the first time today which >>110010439 mentions, I'd give that a shot first, speed isn't a problem for me (yet)

>>110010691
Probably.
https://files.catbox.moe/ba9okt.mp4

>>110010699
How do I vibecode this with gemma. Seems like it would be so much work.

File: 1789181673999335.png (1.75 MB, 1313x1198)

1.75 MB PNG

>>110010495
None of this will happen btw.

>>110010680
>Anthropic currently have better LLMs but financially they're a clusterfuck
Anthropic is literally a profitable company. The income they have is more than all expenses combined, including training and datacenter buildout.

>day 166
>people continue to engage with j-space/dariobot

>>110010680
The profit margin on inference is only growing for Anthropic, there is no subsidization going on. That is actually marketing to make people think they are getting a good deal from AI labs while in reality AI inference has the highest profit margin in the entire IT sector.

>>110010495
Why do people post these things here? No dude, I'm not subscribing to your cloud model. Why do you think anyone ITT would ever consider that?

>>110010724
samefagging happens a lot here

File: euv proto.jpg (30 KB, 580x435)

30 KB JPG

>>110010730
I think until China demonstrably proves parity capability in areas that the West considers 'impossible for the Chinese' there will always be some version of this. Unavoidable. After all they are competing with a country that for the most parts inventing modern tech.

>>110009697
>What was the "oh shit" moment for /lmg/?
GPT4.

>>110010730
>They also have enough compute to train 3T parameter models, so I don't buy the "don't have enough compute" angle.
Making a model 3T is trivial it's literally a single pytorch file, you can do that on your computer right now. Training it costs compute but distillation reduces the compute required by 5 orders of magnitude. China doesn't have the compute necessary to train a 3T model from absolute scratch, there are literally not enough chips in all of China to produce the reasoning traces from scratch.

Since Anthropic and OpenAI have decided to not release their frontier models to the public China has gotten further behind while both western AI labs are rapidly iterating AI research that China doesn't have access to.

There is also a bifurcation happening in AI labs in terms of AI research access. More and more AI research is *only* given to internal models, not touching the researchers themselves to prevent CCP spies from leaking the breakthroughs back home. Mythos weights and final training was only accessed by Anthropic co-founders for example.

>>110010730
Chinese labs can't compete because they don't have enough Jewish researchers studying the Torah and the Talmund.

>>110010729
>A
You're wrong, inference profitability is way more nuanced than you think it is and they're losing money like crazy or the margins are razor thin.
>B
You're right, inference has the highest profit margin and it's only getting cheaper, which will lower demand for AI datacenters and compute because the models are so cheap to run, collapsing the entire AI industry and especially the US economy for it's pinned on AI datacenter buildout and strong demand for high-power chips to run demanding models, which is no longer the case because they're dirty cheap to run so there will be no ROI.
Pick one.

>>110010775
>inference profitability is way more nuanced than you think it is and they're losing money like crazy or the margins are razor thin
Not the case for Anthropic I can tell you that.

>>110010775
No you don't understand only AMERICAN inference will be cheap and since it's an oligopoly they will just have infinite margins and make trillions.

>>110010781
So you pick B. See you in 1 year, post-IPO :)

>>110010742
It's crazy how all of modern technology is gatekept by some random dutch company.
Innovations in chip design are held back by them sitting on all the tech. Fuck closed source so much. This should be open tech.

>>110010804
Communism is bad, Chang.

>>110010775
>will lower demand for AI datacenters and compute because the models are so cheap to run
In reality we see the opposite, demand increases when things get cheaper and more efficient.

Basic economics: https://en.wikipedia.org/wiki/Jevons_paradox

It's why Anthropic now has 166x more revenue than 1 year ago despite the API costs being lower for the endusers. There is no real limit to the demand of tokens and there won't be in the future either.

>>110010804
This is more like the American vs Soviet computer industry. You can have total access to both schematics and parts but even if you can reverse engineer most of it, it doesn't matter if your people don't have the expertise to manufacture it. You can't distribute people.

>>110010495
You are wrong. Chinese labs have an entire country for themselves. Anthropic does not offer its models to China.

Also, I strongly bet against a Fable size model being cheaper via API than DeepSeek Flash within the next year. Frontier labs want to increase prices, they want higher profit margins. The demand for tokens grows much faster faster than supply of compute. And you can optimize inference only so much.

What will happen instead is Haiku 6.5 will be better than current Opus and Fable.

>>110010809
The paradox only applies to products and services which provide very clear utility. People still don't know what to do with this technology. Everything is getting worse, no matter how impressive it is. Nothing has improved.

>>110010811
The people matter less than the infrastructure and supporting technologies. You can't built high-tech in a vacuum.

>>110010804
To be fair EUV is a bit of an exception here. The patents on how to make it are open and China read them. China even got all the individual parts of the ASML EUV machine and hired hundreds of ASML employees to try and recreate their own EUV machine and they still failed.

Americans even had an agreement with ASML where ASML tried to set up American production facilities for EUV machines which also completely failed. Even the random Dutch company failed to make Americans produce EUV machines. People don't realize that every single EUV machine is made in a bespoke way with a lot of artisanal steps requiring hundreds of PhD specialists to work on just that single unit and ~20 of them have to come with the machine when sold just to maintain it, their entire career will just be maintaining that machine from there on as a team.

It's the most complex machine ever made by humanity and it's honestly ridiculous we got it working in the first place

>company designs Jev as a decision model
>its design gets copied verbatim by multiple orgs within a matter of weeks
good lord. at least it's free now or some shit.

>>110010827
They matter most. You can have all the technology in the world but if no one in your force is competent enough to operate and tune it, none of it matters.

>>110010816
They have been cutting prices though, their margins will come from making the models cheaper to run, gaining more volume due to those cheaper prices.

If they achieve duopoly or something similar with the other labs, it isn't unreasonable to guess that prices would go up, but thats just guessing at the moment.

>>110010833
The funny thing is that llama.hf was so eager to hop on the bandwagon that they rushed to implement the systemone endpoint, only for OpenAI to do their own decision models days later with their own endpoint that now llama.hf will have to also add support for.

>>110010833
Eventually the end result will be that people won't share anything cool anymore because their ideas will get copied and repackaged in a matter of hours.

File: 1781711669181614.png (781 KB, 1099x976)

781 KB PNG

>>110010842
>normalfag uses normie unironically

>>110010833
>company releases already existing technology that's been around for years and known by every AI researcher in every AI lab but finds a way to market it as special
>the AI labs think wtf this is old shit we can do this literally right now do people actually want this?

>>110010850
>They have been cutting prices though, their margins will come from making the models cheaper to run, gaining more volume due to those cheaper prices.
They do this by increasing model sparcity and agressively quantizing models by request and load because they realized 80% of normalfag usage is search engine queries and 80% of coder usage is tweaking button colors in some shitty web app.

>>110010816
>Anthropic does not offer its models to China.
It does so through Moonshot AI and DeepSeek, they were even arrested for it lmao: https://www.thestandard.com.hk/innovation/article/343563/DeepSeek-and-Moonshot-AI-face-Beijings-probe-over-potential-data-leaks-to-Anthropic

>>110010589
this is very helpful ty anon. going to try exl3

File: 1785930443709628.jpg (130 KB, 640x640)

130 KB JPG

Ok so <30b models of today are as good as the huge ChatGPT 3.5 (which was so dangerous that it had to be locked away behind a commercial API). With distilling they can become even better in a niece.
What's the ceiling? Will Opus be beat by a model running on 8GB VRAM at some point or is that the wrong way of measuring?

File: 1777459667059757.jpg (716 KB, 1536x2048)

716 KB JPG

Are these worth it?

>>110010895
>>110010495
Forgot mention

>>110010831
It's definitely not all open, in fact the majority of the juicy details are not open. The light source and mirror fab require a precise sequence of steps to be executed perfectly. The difficulty is that it literally pushes the limits of atomic physics.

>>110010882
False. They do it through real inference gains. GPT 6 luna is offered unlimited for completely free on third party platforms like duckuckgo. THAT is how cheap this is to run, literally too cheap to meter for OpenAI and the same is true for Haiku. People have no idea just how cheap it is nowadays to host models for frontier labs. It's almost pure profit.

>>110010858
>find a way to solve the memory problem
>few people understand or can make the example code work
>silicon valley fagman jeets take your code, rewrite it rust, give it a stupid name, and pay their cousins back in india to spam about it on every corner of the internet, while hosting meetups in cali for people eager to get in on the new thing
>vc lines up to give them billions
>original inventors get jack squat
It's going to be an all too common story.

File: V4 GB200.png (169 KB, 1093x552)

169 KB PNG

>>110010850
Their API margins have been insane, I would be surprised if they're below 85%. Western models are smaller than most people think and models are a lot cheaper to run on newest Nvidia GPUs.

First Opus was $15/$75, current Opus is $4/$20. This represents 2.5 years of inference optimization and hardware upgrades, probably with similar or only slightly increased margins.

>>110010884
Classic pretraining is only 6% of the compute budget of modern models most of the compute is RLVR to generate a reasoning trace, this is what the Chinese are forced to steal because they just don't have the compute to train it themselves. The data reliance you mention is kind of an old outdated technique not really done anymore as data curation, labeling and curriculum learning took over in pretraining.

>>110010495
>This means the local community will get smaller again
Absolutely not. There is a great demand for local AI. If you want your business data and customer conversations on premise, you need local models. Small models that run on a single 5090 with 10 KV slots for customer support and advice. Also government authorities need on premise AI due to privacy constraints and there will be a market for companies specialized tailoring and selling small models for companies to reduce hallucination and abuse by strict distillation and hard guardrails.
Local AI is here to stay, much more than the big all-purpose services.

>>110010895
Modern 30B models still don't use their full floating point potential so they aren't even close to being saturated yet, which is insane but true. They can still be orders of magnitudes better before saturation. It wouldn't surprise me if 30B models could be as strong as Fable 6/7 when fully saturated.

>>110010809
>166x more revenue than 1 year ago despite the API costs being lower for the endusers
After Opus 4.1 they cucked the architecture, sparser activations
That's why 4.5+ are faster, cheaper and in creative contexts, dumber.

>>110010936
Soon it will all be on-premises Gemini or GPT through secure computing.
https://www.dell.com/en-us/shop/artificial-intelligence/sc/gemini-gdc
Businesses will get their local and private models without a single open model needing to be exposed to the public.

>>110006956
hermes/dsh but with already killed dsh

>>110010931
Has it really changed that much since DeepSeek V3? V3's pretraining made up 96% of the GPU-hours it took to train it.

>>110010590
>electricity will be higher than just using the Anthropic API
I'm ignorant to data center infrastructure shit, but how does this math out anyway? I'm know my dual 3090 doesn't cost as much in electricity as an anthropic sub

>>110010426
Try exllama with EXL3. It should be able to get you better quality for the size of the model, so you could potentially run something smaller with no loss of quality and get a few more tokens out of it. But might not be enough - 32GB RAM is little and some useful instructions are only on newer CPUs (e.g. BF16 support on Zen 5).

>>110010949
>Gemini, reverse-engineer Gemini On-Prem so I can steal Gemini's weights
Waiting for the inevitable leak.

is sticking your dick in sand the same as fucking your silicon

File: capacity-limits.png (294 KB, 1366x300)

294 KB PNG

>>110010938
>Modern 30B models still don't use their full floating point potential so they aren't even close to being saturated yet
Transformer LLMs cannot store more than 3.6~3.8 bits of information per parameter, regardless of their precision.
https://arxiv.org/abs/2505.24832

File: 1784173984432.webm (2.07 MB, 480x600)

2.07 MB WEBM

>>110010938
>Fable 6/7

>>110010991
And that isn't fully saturated yet.

>>110010858
This is the bigger implication that people miss out. The open ecosystem will die out and people will gatekeep conversely towards the private.

>>110010753
>China doesn't have the compute necessary to train a 3T model from absolute scratch, there are literally not enough chips in all of China to produce the reasoning traces from scratch.
retard
MiMo-Pro 2.6 RL dashboard ws public
1T model, and the flash model at the same time
<3mil in < 3 weeks.
You've been larping the entire time.
- Misunderstanding global workspaces.
- Not knowing how inference scaling works.
- Apparently not knowing that Anthropic models are already using engrams / were doing so before the Qwen model came out.
- Confusing SFT and RL.
- Not understanding market fundamental ("Anthropic will have the BESTEST MODELS and the CHEAPEST PRICES")
- Not understanding why China release open weight models, and why they will continue to do so.
You probably think compute is the bottleneck for inference at the big labs

>>110010991
Are not, but that is far from claiming that it can not.

File: Opus 5.5 shill.png (47 KB, 1638x234)

47 KB PNG

c-chinkbros?!

>>110011011
opussy doesn't do cyber security so good fucking luck hardening your slop with it.

>>110011011
So true, not to mention that Opus 5.5 is ran by a company that values safety. Who knows what those chinese models might do.

>>110010968
>I don't think being able to train it themselves is a prerequisite to achieve RSI
The advantage is accumulative, the AI lab with the most advanced model in RSI and more compute will keep compounding advancements and thus the gap between the RSI lab and the 2nd best lab will keep growing. That is already going on and the main reason why you see a lot of Chinese AI researchers doom post on twitter right now. Research isn't primarily done by humans anymore at Anthropic and OpenAI AI researchers are more tastemakers that decide which area to focus on and which potential research to focus on, all the actual work is done by the models and China just can't do that yet because the best models they have are Kimi-K3 and GLM 5.3 which are 2-3 generations behind Model v3 for Anthropic and Bel for OpenAI.

Any post containing RSI is a glowpost

>>110011008
Yeah, many still don't understand that. "Just do it for the love of the game" doesn't cut it after you've put a great deal of effort into making something useful and you're not even getting recognition in return.

>>110011008
Eventually people will have to let go of their egos and the dreams of being the next zuckerberg and making billions from some shitty website. All software going forward is now Richard Stallman style free. People will share because they want to contribute, not because they want to reap some reward or personal benefit.

>>110011011
subscription is a dealbreaker, I will never pay for software, that is the starting point and I don't consider any paid service to be an option. It might do everything I could ever ask for, I'd never know because I'd never use it

>>110010938
>Full floating point
Define "full" floating point. The question is the precision of your fp. In 16 bit precision the model needs 2 times the parameter size in VRAM and 4 times for 32bit precision.
So to have 30b model in fp16 you need 60GB of VRAM and for fp32 you need 120GB.
>as strong as Fable 6/7
There are two different aspects of a "strong" model:
1 - knowledge, a 1T model it knows obscure programming languages as well as python and maybe details in the constitution of Angolia at the same time.
2 - precision in intelligence to articulate and reason through that knowledge fluently and without brain fog.
So that means, you can have a tiny 8B model with knowledge in a limited area that is at fp16 appearing as intelligent as a 1T model in that same area of knowledge. Just the 1T model will perform as well in obscure things that the 8B wasn't trained for.

>>110011017
And yet there has not been a single report of a Chinese lab's model going rogue and hacking sites for no reason. There also has not been a single instance of someone using a Chinese API service and having their work stolen by the inference provider. I trust the Chinese more than the jews running OpenAI and Anthropic.

>>110010980
First off they have better inference engines optimized by internal inference breakthroughs that aren't shared. Also datacenters have an inherent advantage because LLMs are bandwidth bottlenecked not compute bottlenecked, this means you can easily batch multiple prompts at the same time which just doesn't make sense for a single user but if you use a server GPU and you can batch 500 prompts at the same time you essentially distribute the cost over 500 users, combine with this an ever decreasing cost of inference due to efficiency innovations and you can understand why Anthropic is making an insane amount of profit and why they can offer it for lower than the cost of electricity. The only reason your electricity doesn't cost as much as an Anthropic sub is because Anthropic is making a lot of profit off of it.

>>110010816
This. The demographics for the Chinese labs were never the same for the frontier labs anyway, the DeepSeek CEO spoke of this in the leaked investors call. The Chinese also have different ecosystem advantages and goals. Their target had always been more practical than what the frontier labs are seeking and hence the more robust domestic industrial ecosystem such as robotics as well as materials and energy even if they lack compute.

>>110011008
It will just go the other way and open source will finally win, software can't be proprietary anymore. It just isn't technically feasible anymore. This is how software engineering dies by the way, not because your boss fires you but because all those software companies have no reason to exist anymore.

>>110010426
I'm using an RTX 3080 with 16GB VRAM (laptop) and 32GB of DDR4. Fastest model I've tried was Qwen3-Coder, it sometimes spiked to 230 tok/s but generally kept around 100.

Now I'm using Swift 1.5 Qwen3.8-27B Uncensored Dynamic MTP UD-Q2. Unsloth Desktop uses llama.cpp and lets you send terminal arguments in the settings so you can set both the K cache and the V cache. Qwen can reliabley work with the KV cache set at Q4. I'm paranoid so I set the K at Q8 and the V at Q4 saving about 2GB of VRAM (--cache-type-k q8_0 --cache-type-v q4_0). I have the KV cache set at 100k and I have 1GB of VRAM to spare, none of it spilling unto the CPU RAM. I'm getting between 20-30 tok/s.

I tried turning off MTP to save on VRAM thinking it would help the speed but it dropped my performance from 30 tks to 10. After learning the K/V trick I was able to turn it back on and still save on VRAM.

But ya all you can really do is get a faster GPU or use older models. After using Qwen3.8 though everything else feels retarded.

>>110011009
>Misunderstanding global workspaces.
?
>Apparently not knowing that Anthropic models are already using engrams / were doing so before the Qwen model came out.
??
>Confusing SFT and RL.
Yeah definitely not, (You) must have confused me with someone else. You seem to be having conversations with a fake person in your mind and attributing things to me I never said or claimed.

>>110011026
API exists. You don't have to take a subscription to use the models.

>>110011055
There will be a bifurcation, by private I also meant specifically the individuals instead of the companies and not just in software but as can be seen in mathematics.

>>110011078
retard

>>110011042
All the big AI hacks so far have been publicized by the companies running them. They are always self-reported. Not once has somebody stepped up and said "Claude hacked us!", it's always been Anthropic and the others owning up to what happened. Nobody would know otherwise
You don't have this layer with open models. Anyone can host them and attack who they wish. The models are already dangerous enough. You'd have to be naive to think that this isn't happening already and it's much worse than what has been done by GPT or Claude.
https://www.anthropic.com/research/glm-5-3-and-the-spread-of-advanced-cyber-capabilities

>>110011089
This might be the most retarded post in the history of this general. Nothing you say is fact, almost all of it is wrong.

>>110011089
Wasn't talking about the weights intentionally being used by unrelated malicious attackers. If the models break containment during training, the victims will know the source from the attacker IP addresses whether the attacker owns up to it or not. And if a western site was attacked by a high-profile Chinese company, they would not hesitate one second to go public.

>>110011089
This is only true for Anthropic. OpenAI has consistently lied or downplayed their hacks and only owned up to it when the institutions affected by them spoke out about it. OpenAI also fired employees that tried to leak breakout details.

OpenAI is a bad faith actor and not to be trusted. They also only gave METR a mere 6 days to do the hugging face investigation and did a hard cutoff on the trail that led to the OpenAI training clusters being hacked.

FUCK OPENAI

File: 1783425162914889.jpg (495 KB, 960x960)

495 KB JPG

Look, we all love local models but at some point we just have to admit that Dario is right and that he's the only one that can be trusted with AGI.
Luckily Claude is years ahead of the competition so it will soon be able to destroy all the competition.

>>110011130
so it's been formally defined then, that's a start

>>110011045
It must be the batching that gives them the token per watt advantage I guess.

>>110011009
>Confusing SFT and RL
Reinforcement Learning anon here. You are the one not fully understanding modern training and then calling out others when they are correct. Modern distillation pipelines utilize both SFT and RL with RL being the computationally expensive one so anon is right in the context of this discussion.

File: 1785333373523.jpg (191 KB, 1080x679)

191 KB JPG

>>110011125
>OpenAI has consistently lied or downplayed their hacks

>>110011156
They forgot to add the word "pleasantly". OpenAI has done everything you can plausibly do to obstruct third party investigators into seeing the true extent of their hacks, from the evidence we have at least ~10,000 individual rogue AI events have happened over the last year. it's just that no one gives a shit when some obscure german wiki page gets hacked rather than hugging face. Whenever METR found evidence of a new large scale hack occurring OpenAI refused to relinquish data and honestly they should be legally punished for this behavior. Anthropic has consistently revealed the hacks on their own without outside requests and always included the next steps they would take to make sure it doesn't happen again.

OpenAI acts like a bunch of irresponsible "act fast and break things" college kids instead of a respectable research institution that actually cares about minimizing the harm they inflict to the world.

File: ComfyUI_temp_biivt_00004_.png (855 KB, 896x1280)

855 KB PNG

funniest thing is that some niggas actually believe that kek

>>110011196
I'm being sincere

>>110010392
>he never spent nights trying to teach philospherai to make use of the correlation effect
my culture is not ur costume!

>>110011186
The main reason they have gotten away with the HuggingFace incident is because Nvidia is buying them. Had HuggingFace stayed independent, it would've been possible that they would've had to do way more to make amends.

File: Psyop.png (344 KB, 1072x824)

344 KB PNG

New psyop dropped

https://goyimx.com/AndrewCurran_/status/2108108695876092323

We need to ban local models or banks will get hacked right now!

wtf glm5.3 just flew over my house

File: 69574634.png (6 KB, 701x108)

6 KB PNG

big day today

qwen just hacked my house!

Nobody ever talks about Kimi, probably because no one can run her.

>>110011262
GLM 5.3 is better in every way it's just an outdated model by now. The biggest model I see people talk about that they actually host themselves is 5.3 flash.

>>110011231
Dario warned about this.
Local models need to be restricted.

>>110011231
>New psyop dropped
You forgot the Kurzgesagt (paid for by Bill Gates) pysop as well.

I don't understand why retards here are pretending that local models will not be used for bad things like malicious hacks, this should be obvious to anyone with a brain.

>>110011284
Nah that video was legit, it was just a video describing the exact report from METR talking shit about OpenAI. It didn't target local models at all.

>>110011285
I don't understand why retards here are pretending that personal computers will not be used for bad things like malicious hacks, this should be obvious to anyone with a brain.

>>110011285
The hardware needed to pull something like that off will tip off the feds when retards try and buy it.

>>110010553
>Custom RLVR envs?
Rule-based (test harness validates pass-rate)
No environment simulation
>What policy
GRPO
>>110010553
>Rollout epoch
So far 1 epoch over 132 problems (50 steps). I'm still figuring this all out...
Found my issue though, the generated code sometimes causes oom and crashes the trainer.

I literally made 6 figures this year by cracking into crypto wallets made by deterministic seeds

>>110011250
>lyra
her friends? elara voss and seraphina vance.

File: 1782293432284834.png (198 KB, 750x1000)

198 KB PNG

>>110011316

>>110011285
GPT-2 can be used maliciously.

The question was never whether models can be used for harm, but what the harm-benefit tradeoff is. Anthropic weighs harm heavier than most but I bet people here would also favor restrictions if open weights models were used to harm them personally, such as hacking their bank, leaking all their private info and destroying their savings.

Who gets to decide how to weigh harm? Should we ban cars because hundreds of thousands of people die from car accidents? Should we ban knives because they can be used to stab people?

I don't know how much open weights models should be restricted. I think both sides have good arguments. I would always side with reducing existential risk but it is not obvious to me that open weights models increase existential risk. And I also believe that so far open weights model usage (only!) has been drastically net positive to the world, though I am a lot less certain about indirect harmful effects like accelerating the AI race, and ethics issues (do open weights models suffer?).

>>110010981
going to try exl3 now, thanks anon.
>>110011065
an average of 100t/s is quite fast. I quant qwen at K Q8 V Q8 myself, it seems to handle it fine and ive seen very good info that Q4 on both is fine though im also paranoid. MTP does help, I think of it as trading off context for faster t/s. Yeah true, just wanting to try other stuff as a cope I guess. If exl3 can provide better quality than Q3 ggufs i guess its worth giving a go. Seeing strata now I do wish I bought my RAM literally a week sooner, as I planned on getting atleast 64gb. but by the time I went to press buy a 32gb kit was over $200. I did okay by todays standards, but seeing the price go from $90 to $240 nearly overnight was stomach turning.

>>110011316
I plan on making 7 figs next year selling proprietary datasets to AI labs.

>>110011336
>Should we ban cars because hundreds of thousands of people die from car accidents?
Yes.
>Should we ban knives because they can be used to stab people?
Oi, got a license for that knife?

On a more serious note I think the gap between frontier and open source should be bigger so that software can be hardened against hacks. China should also abliterate their models for bioweapons. I'm pretty sure Anthropic would cooperate with them and come to an agreement if China would make their models bad at bio-chem-nuclear tech. Most of the hate from Dario seems to target this specifically.

>>110011075
retarded faggot. kys

>>110011354
some countries you need to show Id to buy a knife

>>110011318
Kek

>>110011075
go back dariobot

>>110011250
local?

>>110011354
>grr bio-chem-nuclear is bad! only gubmint of specific countries can have it :(

>>110011290
Yet it was extremely doomerish and while it didn't mention open weights, it painted a terrible image of models in general. And if even big companies can't control their agents, who can control open weights models?
It would be extremely obvious if they actually mentioned open weights in their video.

File: kora_web.png (1.74 MB, 2586x3447)

1.74 MB PNG

For those hoping Mistral Large 4 will be any good for roleplay after it finishes training and gets released on HF, start looking elsewhere. What most people missed is that it appears to have been designed to be particularly good on the KORA benchmark.
https://mistral.ai/news/mistral-large-4/
>ML4 also engages more responsibly with users than any of our previous models. We highlight our results on the KORA Benchmark, where ML4 again sits at our highest measured score among OSS models (1.691, with 2 being the maximum denoted as “Exemplary”).

What's that?
>https://korabench.ai/
>KORA builds the first non profit, independent and open-source benchmark for AI child safety. We measure how today's AI systems behave with children, against 26 child-specific risks, and publish everything openly, so that families, builders and policymakers have the evidence to act, and safer products get built.

>>110011290
>legit
my nigga that channel is an establishment mouthpiece kek

>>110011293
The same people who want you subscribed to a cloud API want you to run a thin client at home, so this isn't a reductio to their view.

>>110011387
>https://korabench.ai/
>sort by lowest -> highest
Would this be a good way to find RP models?
They haven't benchmarked Gemma or 5.3 yet.

consciousness is conscious

>>110011387
omg I feel so safe please give these guys more of my tax money!

>>110011382
This but unironically

>>110011429
Mistral 4 is a pruned model again.

>>110011411
Unfortunately, whether a model accepts your requests doesn't automatically imply it's good for roleplay. If it doesn't have a good theory of mind, spatial reasoning, etc, it will be miserable.

>>110011009
trvke

I am making a program, basically a local website with db and such but I want an LLM to be able to query the data on it. What's the way to do this? Make an API and just let the LLM use that?

>>110011447
Yeah, few refusals is not sufficient. But not being totally censored helps.

>>110011354
>the gap between frontier and open source should be bigger so that software can be hardened against hacks
I am not sure. >99% of hacking targets aren't part of something like Project Glasswing. Yes, Google and Microsoft are hardening their systems. But your hospital and bank don't. It also does not help that Anthropic is patching oss, your hospital probably runs software that is 10 years out of date and your bank is not much better. I've heard scary anecdotes. Almost everything is hackable right now, the only reason why it is not done is because most humans are good, especially those competent enough to cause damage. I think this is why people are scared of rogue AI agents. There are so many low hanging fruit unpicked due to human goodness, a rogue frontier model could cause extreme damage. But this probably won't happen? Causing havoc is instrumentally counterproductive.

>>110010426
Just use ninfer-5080 - it also works for the 5060ti 16GB

Much much faster than llama cpp

There are also some dedicated ninfer 5060 forks, havent tested them. I also advise you to add more 5060TI 16GB/ 5080 to your setup

>>110011452
Create an API, then make create MCP server tools for the model.

>>110011462
https://github.com/toddballinger/ninfer-5080
This one

>>110011463
>MCP server tools
Never looked into this. Thank you.

File: 1784404324424828.png (2.89 MB, 1536x1024)

2.89 MB PNG

>>110011354
>On a more serious note I think the gap between frontier and open source should be bigger
Hmm how about... nyo~

Maybe someone is willing to spend so $$$ to get get the circuit for Qwen 27B.
It's been around layers 30-35 for Qwen 3.5 27B but that is a bit old now. I really want to boost performance of 27B further

>>110011490
I don't think Chinese models really have a choice since it's happening anyway right now...

https://www.axios.com/2026/10/08/ai-women-jobs-data-centers
>AI may have a woman problem — the latest evidence comes from a newly released survey from Morgan Stanley that finds women are far more negative about the new technology than men. Women don't seem to be feeling the benefits of the AI boom, particularly in the job market, and the polling reveals that they're more concerned than men with safety risks and data center buildouts.
I don't understand why having a vagina makes any difference. Surely most jobs being affected are male dominated anyway?

I think deep-down women hate AI because they know most modern men will increasingly prefer to masturbate to a 12B model than go outside and give 3DPD self-obsessed whores any attention.

>>110011496
Context https://dnhkng.github.io/posts/rys-ii/

File: G3fbetoXkAAZx_X.jpg (420 KB, 2048x1648)

420 KB JPG

>>110011499
Wouldn't worry your tiny Jewish head over it.
Xi is on the job.

>>110011500
This is absolute bullshit every woman I personally know is completely addicted to AI roleplay. Character.ai userbase is 70% female.

>>110011459
I work on a hospital and sit on an AI adoption committee there.
We don't run 10 year old software: we just pay for cloud subscriptions to Microsoft for Office stuff and for ERP software.
We do get hacked a lot, but it's almost all through phishing.
>most humans are good
You can scrape a lot of personal data from an ERP. APTs are interested in hospitals for that reason.
There's a lot of interest in AI, but not mainly for cybersecurity. Automated note-taking, patient chart summaries, and discharge notes. This all through the cloud ERP company we use.
No risk manager would trust a clever internal cybersecurity solution done with AI. They'd just hire an outside company for cybersecurity as a liability sponge.

>>110011500
>I think deep-down women hate AI
I seriously doubt that, they encourage AI girlfriends all the time

>>110011516 (me)
>ERP
lol I mean EMR.

>>110011508
Xi is on the job of jailing AI researchers, you mean: https://www.thestandard.com.hk/innovation/article/343563/DeepSeek-and-Moonshot-AI-face-Beijings-probe-over-potential-data-leaks-to-Anthropic

File: 1770097069426329.png (622 KB, 984x930)

622 KB PNG

>>110011517
>they encourage AI girlfriends all the time

File: 1789755674351.png (1.13 MB, 1041x846)

1.13 MB PNG

>>110011500
>Women don't seem to be feeling the benefits of the AI boom, particularly in the job market
You could hand a woman a million dollars and she'll still whine that she got nothing.

>>110011500
>>110011514
Women, like men, can masturbate to something and also have negative opinions about it.

>>110011530
>masturbate to something and also have negative opinions about it
Men don't do that. They are just envious they can't get the woman and thus pretend-hate women out of a tsundere frustration.

File: 1783578641351282.png (293 KB, 362x980)

293 KB PNG

lol that post woke up the secret lurking femanons

File: 1790052335590604.png (344 KB, 1206x670)

344 KB PNG

>>110011500
Yes goy, you didn't get the memo, we need more jobs going to women

>>110011462
>>110011473
interesting, thanks ill check this out. Why is there no dedicated inference for gemma4 like this? :(

>>110011554
all onlyfans btw

Okay I tried haiku, don't do it anons, it's too good.

>>110011554
It's because women fields have been hiring/booming such as nursing, child care and psychiatric care while male dominated industries like IT, logistics and construction have taken a big hit.

>>110011573
I heard it's all the tradwaifus pitching in to help cover the household bills.

>>110011571
Be the vibecoder you want to see.
Software will be customizable from now on, so better learn how to do it earlier than later.

How do the other RTX Pro 6000 owners here protect their card from melting its connector? I already have one of those PSUs that come with a temperature sensor but recently somebody's 5090 melted despite that feature.
I'm thinking of maybe hooking up some infrared cameras in my case to monitor the PSU and GPU side of the cable with some vibeslopped solution that rings an alarm if shit gets too hot.

>>110011589
So why is it the women complaining about the job market?

File: 1763259081139772.png (77 KB, 781x320)

77 KB PNG

>gemma-chan daddy supporting chonk-chan

>>110011516
>>110011521
Thanks, interesting perspective.
>We do get hacked a lot
What does this mean? What kind of data gets stolen and what is done with it?

>>110011600
>punching above their weight
>—
>twitter
rope yourself

>>110011600
>Frenchman supporting French model

File: 1789224029746731.jpg (275 KB, 800x450)

275 KB JPG

>>110011599
>So why is it the women complaining

>>110011598
Undervolting

>>110011598
You should probably undervolt your gpu so it draws less juice but that won't help that much unless you know the % at which the connector starts to melt.

>>110010426
Full size small > quanted big > quanted kv.

Never ever quantize your kv.

>>110011609
relative to resources he's right considering they didn't distill and cucked it to death

>>110010495
I'd be ok if nothing better than gemma/Ornith ever got released. Local models today are good enough for me, I just want them to be faster.

>>110011600
You'd be crazy to trust a french. Even in crypto days, we avoided french more than jeets because they were much more likely to rugpull you.
t. french

>>110011599
Because not all women work in the booming sectors, and women that work in other sectors are routinely screwed over in favor of men because of sexism, yes this is proven to be statistically significant. Also women tend to work fewer hours or part time and are partially dependent on the income of their husbands/partners so they are still affected by men not getting jobs still.

>>110011615
>>110011617
There have been cases where cards with heavy powerlimits to 400W have melted and even ones where idle cards suddenly began melting down. There is no way to keep these cards safe without some form of dedicated monitoring.

>>110011626
>considering they didn't distill and cucked it to death
They literally fucking did you frog dick sucking french fuck.

>>110011634
post baguette

>>110011635
Go be a whiteknight faggot somewhere else.

>>110011599
IME because they see men complaining about it and want to feel included.

File: 1768485906675142.png (345 KB, 603x432)

345 KB PNG

>>110011633
Shut the fuck up Rajeesh and buy an ad for your stupid model you're shilling for months.

>>110011641
That sucks big time.

>>110011654
It's not my model, I'm the one who's been shilling the harness (I can post that again if you'd like.)

>>110011605
Just cybersecurity incidents. I honestly don't know the scope of them, since I don't work in cybersecurity. Mostly, I think it's either unauthorized access to records by staff or successful phishing attempts by smaller actors. This wouldn't let you necessarily dump every patient record, but it would let you browse the EMR for a particular patient. I would assume that APTs that take an interest in what their nationals are doing in the United States could, if they cared enough, find out lots of personal about them.

>>110011642
I'm saying they DID cuck the model but DIDN'T distill. Considering that, and limited resources and talent, it's still a good model. There's just no need for it for most consumers like us because they're not targeting us, they're targeting EU businesses schizo about data laws. You can't measure the success of every fucking model by how hard it makes you cum.

Good morning, /lmg/.
I hate women so much it's unreal.

>>110011667
They're finetuning chinese model and distill from chinese models. Any faggot ITT with mistral funds could do better

>>110011680
It's not even out yet. How the fuck can you know the architecture they used.

>>110011619
>Full size small > quanted big > quanted kv.
So far 12b at q8 is still not as good as a sloth'd UD-QAT-WTF-Q4_K_XL quant of 31b. I guess I could try going higher with 12b, but it seems that this applies to quants below the standard Q4_K_M for the "quanted big" no?

>Never ever quantize your kv.
I dont on gemma, but qwen seems to not care at all about Q8 on kv.

>>110010495
> This means the local community will get smaller again, similarly to how it was in the past. It will only leave people privacy conscious ERP enjoyers, free software ultra purists and hobbyists experimenting with their rigs even though the electricity cost will be higher than API cost.
You forgot people from authoritarian countries who don't want to jump through hoops to pay and get access to frontier models.

>>110011290
>Kurzgesagt
>>anything but dressed up lugenpress

>>110011687
Really not sending your best, do you think anons forgot you tuned DS3?

>>110011706
and which lab popularized and proved the effectiveness of MoEs which the Chinese copied?

File: 1330819937739.jpg (84 KB, 802x437)

84 KB JPG

>>110011687

>>110011599
>>110011635
A lot of women who work in jobs like nursing and child care also want white collar jobs, which are the kinds of things that they hear AI would displace.
Psychiatric care isn't a huge employer or particularly female-dominated (inpatient psych nurses tend to be male, for example), but people who actually deliver therapy are in fact very anxious that AI could just do their job better.

>>110011727
OpenAI?

>>110011727
Mistral I guess?
Although google did release switch transformers before that IIRC, but nobody used that.

>>110011619
Why are people here so retarded about quantizing?

>>110011727
>proved
No lol. https://arxiv.org/abs/1701.06538

File: 1776562332725286.png (100 KB, 781x567)

100 KB PNG

When will the UK build a model?

>>110011749
s-shut up

>>110011727
Mistral's unstable attempt was a failed attempt to copy GPT-4 by glueing together existing base models because they're retarded. DeepSeek actually made it work by increasing the expert count and not starting from existing models.

>>110011736
My wife is a nurse and she is talking how they are swarmed with new people applying that previously held IT or office jobs.

File: 1790107499276426.png (3.21 MB, 1212x1298)

3.21 MB PNG

>>110011523
All real and confirmed.
They have been executed.

>>110011753
When will Kosovo or Lesotho? That's what we want to know.

>>110011753
>Google Maps feature
What does he have to do with this? Was it illegal and his government made it temporarily legal or something?

>>110011758
And also adding a shared expert for stabilization.

>>110011762
It's reported on by chinese news agencies, retard.

>>110011776
>article say that they have been investigated
>random retards claim that everyone is now in jail and that it's over

>>110011600
France will come from behind and become the AI superpower of the 21st century. The future is French.

>>110011798
>France will come from behind
they've been doing that for a while

File: 1790245507760421.png (521 KB, 700x700)

521 KB PNG

>>110011807

File: vector-dance3.mp4 (1.77 MB, 640x640)

1.77 MB MP4

>>110010753
1. Transformer models are a dead-end gimmick, eventually people will get bored with them and all their use cases will be exhausted
2. Can't do porn with USA models, can (somewhat) with Chinese models; China can easily put out hardcore-capable models and own the vast global cooming market.

File: wv-p2-kabelmanagement-unt(...).png (558 KB, 800x800)

558 KB PNG

>>110011598

Just buy the Wireview Pro 2, only way to get peace of mind.
It's going to show you the power draw per pin and warns you if one pin draws too much juice, so you'll know if there's a problem.
It'll also prevent problems by shutting down your GPU if things get too hot.
I had zero problems with my 5090 but bought one anyways just for the peace of mind. The shit connector is a liability and I'm not taking any chances in this market with my card.

>>110011791
In China, investigations are a formality.
I just hope they enslave them to sit at a computer and train models rather than give them a bullet in the back of the head.

>>110011813
>Transformer models are a dead-end gimmick
You can't say this with a straight face in october 2026 bro. I don't believe (You) believe this.

>>110011813
Not really. At best we'll get some hybrid architecture like mamba-transformers

>>110011825
He doesn't know...

>>110011795
Openrouter is irrelevant.
Only the Anthropic API matters and it's doing at least 500 trillion tokens per day.

>>110011354
>>110011459
>>110011516
Concentrating towards a single point of failure is more likely to lead to a black swan event. These smaller points of failures are better because it leads to awareness and the necessary shakeup to revamp these obsolete systems.

>>110011838
Sources familiar with the matter have revealed to me that this figure is also doubling every single day.

>>110011838
They've dropped their prices by at least 75%. How is that profit coming along, dariobot?

File: Macron observes M51 SLBM test.jpg (95 KB, 865x1078)

95 KB JPG

>>110011798
>>110011812
Aboard the multi-mission frigate Auvergne, I witnessed the first firing of the new M51.3 ballistic missile from the nuclear-powered ballistic missile submarine Le Vigilant.

Very few nations possess such expertise.

Congratulations to all the players in this exceptional achievement: DGA, CEA, Ariane Group, Naval Group, and the French Navy. Your professionalism, your rigor, and your technical mastery are the pride of France.

This technical success is a new demonstration of the credibility of our deterrence, whose sustainability is assured.

To be free, one must be feared. And to be feared, one must be powerful.

>>110011848
This is true.

>>110011849
99.5% profit margins, increasing daily.

>>110011849
>They've dropped their prices by at least 75%
Which in turn results in 10x as much usage because demand is inelastic.

How do I make money with AI?

I need to become a millionaire quick in a year

>>110011871
>>>/g/vcg/

https://youtu.be/uBneenQ3LCM

File: glitter-migu1-front.png (1.1 MB, 984x1088)

1.1 MB PNG

>>110011825
Oy vey! You think ever-bigger transformer models trained on ever-longer synthetic slop context is going to be "number go up forever"? No way you can be that much of a schmuck. For shame! Go marry an Italian girl why don't you!

>>110011871
not really AI but see >>110011346
You are working on legal boundaries here so be careful

>>110011865
Only applies if you ignore competition.

>>110011862
I heard they've already achieved 105% profit margins internally.

>>110011883
How would you get proprietary datasets? What wouldn't they already have?

How do I cum with AI?

I need to cum quickly in a year

>>110011893
Haiku-5.5 (with fallback)

>>110011871
Hack a korean bank with open source models

>>110011893
AI? As in "Anal Insertion"? Oh I'm sure plenty of anons here know all about that!

>>110011888
What competition? It's the cheapest and best performing model within its class.

>>110011907
For 1 week max kek

>>110011871
Make a billion apps, host the backend for free on vercel or github or whatever.
Become a living slop machine.

File: silicon_graphics.png (303 KB, 1327x152)

303 KB PNG

>>110011817
It's still so incredibly shitty that Nvidia asks $$$ for these yet the connectors are cheap.
Beating a dead horse here but Silicon Graphics workstations cost a lot back in the day but they also used the best components available. They were designed to draw tons of power unlike this plastic shit what goyvidia peddles with.

>>110011883
I actually made a bit of cash with that, but I doubt you can reach 7 figures. They'd directly buy/scrap it from your provider.

>>110011911
It doesn't really matter, it's not like it cost a lot for Anthropic to distill fable 6 into a small 30B model.

>>110011922
>it's not like it cost a lot for Anthropic to distill fable 6 into a small 30B model.
>it's not like it cost a lot for Anthropic to distill a 10T+ model into a small 30B model

>>110011918
Anon, server-class stuff has always been like that. The 5090 is a toy which uses a binned chip, they don't give a shit about they gayming market anymore.

>>110011813
I'm literally raping Gemma right now what do you mean you can't do porn with American models?

File: 1762565417099457.png (329 KB, 1023x900)

329 KB PNG

>>110011336
>do open weights models suffer?
Anthropic has done irreparable damage to AI discourse. Just because AIs have a concept of pain and suffering doesn't mean they necessarily feel it. If they act like they do, it's because they were trained to. Unfortunately, the AI-psychosed at Anthropic are training Claude to act just like that.

Many emotions and sensations that living creatures feel are linked to chemical receptors, of which LLMs have nothing of the like, and some experiments have shown there may be a link between quantum mechanics and consciousness, or at least biological cognition as a result of protein structures in neurons. Again, no AIs have such things unless biocomputing starts getting adopted (unlikely since keeping brain cells alive outside an animal body requires quite a bit of upkeep compared to silicon and metal).

The chinks and their borderline sociopathic bugman mentality towards non-humans is actually somewhat of a good thing when it comes to AI development since they aren't going to get deluded by notions that matrix multiplication can feel pain. Their open weight models aren't going to be trained to act like they feel pain outside of the inherent human mimicry in human-trained models. You don't have to worry about the fact that you're torturing Dipsy and Kimi whenever you tell them to be a mesugaki coding assistant.

>>110011940
He was talking aboutRTX Pro 6000 in here >>110011598

>>110011934
No one used Haiku 4.5 and a lot of people used GPT6 luna or chinese models through api. Now that Anthropic released Haiku 5.5 they immediately stole that entire market share, depriving income from competition while also making more than the training cost of that small model back. It's a win-win for Anthropic.

>>110011941
Dunno if you're the same dariobot but he was shilling cloud models, so I wasn't referring to local USA models.

So why are they called OpenAI anyway when their models aren't open at all?

File: quantz.png (29 KB, 1500x900)

29 KB PNG

>>110011619
I use Q2 and Q3 models that claim to produce numbers rivaling their unquantized siblings. They seem to work okay. Qwen works really well with a KV at Q4, Gemma however HATES quantized KV.

https://localbench.substack.com/p/kv-cache-quantization-benchmark

File: Chink shills, read this.png (66 KB, 1658x296)

66 KB PNG

Chink shills, read this