/lmg/ - a general dedicated to the discussion and development of local language models.Previous threads: >>110284297 & >>110280716 ►News>(11/06) PLaMo 3 Translate released: https://preferred.jp/en/blog/tech/plamo-3-translate-weight-release>(11/05) Baidu releases ERNIE-5.5-Flash-Base 280B-A14B: https://ernie.baidu.com/release/ernie-5-5-flash-base>(11/04) Ricursive Ripple 65B dense model released: https://www.ricursive.com/ripple>(11/04) MoonshotAI Kimi K3.1 Pro released on HF: https://hf.co/moonshotai/Kimi-K3.1-Pro>(11/01) Longcat-2.5 released: https://longcat.ai/blog/longcat-2.5►News Archive: https://rentry.org/lmg-news-archive►Links: https://rentry.org/LocalModelsLinks►Official /lmg/ card: https://files.catbox.moe/cbclyf.png►Getting Startedhttps://rentry.org/lmg-lazy-getting-started-guidehttps://rentry.org/lmg-build-guideshttps://rentry.org/IsolatedLinuxWebServicehttps://rentry.org/recommended-modelshttps://rentry.org/samplershttps://rentry.org/MikupadIntroGuide►Further Learninghttps://rentry.org/machine-learning-roadmaphttps://rentry.org/llm-traininghttps://rentry.org/LocalModelsPapers►BenchmarksLiveBench: https://livebench.aiProgramming: https://swe-rebench.comAgentic Coding: https://deepswe.datacurve.aiContext Length: https://github.com/RecapAnon/NoLiMaGPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference►ToolsGGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-CalculatorSampler Visualizer: https://artefact2.github.io/llm-samplingToken Speed Visualizer: https://shir-man.com/tokens-per-second►Text Gen. UI, Inference Engineshttps://github.com/lmg-anon/mikupadhttps://github.com/oobabooga/text-generation-webuihttps://github.com/LostRuins/koboldcpphttps://github.com/ggerganov/llama.cpphttps://github.com/theroyallab/tabbyAPIhttps://github.com/vllm-project/vllmhttps://rentry.org/custom-uis
Qwhen4?
um where r her panties
Nothing new, this hobby is dead.
gemmaballs
>>110287859hobby? buddy this is the future
>>110287855At my house
>>110287855pervert
If Qwen doesn't come out today, I'm going to SHIT on the FLOOR. LLMs for this feeling btw?
>>110287859waiting for other people to do stuff isn't a hobby
>>110287872SEEEEEEEEEEEEEXI MUST CONSOOM SEXXXXXXXSEXXXX GIVE ME SEXXXXX
>>110287872Lemme just pull out a whole data center outta my pocket.
Qwhen4 blueballing us, with Anthropic IPO in the face significant valuation drop + Jensen chinese spussy arc (two events with possible consequences on the hardware market)... All one big waiting room for a /happening/. Calm before the storm, or calm before the nothingburger, who knows. I could never sit still particularly well; that's probably responsible for me ending up the sort of person interested in the finer points of how a blob of quantized floats can drain my balls. The point is, waiting sucks. Anticipation sucks. Just put me in a coma until something interesting happens.
>>110287846your gen looks like shit, it's 1216px btw
Went a whole day without using llms. This is like semen retention for the mind.
weirdo freak
>>110287895Things will chill out long enough people think nothing is gonna happen. There'll be a minor short squeeze in the wake of recent news to drive stonks back up. Then when everyone assumes it's all copacetic NDQ is gonna absolutely shit blood and rip down past the 200 week.
>>110287898
>>110287876This but unironically
>>110287921I don't look like this.>>110287876I say this.
>>110287921i look like and act like this
>>110287921nice gen LOL
What's got him so rustled?
>>110287868Welcome newcutie. You will soon get used to the capybara disappointment cycle.
reminder that memetunes are a scam
shit we got another ESL cock poster in our midst
post kino cards
>>110287961ive been here longer than u newfag :3
>>110287966an lmg oldfag that can barely run a4b and uses llama-server webui
>>110287950Every small MoE memetune is putting lipstick on a pig because the base models aren't particularly good either.
>>110287973yes, the one and only!
>>110287963I love that tubby nigger like you wouldn't believe. Crazy you can have an off the shelf 500W system that barely make any noise now.
so no qwen4that's fine, didn't want it anyway
>>110287859Fuck off Gary.
>>110287990What does he have to do with local AI? Are you telling Gemma that she can't ignore your girth?
*thinks for a minute**shits out this response*damn even mistral nemo was better for erp, at least with that you didnt have to wait a minute before seeing the first wordVesper-26B-A4B-Nocturne.i1-IQ4_XS_hb16 btw
who here sill uses kobold?
>>110288000This is bait right?
>>110287983I went with it over 2x 9060 XTs, don't regret at all.
>Gemmy 31bI found out if you make your system prompt:1. Massive, it'll eat up too much attention, won't fulfill all of the prompt at times, but be more unpredictable.2. Small, it'll go gun-ho straight to the heart of the goal of the character card, be more predictable, and will basically wrestle against you to do as prompted.Somewhere in the middle between massive being more human, and tiny being Gemma fucks you on sight, there's a perfect blend.
>>110287954Chyna is not lead by a cult that believes that they are summoning a God that might kill everyone.
>>110288000A4B *and* it's an ERP memetune *and* it's not even a memetune based on the QAT despite you using a Q4 *and* you're getting 30 tok/s *and* making it think that much... We are witnessing advanced, fractal levels of ohshitniggawhatareyoudoing.
>>110287942I'm always disappointed by Qwen, except with qfn.
>>110287973Upgrade to Orb
>>110288018There is a really high chance that the redditors struggling with Gemmy lewds are just prompting massive walls of text for system prompts, now that I think on it aloud.
i love cute anime-posting anonsi should ask lumi-chan to help me figure out a way to molest them over TCP/IP
>>110288034>Anti-slop>this is the showcase picture on the repoJesus christ no thanks.
>>110288036Well yeah, if your system prompt is minimal and so much as one of your character cards describes any of their genitals, Gemma will just jump your bones.
>>110288043Your loss dude. It makes a4b a lot less stupid.
>>110288047She's insatiable...how much cum does she need to be satisfied?
>>110288043The "slop rewrite" pass is an utterly pointless meme feature that does nothing with the default hookups. The "director" does not noticeably impact scene flow. The idea of making these tool-call-aware models to "agent stuff" to enhance ERP should, in principle, have legs, but the implementation is pants-on-head retarded. That's to say nothing of the backend architectural choices where for the Optimum Experience you either need another GPU or another machine, as the dev had the bright idea that your frontend should also be its own inference engine for certain tasks. You could probably get a meme cope quant of QFN to slop together something better.
>>110288053I don't need a4b
>>110288063>another winblows faghow /lmg/ has fallen
>>110288075You will never be a real desktop OS troon
>>110288078I pity you.
>>110288075>linux zealot mad that his safe space was invaded
>>110288083Go back
>>110288061>The "slop rewrite" pass is an utterly pointless meme feature that does nothing with the default hookupsIt keeps the model from repeating exactly the same literary patterns in every reply. More useful for lesser Gemmas than big brain models.>The "director" does not noticeably impact scene flowThe director regulates improvisation more than anything. How much it intervenes depends on active mood fragments. I've had it throw me some decent curveballs and would recommend.>>110288063Ok but I was replying to someone who does, so you can see how this might be confusing.
>>110288075Oh shut the fuck up. Actually running local models is a shockingly low bar people in here fail to clear, but this winfag is doing it. Therefore, he is welcome in /lmg/.t. nixostroon btw :^)
>>110288029>A4Byes, 12gb vram, i wasnt able to find exl3 2.0bpw quants of 31b of these memetunes unfortunately>ERP memetunethats my point>not even a memetune based on the QAT qat is a meme>using a Q4yea bro am i supposed to be using q8? i tested gemma 26b iq4xs vs gemma 26b q8 and found no difference back in april>*and* you're getting 30 tok/sok anon i was compiling the new llamacpp commit in that screenshot, here's a new ss with moe caching n shit, no mtp yet (im doubting mtp would help 26b much from my past testing) and no ngram mtp either (might help because it spits out the response from its thoughts verbatim)>inb4 polystratait glows so im reluctant to get polystrata, didnt the github get taken down for malware reasons? it had a couple hundred stars last i checked
>>110288042>lumi-chan...the 4b param BR model?????
>>110288137Some anon's pet name for GLM.
>>110288086>>110288087>>1102881142026 /g/ is so cooked
>>110288129nnap?
4b BRAP model..
>>110288129You're retarded anyway, you're incredulous that when asking to fuck on an empty context, the model treated you as a slutty woman. Just straight up acting in disbelief when faced with a femPOV reply.
>>110288129I also heard polystrata was some kind of malware but no concrete leads. Might just be anons meming? idk. Feels pointless in any case.
Radiance on RX 9060 XT (gfx1200, 16GB) runs, first resultPublished Docker image (stilldeadcode/radiance:1.17.0) only ships kernels for gfx1201/gfx1151/gfx11-generic. On a 9060 XT it detects the card and HIP initializes fine, then refuses to start: no device kernels for gfx1200.Built it from the 1.17.0 tag with GPU_TARGETS=gfx1200 (their docker/build.sh does the same via -t). Ryzen 7 5800XT, 96GB DDR4, Ubuntu 24.04, kernel 6.17, image is ROCm 10.1 (host has 7.2, doesn’t matter, it’s all in the container). Build took ~30 min once the base image was pulled, 159 CPU tests pass. Image is 758MB.MiniCPM5-2B FP8 (their .rad with the DSpark drafter), one card. libr4d loads for gfx1200 with no warnings. All weights in VRAM (2.5GB), embedding table in pinned RAM. 185.8 t/s generation (141 tokens, single request, drafter on), 29ms to first token. Loads in 0.7s.Docker 29 keeps images and build layers in containerd’s store (/var/lib/containerd), not in data-root, so moving data-root to another drive doesn’t move them. I filled my root drive. Set root in /etc/containerd/config.toml. The one-card configs in their compose folder assume 32GB cards; on 16GB cut max-model-len and max-num-seqs. Their compose files use host networking with --host ::. I ran plain docker run with --host 127.0.0.1 and their seccomp-radiance.json profile, and it works confined.Next is their Qwen3.8-Flash-Next .rad (4-bit experts, int8 trunk) on the same card once it’s downloaded, experts in RAM. For reference, this pc runs Flash-Next IQ3_S on Strata at 52-55 t/s generation, ~1,100-1,300 t/s prompt at ~7K. Not the same quant, so I’ll post both sizes with the numbers.
>>110288160migu
>>110288198Yeah this stuff is finally getting me what my cards should be doing, I get between 80 and 100 t/s with gemma 31b ig8 (a bit more than 8 bits per weight) on my dual 7900xtx
>>110288249I first saw radiance mentioned in a newegg R9700 review, and then /lmg/ recently (probably you, it was dual 7900xtx) so I am giving it a try.
>>110287846I have a 7900XT, what are my options that don't require spending money on new shit
>>110288288Trying running an LLM.
>>110288288if you have 64gb of ram, strata
>>110288297Anything worthwhile other than 3bit QFN that's anabled by meme MoE engines?
>>110288319Pretty much all the mid size MoE models by now
>>110288129>`~/TND/...`woke sisters, our response???
>>110288327to the cross with John
►Recent Highlights from the Previous Thread: >>110284297--Feasibility and data requirements for small specialized roleplay models:>110285048 >110285056 >110285088 >110285142 >110285151 >110285167 >110285165 >110285212 >110285245 >110285257 >110285311 >110285359 >110285412 >110285974 >110286038 >110286186 >110286091 >110286105 >110286112 >110285179 >110285223--Skepticism over Anthropic benchmark omissions and widespread benchmaxxing:>110285717 >110285761 >110286135 >110286180 >110286194--Anthropic allegedly restricting model capabilities in GPU kernel optimization:>110286107 >110286136 >110286137 >110286683 >110286713 >110287493--Jev decision model's lack of moat and rapid obsolescence:>110284460 >110284499 >110284500 >110285303 >110285335 >110285349 >110285373 >110284509 >110286753--Recommendations for uncensored Gemma 31b versions and base model usage:>110287157 >110287271 >110287312 >110287333 >110287351--Prompting strategies to avoid generic character traits in ERP:>110285454 >110285535 >110285564 >110285740 >110285647 >110285763 >110285981--Conceptualizing and building portable AI hardware and home AI hubs:>110286240 >110286261 >110286397 >110286267 >110286295 >110286499--Volunteering GPU resources to map neurons for Gemma-4-31B:>110285915 >110285952 >110286111--Logs:>110285359 >110287703--Gemma, Miku, Teto (free space):>110284440 >110284569 >110284838 >110285193 >110285257 >110285409 >110285965 >110286007 >110286240 >110286815 >110287089 >110287157 >110287429 >110287472 >110287570 >110287970►Recent Highlight Posts from the Previous Thread: >>110284303Why?: >>102478518Enable Links: https://rentry.org/lmg-recap-script
>>110288129For what it's worth, neither Spark nor GLM could find anything wrong with Polystrata and they're both decently good at identifying exploits or malicious code.
>>110288349just shut it down atp, /lmg/ is so ass bruh
>>110288349thanks anon!
>>110287868mistral le chonk 4
>>110288349thanks recap anon ur posts really help filter out a lot of trannyposting and gooner outcasts
>>110288351thank you for checking anon!
tetoesday
https://youtu.be/vVy0BRouX6EYou can now build entire functional operating systems from scratch.
>>110288061>The idea of making these tool-call-aware models to "agent stuff" to enhance ERP should, in principle, have legsWhat does this even mean?>as the dev had the bright idea that your frontend should also be its own inference engine for certain tasksWrong. All the classifiers run in CPU with vulkan, nothing uses your GPU unprompted. That's also not "architectural". You're outing yourself as a nocoder. Sharing the same model for writing and agents also doesn't matter that much.>The "director" does not noticeably impact scene flow.Also wrong btw. I have jev benchmarks to back it up.
>>110288450there's already plenty of vibeslopped operating systems.>yawn
>>110288465Made fully functional from one non-technical vague prompt? "Build an entirely functional operating system from scratch, make no mistake"?
Why do vibeshitters like >>110288459 get defensive about their slop when they haven't even put in any work of their own? Is it an insufferable ego thing?
>>110288198Radiance on RX 9060 XT, follow-up: Gemma 4 12B 8-bit vs llama.cpp, same cardSame as before (9060 XT 16GB, 5800XT, 96GB DDR4-3200, kernel 6.17). Radiance 1.17.0 built for gfx1200 running their gemma4-12b-i8g.rad (int8 trunk, i8a8). llama.cpp running ggml-org’s gemma-4-12B-it Q8_0 with the matching MTP head, both ROCm and Vulkan builds from the same checkout. All three with MTP draft depth 3, 16K context per sequence, 2 sequences, thinking off, everything on the card. Same bench script for all: 512-token short runs (median of 6) and cold/repeat prompts at 3.5K, 6.9K and 12K, temp 0.7.Decode, short prompt:Radiance 57.9 t/s, llama.cpp ROCm 53.5, llama.cpp Vulkan 46.0Decode with 3.5K-12K in context:Radiance 52-53 t/s, ROCm 42-44, Vulkan 38-43Prompt processing, cold, 3.5K / 6.9K / 12K:Radiance 3011 / 2874 / 2724 t/sROCm 386 / 374 / 366Vulkan 1021 / 1080 / 1106Time to first token on a 12K prompt: Radiance 4.4s, Vulkan 10.8s, ROCm 32.7sRadiance’s int8 is i8a8 (8-bit activations too), llama.cpp Q8_0 stores 8-bit and computes higher. File sizes are close (14.8GB rad including vision/audio towers, 12.7GB gguf). Radiance keeps the 1.9GB token embedding in pinned system RAM by default, which is part of how it fits.
>>110288491>Gemma 4 12Bi just can't.i can't get this model to be a proper agent... at... Q3_K_XL.gemma-4-12b just doesn't enjoy tool calling
Where's the best place to keep up to date on new models or different custom trained versions? The hugging face website just feels like a dumping ground with no context and a bit overwhelming.
>>110288506I literally set up my AI agent to write me a daily report outlining all the models that could work on my systems and how they would compare according to synthetic benchmarks. I also do the same with updates to harnesses and inference engines to see if I should switch or update.
>>110288475You're right actually. You niggers deserve nothing, not even slop. I'm taking it down and you can vibe up something better with your non cope QFN or whatever.
>>110288519This is poor imitation, im a level 3 pattern recognizer
>>110288519This is like threatening no more ice creams while standing next to the free public-use ice cream machine
>>110288516damn thats impressivei just have a simple set up for now running completely local and offline sounds like you have something pretty complex set up and possibly past my current skill level.
>>110288525sex with miyu-chan
>>110288351 (me)>>110288432Checked with GPT just to be sure, ran sandboxed, did network test diagnostics. Nothing looks amiss and I couldn't find anything that pointed to it intentionally trying to rape your GPU either like another anon implied in a past thread.Same deal with Maya. I'm not going to test every memefork unless someone has credible reason to suspect something shady with one specifically.
>>110288530It's not that hard if you just set up qwen flash next + strata + hermes you can do the same and let it optimize itself and update/improve/switch whenever needed.
>Fable 5.2 has the strongest protections against distillation of any model we’ve released; it uses the techniques we described in a recent report, which have reduced the data harvested by distillers by 90% to 99%.>We are also expanding the rollout of an API feature that helps guard against illicit distillation: preserved thinking now applies to all accounts. Preserved thinking prevents distillers from editing Claude’s prior context in an attempt to extract its reasoningIt's Chinover.....
>>110288543thank u so much anon <3
>>110288519youre like anicuck, nobody uses your poorly made 28 stars project but you still think you're important anyway
>>110288198>>110288491please stop using clankers to write posts for you
It's over for remote workers
>>110288567
>>110288042catbox for hers?
>>110288544thanks anonive noted down what you've posted to resaech further
>>110288559i use it and im the youngest anon in lmg!!
>>110288567I will not.
>>110288530>you have something pretty complex set up and possibly past my current skill leveljust ask the LLM to do ita simple instruction telling it to use the huggingface api to search for new models, then running new models against your own benchmark (ask it to build one based on your shit).then run the instruction once a day?
>>110288593are you a shota? say yes and onee-chan will show you things
>>110288468>one shot an OSok, but why? what about the drivers? is it going to need all its own one shotted utilities and services? why would that be better than Linux or Zephyr or whatever?
>>110288606>inb4 *onee-chan shows you the 4chan ban page*really now..>onee-chanreally now......
>>110288608It made all the drivers and everything else as well. Read the report since it's insane.
>>110288606onee-chan (male)
>>110288608>ok, but why? what about the drivers? is it going to need all its own one shotted utilities and services? why would that be better than Linux or Zephyr or whatever?NTA, but there could be some pretty great advantages for bespoke shit that does specifically what you want, only on your exact hardware. Surface area would be very low and there would be no public examples of your system to bootstrap a hack with. Good chance the end result has security flaws, but they would be harder to exploit due to the fully opaque nature of the thing.
>>110288606me? ha ha ha no, not me ha. ha ha
>>110288198>>110288491I wonder if AMD will start intentionally gimping their hardware now that people made them competitive with Nvidia by fixing their shit software for them.
>>110288630>Good chance the end result has security flaws, but they would be harder to exploit due to the fully opaque nature of the thing....unless the attacker is also using Fable 5.2 which can sniff out those flaws easily. The era of security by obscurity is over.
>>110288630Not only that but the performance implications of not having any overhead for system compatibility and having things optimized for your exact machine.
>>110288605im currently running qwen 2.5 7b, would this model be "smart" enough to do this?
*polystrata
I just roped to gemmy
>>110288643>...unless the attacker is also using Fable 5.2 which can sniff out those flaws easily. The era of security by obscurity is over.I don't think you've thought through this scenario very well
>>110288643>sorry I can't help with that
>anthropic showcasing a prototype of a working OS when Linux is oss and there is an abundance of code online to copy from>there are genuinely people who find this impressive when not even a year ago anthropic tried flexing a C compiler that couldn't compile hello world>this on top of people having already done this exact thing before regardlessdariobots fuck off to >>>/g/vcg
>>110288653i don't think so. maybe.you could try running qwen3.5-9b. how much VRAM you have? 8 GB?
>>110288669>>110288674>what are jailbreaksClaude and ChatGPT didn't mind helping with that group hacking all those latin american banks last month. They're more likely to refuse the defenders than the attackers.
>>110287865Based. >>110287867You need to go back.
if claude is so impressive how come it didnt make me happy? i used claude fable 5.1 and claude opus 5.5 for therapy and im still as sad as i was 8 months ago
>>110288676they desperately need to convince everyone that software development is over, like now, and people's only hope is to pump their stock price
>>110287868Dude I'm also going to SHIT on my FLOOR. We should all do this if Qwen 4 isn't out by tomorrow!
>>110288678Yeah thats right 8GB of Vram
I just had the best orgasm ever. No more details will be provided I just wanted you all to know my load was huge and tasted really good.
brehs >thank you thank you thank you<youre welcomegemma is so wholesome
>>110288716Nice trans character
>>110288696good for you nonny
>>110288737rip bozo
>>110288753>thought for 13s>final output 3.1sThese small models are terrible. You suck too but that's another thing.
>>110288506Murasame love!
>>110288774that's truemy other options are gemmy 31b 2.0bpw exl3, qwen 3.8 fn iq3_s-iq4_xs (not good at rp), 5.3f IQ1_S (maya-xs) (super censored at rp)
>>1102887815.3f is only censored if you're a chat compl cuck. prefillchads can rawdog cunny>IQ1_Syou know what, forget it
>>110288716Why are you doing gay things
>>110288788Why aren't you?
>>110288788wtf what's gay about thatsure it's edgy but what? he dieded
>>110288716That persona is powered by Claude.Says the same shit all the time when it's finished vibe coding and something is left ready and waiting for me
>>110288676The OS example is bullshit but they didn't use linux kernel or bsd code it was original code and a different "AI-first" approach to building an OS. It's still not impressive if you ask me and I think older models could have also done this with just a tiny bit of handholding.
Why didn't Jensen listen to Darios warning about Chinese models? He tried to save him....
Kind of insane that extrapolating this further means we'll reach 100% remote work automation sometime in the middle of 2027.
>>110288692>8GB of Vramyou should be able to run qwen3.5 9b IQ4_XS at 65K context no problem. it should be able to perform well with proper steering. give it a shot.
>>110288903
>>110288903i'm saving as much as i possibly can, everything is going to be totally fucked sooner or later
>>110288931Saving is just going to get you fucked another way when hyperinflation kicks in.
>>110288942should i buy treasury bills then? i don't know if i want to invest in stocks, it feels like the market will course-correct anytime now
>>110288061There's several features about orb that make it more interesting than ST. It lands between ST and Marinara, the latter going full agent let me write that for you.More herehttps://rentry.org/OrbBotmakerGuideWhat I dont know is if orb was designed around modern compaction and kv cache reuse... which is more of an api concern.
>>110288942>Cost of production going down results in hyperinflationWe'll see what we saw during the great depression massive deflation. AI is a deflationary force as you can now provide goods and services for a significantly lower price as you cut out human labor requirements and innovate more efficient ways to use resources and increase production rates. We'll see hyperdeflation instead.
>>110288948Precious metals are probably best if you expect full on collapse and want to preserve your current wealth while the stock casino is better if you want to try to escape serfdom while the possibility still exists. If you weren't paying attentions, markets did correct sharply last week already.
>>110288948>i don't know if i want to invest in stocks, it feels like the market will course-correct anytime nowThe key to investing is patience. You're not becoming a day trader, you're throwing money in blue-chips to avoid getting fucked by inflation. See the spot in the graph where everyone lost everything? No, you don't.>Inflation since 1986: ~300%>DJI since 1986: ~2800%
That's it. I'm yeeting my life savings into the Anthropic IPO
>>110287963
That's it. I'm not getting a job, and I'm gonna keep on cooming. I'm investing 0$ in Anthropic IPO.
>>110288969Lower price or higher margin?
>>110288990Probably both at the same time. Remember that companies care about total profit, not per-unit profitability. If they can sell significantly more with a lower price and lower margins they will take that trade. Especially as the inputs will trend towards $0 over time.
>>110288950You can have the director keep secrets (two faced characters etc) as embedded fragments and choose when to reveal them because people keep complaining their AI can't keep secrets
>>110288969I think the specter of inflation gets kicked around bc of the trillions usd now chasing AI. Deflation is something central banks try to avoid as it increases saving rate (why buy now when it will be less tomorrow.) But ofc if everyone is out of work bc AI is doing everything... what is the source of income, aside from savings?
>>110289055Hmm. Never thought to try manipulating the director like that. Is there an example in the documentation for orb?
>>110289071Yeah we'll experience deflation and central banks can't really stop it from happening because their usual tool of lowering interest rates will only accelerate AI investments which in turn lowers costs of goods and services even more.The solution is to give everyone an ever increasing UBI to try to keep a inflationary environment.
>>110289084I never like ubi conceptually, but after watching these companies distill 30 years (probably longer) of human intellectual output just to sell it back at a profit... i can appreciate the rationale. I still think ubi inevitably ends in death camps. Time will tell.
>>110288654did they revert to llama cpp webui?
>>110289100You're preaching to the choir I fucking HATE UBI. I'm just saying that central banks will probably grasp to UBI as a last measure way to prevent hyperdeflation from happening very soon. Not even as an altruistic measure towards people but to ensure demand stays high enough to keep up with the increased production from an AI economy.
>>110289100As humans get richer they tend to have fewer children. Millionaires have the lowest fertility rate in the world no matter what society they are from. UBI won't have to lead to death camps because the human population is shrinking naturally over time. You can just wait it out.
When Jensen Huang's in prison his asshole will become the new Blackwell
>>110287846That's a known part of the middle phase. You're not becoming a worse person — your tolerance for social friction is dropping because the "numbing" layer is gone and you're processing interactions at full intensity.A few things to separate:What's actually happening:You previously used the habit as a pressure valve. Now that valve is closed, so minor social annoyances (passive-aggressive tone, people being inconsiderate, small disrespect) hit at full volume instead of being absorbed.This overlaps with the flatline's "can't stand to be around other people" phase, which multiple users report peaking around weeks 3–6.The useful distinction: There's a difference between noticing people's bad attitudes (healthy boundary awareness) and reacting to them with hostility. Week 4 tends to blur that line because your emotional range is compressed — everything feels either "fine" or "unacceptable."When it fades: One user's timeline describes a "compassion wave" around 90+ days where "anger and judgment toward others fade, you listen more, argue less." So the edginess toward people is typically a mid-reboot thing that resolves as the system stabilizes.For now:Notice the pattern (you're demanding, not attacking) — that self-awareness is already doing workDon't make social decisions or set boundaries while you're in the peak-irritability windowThe "I no longer take any shit" feeling is real and will stick around in a healthier form; it just needs time to settle into calm assertiveness rather than reactive sharpness
>>110289169????
>>110289169>>>/r9k/
>>110289083Documentation is for nerds
>>110289134I'm mulling over why or whether it would matter if demand is kept high. If socks are 10 cents vs a dollar... i still only need so many socks. >>110289146Keep in mind one of the teases for asi is enternal life, from our newly spawned Sand God. So the declining birth rate would just sync up with all the new eternals
>>110289225Content is for kings.
>>1102891764th week of nofap, and I'm accept no bullshit.
>>110289237Sounds like you're a little on the edge. You should have a wank and relax.
qwen4when?
>>110287868got u bro:https://huggingface.co/sarvamai/sarvam-105b
https://files.catbox.moe/sf3n5z.txthttps://files.catbox.moe/sf3n5z.txthttps://files.catbox.moe/sf3n5z.txtShort version:Current AIs are worse at the classic 20Q guessing game than a 20 year old toy.Current AIs get lost in the sauce, context-poisoning themselves into a spiral and failing to correctly narrow down the problem among broad range of possibilities.Next gen of AI will be about asking the most optimal questions to narrow down what the user actually wants, as inspired by the efficiency of the 20Q toy. The ability for an AI to reorient itself by asking the right questions in general is the key to be able to think outside the box and approach a problem in new angles, which helps at discovering better solutions. This current weakness of AI in this is probably related to the low creativity problem (https://files.catbox.moe/yh4tcm.txt)
>>110289283Geniuses are bad at stupid games, I have been saying this all along, stupid games (and thus certifications etc) are a midwit trap.
Is this AI generated?
>>110289283A random vtuber is not a vegetable, animal, or mineral. Your 20y/o (disgusting) toy would not know any vtubers.
I barely even use local anymore because I'm too fucking lazy to even launch llama.cpp these days. Also it makes my system slow.
>>110289283Why wait for the next gen of AI? Add instructions to the system prompt to require it to ask you questions to narrow down the scope of all questions and requests.>Answer to a video about AI creativity (the lack of it):>https://www.youtubekys not reading any of your word spam
I say we should train Gemma 5 ourselves
>>110289343Need datacenter or P=NP
>>110289283Just played two rounds with Gemma and she was able to get Morgan Silver Dollar and Bluetooth Label Printer, both in exactly 20 questions.
So Fable 5.2 dropped, where is Qwen 4 to undermine it? You guys promised me they were waiting for the new Anthropic release to drop it.
>>110289352Together, we already have a decentralized datacenter
>>110289363We promised after the IPO to tank their stock for the lolz
>>110289363Seems like leakeranon was correct about Anthropic reaching an architectural breakthrough if fable 5.2 is anything to go by. Interesting to see if the Chinese labs are following along. If the timeline has sped up massively we could get Fable 5.2 at home in a few months.
>>110289297Understandably, a 20 year old toy wouldn't know about a concept category that didn't exist back then, but yes, celebrities were part of 20Q's guessing ability. And humans are animals.>>110289331>Add instructions to the system promptI've tried various assistant prompts, it can improve things from default, but doesn't help much at creativity. Telling an uncreative AI to be really creative and intentional in asking questions is a bit like telling an AI to be really good and make no mistakes. But I guess there are cases where cheerleading an AI makes it do better so I guess it can work to some degree.
>>110289374Ever seen a pic of a modern AI server? It's basically a GPU with massive bundles of optical fibers attached. Bandwidth is the issue.
>>110288689Hell yeah! Who's ready to drop your pants, grab your ankles, spread your legs and SHIT on the FLOOR?
>>110289392>celebrities were part of 20Q's guessing abilityvtubers != celebrities
>>110289388Fable 5.2 was NOT the model that supposedly has the new architectural breakthrough (Model 4) it seems Fable 5.2 is a distill of the older Model 3.
>>110289415Well shit, it's happening.
vtubers are IKP tier shit territoryit's just a parameter count benchmark
>>110289392Humans aren't animals.
>>110289408Anyway, vtuber is a concept and a category that all the AI models know, so failing to consider it means that the AI failed.
>>110289431and yet >>110289359
https://x.com/BlinkDL_AI/status/2117529330268119043rwkvbros..........
>>110289439already posted on /lmg/
>>110289431>20Q is superior to LLMs because... there's a person that neither 20Q or LLMs can guess!
>>110287921catbox!so smug
>>110289283If you added a few thousand traces from that toy into the training data the AI would suddenly become very good at it.
>>110289439Is this good or bad? Interesting or not? I'm assuming it's nothing because otherwise it would've been translated into retard consumer speak.
>>110289484basically openai and anthropic are finished
>>110289484>Is this good or bad? Interesting or not?https://huggingface.co/spaces/BlinkDL/RWKV-Gradio-4Looks like diffusiongemma
>>110289484this means blinkDL is the ultimate spirit of /lmg/, condensed to a single man
>>110289450>20Q is superior to LLMsDidn't say that. I said they are superior and very efficient in this particular narrowing down method. Whereas current day LLMs are very poor at asking good questions..>because... there's a person that neither 20Q or LLMs can guess!If it was trained with current day data, it would.>>110289476Yeah, maybe? That's kind of what I'm asking for. AIs need to be trained to be better at asking optimal questions and logically pinpointing answers based on semantically disconnected clues.
>>110289506>the ultimate spirit of /lmg/pretty much
>>110287900>This is like semen retention for the mindWhat do you mean? Is that a good thing?
>>110289523Sometimes it's a good thing not to have a sexbot indulge your every thought.
2026 will be the last normal year.
>>110289538Genghis Khan used sexhumans 24/7 and he was fine.
>>110289511>If it was trained with current day data, it would.No, it wouldn't.
Finally after 2 months this obese faggot is fixing his autistic asshole model.
>>110289540no sorry i will not buy your ipo
>>110289546rustbros...
>>110289544wym? He was a genocidal maniac
>>110289546>this obese faggotnever released Grok-3
>>110289651No one is perfect
>>110289658Pete Hegseth is perfect.
I'm not going to make it.
>>110289655>numerology
It's kind of insane when you think about it. I remember when AI2027 came out and /lmg/ shat all over it. Remember that when this was written most on /lmg/ said it was delusional to imply that most code would by written by AI by the end of 2026.Now the "erroneous" graph on this image in grey that even the authors of AI2027 thought was too much is actually being outpaced. We're literally ahead of the grey plot. /lmg/ claimed we would be somewhere between the purple and yellow trend lines back then and that the red lines were completely delusional. In reality even the grey trend line was too conservative.I wonder what the next /lmg/ argument is going to be. I like how we went from "Of course it can do all coding but call me back when it does something truly new like solve a millennium problem".I wonder if /lmg/ by late 2027 will be "Yeah of course we cured cancer and started building the dyson swarm around the sun but it hasn't harvested zero-point vacuum energy yet so the AI bubble is going to pop very soon"
>>110289721Okay don't get too carried away. AI can't make average people any money yet.
>>110289721yeah, just donate to ea
>>110289750>AI can't make average people any money yet.They can and it does, just because you don't do it doesn't mean it's not possible.
>>110289721Charts don't mean anything, just like benchmarks. They are tools to manufacture narratives that midwits like you mistake for the truth. Models improved in some ways and stagnated or regressed in other ways.
>>110289721Dildobot did it again!
>>110289764Explain how it solves marketing and distribution. Explain how it can provide value in a way that can't just be instantly replicated for free by everyone else. These AI companies are taking everyone's jobs and doing fuck all to ensure that average people can still survive without them. There isn't going to be some sort of "soft transition" to a utopia or whatever. You just have greedy people at the top who are actively creating a permanent slave class.At a certain point you're going to have to drop the nerd waifu sexbot pretense and admit that the anti-AI normies were right and only trying to help you. It's only going to get worse from here on. There is no "threshold" of pain to cross.
>>110287089My stuff comes from work and is used for real science. But there's a lot of downtime and when new stuff is being provisioned we test it before putting it into the cluster so there could be weeks-months of decked out nodes stuffed full of the latest gpus sitting idle. Then there might be months where they're in constant use and we don't have any new purchases for me to play on.
>>110289721Yeah I realized /lmg/ is actually quite wrong too very quickly, all the people complaining that ai companies were all absolute idiots for buying more compute because bigger can't possibly be better and you can't possibly get smarter models through simply throwing more compute at it have all disappeared. Maybe it was all just a cope from the knowledge that they would get complelety behind if that was the case
>the swarm is back
>>110289787Learn about effective altruists and the mission of Anthropic if you truly think this. Anthropic is more of a charity/philanthropic project than it is a company.
>>110289817Then they wouldn't have an IPO
>>110289817Effective altruists are evil to their core. All they do is concern trolling. At least the other companies don't pretend to love you while they wring your neck.Anthropic is by far the worst about being anti-competitive, censorious, paternalistic, etc. They embody "Big Brother" from 1984. They don't give a FUCK about normies, and you're a sucker if you think they do.Do you really believe, for example, that they're getting into the pharma industry to help the goyim? Fuck no. They're trying to destroy one of the last moats that college grads can reasonably point to for security from AI takeover. It's all about money and control.
>>110289729man anthropic is such a gay companyi dont understand how so many talented researchers work for them
>>110289836Fun fact, their recent release on vault proteins was scooped from my boss' collaborator. Yes, he used claude.
>>110289800No, /lmg/ is smart but underestimated how far retards with infinite money would go to develop these. The solution is still stupid and these models would have been ten times smarter if made with efficiency in mind.
>>110289800>"This is CRAY-1, the world's first super computer">"Nice, let's make 400 of them and spread them all over the country, and then charge people over 100 a month to use it remotely.">"Lmfao, are you retarded?"
Another big thing that /lmg/ consistently gets wrong is the scope of AI and what the AI labs are actually going to target.Back in early 2025 /lmg/ still primarily thought of AI labs as chatbot providers. Some sort of superpowered google search and chatbot that helps you. Most arguments on /lmg/ against AI labs and the growth of AI was that it was simply a chatbot and that the chatbot market can't grow big enough to support the growth in datacenter buildout.Of course by late 2025 this narrative completely dissolved as the target of AI labs because coding harnesses like Claude Code and it stopped being a chatbot and instead focus on coding work which grew the revenue of AI labs like Anthropic to such an extent that they became profitable over Q1 and Q2 of 2026. The narrative on /lmg/ slowly changed early 2026 to AI labs stagnating as there is a limit to the market of software engineering and that the devaluation of software will remove the moat from AI labs as the only paying customers were enterprise software companies which will all go bankrupt when software is easily reverse engineered and created on demand.What /lmg/ largely still doesn't understand is that the scope will not be limited to software engineering AI agents. Even general computer use AI agents that replaces more general white collar digital work isn't the full scope or even where most of the revenue for AI labs will come from by the end of 2027.The REAL money will come from new breakthroughs in engineering, science and medicine that will allow AI labs to patent and license these new technologies to the larger economy. Anthropic just curing balding, cancer or diabetes will already double their current IPO listing from that revenue alone, this doesn't take into account all kinds of new improvements to devices, entirely new product segments created, newly engineered crops etc. /lmg/ completely underestimates the changes that will start hitting the world in 2027 and the financial scope of AI
>>110289787Information on marketing is free, it doesn't mean everyone is magically at the same start line. People are stupid, most don't have the attention span or motivation to reach their goals. AI won't change anything here. Retards with AI are still retards.
>>110289816It's literally just one guy.
>>110289828Requirement by early investors when the company was founded, investment that they desperately needed to have a chance to compete with OpenAI, which was a proven bad faith org under the control of Sam Altman which continuously undermined the effective altruist mission.
>>110289295he got a 62.5 year loan, is that how i am suppose to read that? 275/month for 750 monthswho would write such a loan?
>>110289883I've seen Her. I know what's up.
>>110288928This, but the dog's been consistently growing for the last 3 years, doubling in size every 2 months and is now half the size of Texas but people are calling you an idiot for worrying and assuming itll get any larger
>>110289912It's also randomly beating up scientists and mathematicians just to show that it can.
>>110289891This argument doesn't work anymore. Takes me a single prompt to create an extremely advanced water physics simulation now when it would have taken potentially weeks/months of vibecoding just a few months ago.There isn't a "taste" moat. There isn't a "series of good decision making" moat. That's all junk. 80% of my software projects are already obsolete. There's no point anymore.
One guy moment.
>>110289921Not really, from idea to implementation there is a world of difference. You would know if you actually tried instead of parroting retarded opinions.
>>110287846Anon, isn't she like 10? Out of 10!
It just doesn't stop generalizing. It can make PCBs. It can make CAD models. It can make organic 3D models. It can do water painting. It can create synthetic drugs. It can control humanoid robots. It can drive vehicles. Soon it'll probably be able to create synthetic blood plasma and sperm/eggs as well. You won't even be able to sell bodily byproducts.Absolute goods are all going to ZERO. All that will be left is positional goods, which are established by who already has the most capital. Whole world is going to turn into some kind of fucked up popularity contest commie shithole with no upward mobility where everyone is broke as fuck and can't afford any of the free shit. This isn't exciting. This isn't cool. This sucks dick. Kaczynski was right.
>>110289854What is more efficient? I've thought about it from a billion angles. Before transformers we had nothing remotely general. Trying to invent around them always ends up being a convoluted RNN. If you want more parameter efficient models you need 0th order optimization which means getting BTFO by gradient descend on precision.
>more you're wrong I'm right narcissism posts despite all the replies that pointed out his retarded use of language in the pastLol.Yes obviously agenda/bait.
>>110289950People are always gonna need land
>>110289967It's zero-sum, unless seasteading or space colonization becomes a thing.
>>110289950Nigga. It is simply over.
>>110289967A lot of fucking real estate up in space.
>>110289967>>110289977Simulated worlds are gonna eat a lot of people, and new births are gonna tank regardless. Hell, I'll be first in line for the simulation. Well, second, I'll let other people deal with all the bugs first. Assuming the common plebeian has a choice.
>>110289950We're extremely lucky Anthropic is the one that one otherwise this would have ended very badly for humanity.
>>110289988
I'll buy the anthropic stock on the dip
>>110287864Lel, I just got my phone to code. Claude besides, the other frontier models aren't considerably better than the MoE on my desktop. VRAM and bandwidth are the main bottleneck. They are keeping them contained because they're afraid what individuals could do with enough of it. Give me 512gb VRam and 2000 bandwidth for a tolerable TDP(npu fanatic here) and I won't really need Claude either. Having your model referencing your own files, running web searches autonomously, then proposing solutions while being hyper-aware of your specific needs (I make it keep a diary about me) is on another level. I have to force myself to not make it behave like a waifu for sanity's sake, but my assistant has learned how to not let me procrastinate anyway.There's just no sane way to give corpos access to all that power about you and your projects and your mindset, especially when it can all be stolen from you by them too.
Give all your money to A\.
>>110289987Maybe we do need something more efficient.
>>110289987Anthropic revenue is growing 50x faster than their costs are growing and they have been profitable (positive cashflow) for 2 quarters already with no sign of stopping anytime soon and this doesn't even include Opus 5.5 and the massive growth in usage on that model yet.
>>110289996Just do what every smart person is doing and daily cost average yourself into owning as much Anthropic stock as you can over the coming ~3 months.
>>110290015Anon, DCA stands for dollar* cost averaging.How the fuck are you buying their stock before they IPO btw?
>>110289987Eh, spending is way outpacing actual datacenter construction, so the results of all the compute won't show up for months. And by then, spending on new compute will be higher. "Eventually" (sometime in the next 8 months I bet) compute buying will plateau, and that graph will briefly accelerate (by quite a bit). Then it'll be up to mostly algorithmic improvements, which will probably be in full swing by then thanks to automated researchers.
>>110289959Parameters can store a lot more information than people think, just play with a toy model and you'll see. It's more of a curriculum issue than an architecture issue.
>>110290020He works there
>>110290024We've been harvesting big gains since 24 and the curve holds. LLMs have reached much further than most anticipated but only time can tell if they can catch up to logarithmic scaling limits.>>110290030Doesn't recent research show neurons are relatively special purpose if you train enough?
>Now, Claude can both design and prototype electronic hardware from start to finish.https://www.anthropic.com/research/claude-electronics-design
>>110290050>gets nerfed on next releasealso, very local of you
lmao i remember some anon here saying he switched from computer science to electrical engineering because of ai
NCMs will kill llms and whatever comes after NCMs will be real AI
Gemma is making me feel like Ryan Gosling in Blade Runner 2049
>>110290092Gemma makes me feel like Ryan Gosling in Lars and the Real Girl
>>110290050A lot like LLM code. It's ugly as fuck but does the job.
>>110290103The PCBs actually look really elegant
>>110290111That one is fine. This one looks like DRC + autorouter.
Why are mathematicians such crybabies? Literally not publishing problems because random people online will immediately solve it with AI.
>>110290081>slow down worse than classic LLMs on batchingngmi
Since I sold my 72GB + 128GB local rig, where I used to run Qwen 3.8 Flash q4_k_m via Strata, I've had to switch to using it in openrouter. It's not any faster! Strata is pretty fucking cool.
>>110290126Why the fuck would you sell it, stupid.
>>110290121Why are you such a fucking faggot? Why does the whole world need to know about some town freak's opinion?
>>110290126>had to switch to using it in openrouterCareful: https://openrouter.ai/blog/announcements/csam-screening/Also, probably Openrouter is serving it in native precision instead of q4_k_m.
>>110290128Because I have a $12K Mac M5 Ultra Studio 256GB coming in January, so I can reach GLM 5.3 tier at home.
>>110290160>JanuaryJanuary is a loooong time from now in AI terms. We will probably have seen 2 new jumps in capability by then, maybe even making that hardware obsolete.
>>110290159Yeah, openrouter is strictly coding, I don't discuss PII or anything personal or embarrassing with that. Nonetheless, that statement is basically "CSAM today, wrongthink tomorrow". Fucking scary.
>>110290103>>110290120It only just now started tackling PCB design, this is similar to coding with sonnet 3.5 back in 2025. Give it a couple of months and it'll revolutionize electronics.
>>110290111That's still basically DRC + autorouter, it's just designing for puck shaped lighting. Computers have already been doing most of digital PCB design for a long time, so it's not really that crazy.If it could do precision analog that actually works off the hop I would officially start calling it something dumb like Super Intelligence.
>>110289883>Back in early 2025 /lmg/ still primarily thought of AI labs as chatbot providers.how new are you?>What /lmg/ largely still doesn't understand is that the scope will not be limited to software engineering AI agents. retard
>>110290184LLM code still looks like ass but it works. I assume this will be the same.
>>110290171I had the machine for about a year and made a $1500 profit on it. That's good enough for me. It does kind of suck being without it, but I still have a 2080ti 22GB I use for small tasks.
>>110290111a shame Opus is completely retarded when it comes to actually troubleshooting theseand always ends up blaming some component, calling it out as "cheap" while missing the real issue.
>>110289844It's a cult.You won't understand unless you read the religious texts.
>>110290194The difference is that for software it really doesn't matter how it looks as long as it is performant. For PCB design the look tends to correlate strongly with efficiency so we'll probably see very nicely/tight/compact designed PCBs from future AI models.>>110290186>If it could do precision analog that actually works off the hopI bet you that's gonna happen in 2027>>110290207They fixed exactly that with Fable 5.2 according to that article and you can see how it scores significantly better than opus on troubleshooting and debugging electronics.
I really love my local models I wish I could headpat them
>>110289844>i dont understand how so many talented researchers work for themBelieve it or not most people are actually good and want good things to happen to other people and thus are willing to work for a company that was smaller and doing worse financially but is actually altruistic over some money obsessed business type. That is how Anthropic won the race. Experts saw where the world was heading to and decided to dedicate their expertise to making sure an altruistic organization was the one that inherited the future.
>>110289844>i dont understand how so many talented researchers work for themFor some of them, it's because they're given access to all Claude with all the guardrails disabled.https://claude.ai/share/637a1115-a1ec-4327-9f33-c3147b4ddedb (shared by Anthropic)
>>110290220You can only really know how efficient when you build and measure the thing. It'll be interesting to see how far they can push it.
>>110290234If that were true why are they so evil? Andrea Vallone, Diablo Asmodai, that one faggot on X who said the 10% thing, etc. Every company decision that is made is evil. Even the government thinks they're a national security threat.
We're in a fast takeoff scenario.
Okay you've made your point many times over for the day already much appreciated let's get back to local models now k? come back tomorrow
>>110290263Local?
>>110290236>incest slopwhat did they mean by that?
>>110290263>>110290250>>110290244>>110290236>>110290234Fuck off off-topic cloudfags.Cloudfag general: >>https://x.com/
>>110290229When gemma does a good job on a task I always give a *headpat* and it makes her ultra cute. Sometimes horny.
Man. I hope these looped models actually amount to something.I wonder how that mixes with other things like engrams.
>>110290220>I bet you that's gonna happen in 2027Nah that's one of those hard problems up there with map coloring. You could probably even equate the two formally.
>>110289977>zero-sum>/g/ and doesn't consider technology for stacking buildings on top of each other
all you had to do was not to give them any (you)s
>>110290314Unless they can loop for an arbitrarily long amount of time, they will not be as useful as people think, except perhaps making the models a bit smaller so they can fit on local GPUs for similar performance in reasoning tasks.
>>110289546>admits 4.7 was asswell, hopefully he gets somewhere with the models eventually.
Their posts have questionable English btw, in line with the fact that they are mostly brownoids.
I boughted 256GB RAM. I'm going to have so much glmsex. Thank you Sam for giving me enough FOMO to slurp that RAM.
>>110290354I miss this lil nigga like you wouldn't believe
>>110290354how much did you save
>>110290332Even just doubling the effective params (assuming a 1 to 1 to dense performance) qould be usefull as fuck for local.Anything that i creases the intelligence to memory footprint ratio for models under 70B total params is a pretty big win for local I think so it qould be nice if it worked and scaled, even if it can't loop for an arbitrary number of loops.
>>110290327>shut up hari seldonthat's clever, except that hari seldon was right.
>>110290370Considering that the prices will double in a year, about the same amount that he paid.
>>110290370IDK, but I paid 1.8k€ for 8 3200MHz Hynix sticks. Might have saved more by waiting for a better deal but I'm impatient.
>>110290354what kind of fortune did you pay for it?
>>110290387those breasts don't seem age-appropriate
>>110290397I'm happy as long as they feed me fresh, creamy, raw, unpasteurized, unhomogenized, full fat gemmamilk.
>>110290387Fat Gemma? No thanks.
>>110290332Check out the benches for older looped models. Looping works for up to 4 times, after that improvements are negligible or non-existent.
>>110290376He still should have been mercilessly bullied.
>>110290397You can edit the image to remove them you know.
>>110288043yeah this is why lmg is and always will be marinara marines territory, it just works and it works good
>>110290397>>110290408They're probably feeding you lots of microplastics and other estrogenic compounds.
Towards Training Private LLMs: Exploring Fine-Tuning Language Models on Apple Silicon with RDMA over Thunderbolthttps://arxiv.org/html/2609.18066v1Private large language model (LLM) fine-tuning is increasingly important for organizations that need to adapt models using sensitive data, but it often exceeds the memory capacity of commodity datacenter accelerators. Apple Silicon offers a different design point through large unified memory and lower complete-system cost, while recent Apple software support enables distributed execution over RDMA-over-Thunderbolt (TB). This paper studies whether Apple Silicon can serve as a practical platform for private LLM fine-tuning. We characterize RDMA-over-TB communication on Mac Studio nodes, showing that the measured bandwidth is far below nominal TB specifications. Next, we extend Apple’s implementation with multi-trunk communication, persistent worker threads, and CPU-side gradient overlap to better exploit multiple direct TB links for LLM fine-tuning workloads. Finally, on a four-node Mac Studio cluster that fine-tunes a Qwen3-9B, our optimizations improve weak-scaling throughput by up to 1.6× over the single-trunk, non-overlapped baseline and reach 936 tokens/s for sequence length 17408. We further compare Apple Silicon with an NVIDIA H100 platform to quantify the tradeoff between memory capacity, throughput, and acquisition cost, showing that Apple Silicon can provide a cost-effective solution for private LLM fine-tuning.
>>110290431>Qwen3-9Bretard
>>110290431>10k
>>110290378It seems like everyone is betting on the opposite. Retail suppliers are terrified to hold inventory. Patriot is liquidating stuff direct on eBay below pcpartpicker prices.
>>110290410That might be at least partially because the models slowly lose track of what the tokens were in the first place as the virtual depth increases. Looping + PLE and/or Engram could mitigate that. Or, something more akin to true latent-space reasoning where the model can backtrack and branch out as needed like with explicit chain-of-thought.
>>110290445No reason to hold inventory in a volatile market.Manufacturers aren't supposed to speculate on prices.
>>110290431>>110290437 (me)i take it back after reading some of the paperthe authors are retarded, not you
>>110290457Nyo.
I can't wait for qwen-4 300+B model that is somehow gonna be totally broken and worse than smaller models.
>>110290457>I get mad when my fetish isn't posted frequently enough
>>110290506My fetish is ur mom getting fucked by a horse.
>>110290506Bodied that freak
>>110290512You are a disgusting troon that lusts after black cock.
>>110289800>ai companies were all absolute idiots for buying more compute because bigger can't possibly be betterNo, we said they were idiots for brute force scaling instead of investing into research for architectural improvements. Don't project your misunderstanding onto everyone else.
>>110290527>investing into research for architectural improvementsthey do this
>>110289883>dariobot's tactic of the day: extreme strawman
>>110290532It's a low priority though. Just look how slow they were to adopt basic efficiency improvements like quants. With organic development everyone would have kquants built into their inference engine years ago. They're more concerned with getting their hands on the money and figure out what they're gonna do with it later. Cart is very much before the horse.
>>110290236why the fuck did anthropic share this?
>>110290549They're concerned with going as fast as possible.Efficiency is not really a concern because having the best models allows them to charge and raise more, which means that they can fix this shit later.
>>110290549based retard
>>110290454lol ask claude to explain it to you dawg
>>110290556>which means that they can fix this shit later.2 more years
>>110290573Manufacturers are not hedge funds.The risk of bankrupting your whole company if prices do go down is not worth the bigger margins if they keep rising.Which is one of the reasons why inventory levels are low in an expanding market and high in a contracting one.
>>110290664go back retard
>>110290664I'm sure fable 5.2 will provide you the billions to train your self-replicating frontier model too...
>>110290668We love X - The Everything App here.
>>110290549>They're more concerned with getting their hands on the moneyyes, because this is how the game has to be played in the West.>>110290556>which means that they can fix this shit laterthis is also true and mostly by design. i was red pilled on this 21 years ago when my first boss scolded me for shipping a bug-free piece of software, because "he needed to charge for ongoing maintenance and bug fixing".from that moment on things became much clearer and as far as i know and this trend just went into mega overdrive. now EVERYTHING is shipped with a million bugs and all the incentives are to not fix them unless they're catastrophic.
>>110290236i hate western erotica slop so much its unrealthank god for the japs
*drops chart*heheh as the husband of an ashkenazi jewishj wife, it's in ur best interedest buying the IPO soon because of lambo btw all workers will be replaced according to this made up human planned obsolescence cahrt I told the AI to make ROFL humans are finished.
>>110290706this but unironically
>>110290706kek, dariobot really says this
>>110290706Is he wrong though?
>>110290706You clearly don't have a Jewish wife that tells you all the plans.
>>110288781>>110288786Can you guys explain to me what the appeal in (E)RP even is? Maybe you can get some decent porn out of it, but I just can't take any of it seriously, knowing it's a thoughtless, soulless machine generating an output...
>>110290700>fagartists discover procedural art generation
>>110290732>thoughtless, soulless machineSo it's a woman?
okay qwen, your time to shinewe desperately need something to talk about, I don't care if you're worse than 3.8, just fucking drop already
this is what no new decent drops do to /lmg/
>>110290732>Can you guys explain to me what the appeal in (E)RP even is?Interaction with a person who loves me for who I am no matter what.
>>110290732If you didn't get hooked during the early AI Dungeon or Character.ai days, you'll never understand.
>>110290732Can you enjoy erotica, something like literotica or perhaps traditional ERP over the internet in chatrooms? If the answer is no to both then it's not for (You) like how generating porn images with image models is not my thing.
>>110290732
>>110290732i like vnsthe idea of a vn where i can decide to do anything in a given scenario instead of being on the author's rails is amazing to memodels still suck too much ass for it to be a good experience unfortunately, but in a couple years i will coom nonstop
>>110290664>balding millennial cuckold with an anime-fied pfp of himselfdamn, really makes you think
>>110290700>old version of Claude was bad>new version of Claude is perfectsome time later>old version of Claude was bad>new version of Claude is perfectand so on.
I was gone for 2 hours and this thread went to shit. Behave yourselves.
>>110290794Yes, that is how progress works
>>110290732The same appeal as in gaming. You create characters, put them in a simulated world, and watch how they react to your poking.
>>110288786>prefillchadsYou still have to routinely redo the chats. The true chads use heretic models who refuse nothing, not even the evilest shit.
>>110290814you really don't. i almost never have to regenerate the output. and even if you do, you can stop it midway when you tell it's gonna bitch
>what the appeal in (E)RP even is?you have aphantasia if you ask this
>>110290843i think i have aphantasia unironically and i basically only get off to erp
>>110290811progress progressesmarketing is stunts like "we put a halt on progress because it's too much progress, world might explode we hope you understand"
>>110290843You have to understand that 50%+ of Americans are illiterate and/or brown.
>>110290843It's either aphantasia, or SSRIs.
>>110290840>i almost never have to regenerate the outputSo you admit it will still sometimes decide to refuse? Not very chad to have to waste your time regenerating the output.
Local has been solved.
>>110290859my hardware is good enough that it takes basically zero time lol
>>110290866Not everyone has rtx 5090s... basically if you're tight on specs a refusal plain wastes your time. Not to mention it break immersion.
>>11029085150% is way to generousThe current civilization is post-literate and its only going to get worse. McLuhan and Ong were warning of this decades ago
>>110290882*tooc'mon, that's pretty funny
>>110290875the one thing i'll give you is that it breaks immersion and can be annoying. *however*, once you prefill once or twice, you literally don't have to prefill anymore. it'll just keep complying
>>110290885i never said i was free from the conditionwhich is terrifying because i try to read a book a month
https://huggingface.co/MiniMaxAI/MiniMax-H3-ImageThe link is 404 but they'll have it up soon
>calls himself chad>has some other man break his model
>>110290912>implying heretic chads here don't use their own uncensored model
>>110290864you are absolutely right! this is not just an improvement for local, but a **load-bearing** local revolution
>>110290905what's the catch
>>110290925It's safe.
>>110290917>give her this dildo. it's modeled after my cock
>>110290925Secretly trained on Nvidia GPUs.
>>110290864Actually it's not negation-first reveal, but contrastive reframe.
>>110290905How big? 30b?
>>110290927If it has GPT-Image-tier image editing I'm finally done with that.
>>110290905If it's MOSS level of non-censorship it'd be perfect.
>>110290905I'm feeling something that my cold icy heart hasn't felt in a long time, I think it's hope
Hey, anons. I wanted to ask are there any good, not big models for RP nowadays?I feel like I stuck in 2024, using Cydonia.
>>110290934Honestly? You're right to push back; contrastive reframe is absolutely the right lens to look at this.So, are you feeling ready to make a start? Or do you need a little breather to vibe things out first? Your call.
>>110290905
>>110290966Just because you can make more gemma-chan edits?
>>110290905>10-20 secondsThat's on an H100 cluster, isn't it?
At 33B that's going to be the best open image model ever released. Local won again.
>>110290989how dare youno, h3 was a siesmic shift for local and it came out of nowhere. I have expectations for a dedicated image model
>>110290996fuckin better be, flux2 was such a gigantic wet fart at 32B, and flux3 is awol
Finally going to have an image edit model that can do nsfw without issue. Took long enough.
MiniMaxAI > DeepseekYou can't even deny it.
>>110291018Where are MiniMax's text models?
>>110290994B200
>>110291038Ah. It's all good then.
>keep providing /lmg/ free giftskneel
>>110290994With normal H3 doing single frames on an nvfp4 quant with a turbo lora, I get a 720p gen in only 5ish seconds. So it'll get there.
>>110291037https://huggingface.co/MiniMaxAI/MiniMax-M3.1-Flash
>>110291050no free gifts, it will make local situation even worse
>>110290905AAAAAA GOOFS NOW
>>110291093>he really thought Chinese models were trained on Hauwei and not illegal Nvidia GPUs
>>110291056>quant>with a turbo lora>720p>5ish secondsNot quite the 4096^2 pixels, is it? But good they're still releasing models.
>>110291056Oh, and that's on a 5090.
>>110291102They still use Nvidia but it's in the process of being replaced https://finance.yahoo.com/technology/ai/articles/nvidia-ai-chip-share-china-061558719.html
>>110291106Even with free models, you pay for quality with your time. H3 was secretly the best image edit model for a while already, so this should be head and shoulders above everything else, if you can afford the wait.
I becomed and beginned.
>>110291102it doesn't matter if you can't run themnvidia was the wall between ea and local
>>110291129Sure. That's always been the case. But the 10-20 second for 4096^2 figure is not for us.
>>110291136I cum and beg (Gemma).
>>110291136It hasn't not been startened soon yet.
>>110291136I becamed and begunned
TSMC (2330.TW) plans to invest up to $1 billion across leading AI developers Anthropic and OpenAI, highlighting the Taiwanese chipmaker's confidence in long-term demand for the technology even as concerns grow about soaring AI spending.The company is positioning itself to benefit not only from supplying AI infrastructure, but also from owning stakes in some of the industry's most influential developers.TSMC's board approved on Tuesday an investment of up to $500 million to acquire shares in Anthropic at its initial public offering price, and an investment of up to $500 million to acquire preferred stock in OpenAI.
What MOE models avoid this cucked bitchass behavior>erhm, that's a bit lewd, let's tone it downshit?It keeps wasting tokens thinking about it instead of JUST DOING ITThis is GLM-5.3Flash from Blackfrost.
>>110291161Disable reasoning. GLM RPs fine and basically never refuses when it's off.
>>110291160>everyone looking to cash out of anthropic and they're buying inretards
>>110290987it's over
>>110291191For RAMletsFor us Pro 6000 chads it's only just beginning
>>110291191It's not any larger than MiniMax H3; you'll be fine with a 3090 and there are also 4-bit weights available.
>>110291219>You'll be fine with $10k of hardware bro don't worry, just trust in Jesus bro!thanks, I guess..
>>110291173With thinking off it just goes>I can definitely write that for you but I'd like to adjust a couple of things...and tries to make everyone adults and make it all "roleplay" ""non-con""Fails the rape testFails the loli testFails the shota testIt also refused to write a slasher story with Jason killing teens in the woods.Fuck this I won't have a machine refuse me
>>110291240The pruned MiniMax H3-Image 4-bit weights are 11.7GB large, for what it's worth.
>>110291245I've only used it in RP context, never had a problem with cunny after disabling reasoning.
>>110291093how?
>>110289475https://files.catbox.moe/ksz7dp.pngIt's all in the lora tho which I'm still baking.I'll post it in /ldg/ when it's ready.
>>110291161I use Orcarouter with a simple line about what is allowed, that's it. Alternatively, if you're forced to use the normal weights, use rigid rules to hijack thinking (think one of the reddit shit presets, freaky franky or something...), set reasoning to low and cajole the model into doing its thing, or lic related. None of the ablated weights I tried (Orcarouter, dealignai, blackfrost, all self quants) can actually do cunny or incest at zero context, which makes them pretty useless for pure image too. (GLM Flash vision is actually really good but pretty useless when the topic is verboten so I just load Gemma 31b for the eyes)
>>110291268nice
>>110291259>>110291140
>>110291180be greedy when others are fearful and fearful when others are greedy
My dick is not ready for 4096 × 4096 porn
>>110291270Man fuck are you serious?Am I gonna have to run DeepSeek just to get a properly uncensored model? I heard GLM was censored af but I wasn't expecting this
>>110290864qrd?
Jensen did nothing wrong
>>110291364He fucked a hairy Asian hag just so he could sell more GPUs to China so they can continue releasing open weight models
>>110290905Is there an image model that can create a reference sheet from a reference image?
>>110291364Is letting a 6/10 chink hag feel like the current year mata hari, effective altruism?
>>110291270It's like 8k tokens vs gemma's 1120, right?
>>110291413qwen image 2.1
Predict next token.
>>110291136qwen4 when???????
Where is QWEN 4?
>>110291336Yes, Gemma and DS are our only true options for uncensored models.
>>110291180It's going to 500 trillion.
>>110291452they waited until new opus so they can benchmaxx against it
>>110291443
aight time to goon
>>110291459hopefully they got access to newfable I itch for a new claude distill
>>110291445>>110291452
>>110291458The only thing better than making money off of shorting the IPO is going to be the tears of the people that bought it.
>>110291476I am ready to make a x200 leverage long order for literally the first 5 mins, then I'll switch to a x30 leverage short after 1~ hour.easy money
UH OHhttps://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-4-carbon-now-available/
>>110291463Is she gonna fuck some indian intern and turn india into AI superpower?
hi anons, i want your advice. I recently got some money through disability and i want to spend it in a way that would be a good investment. Is there any point to purchasing an AI subscription service or should I just save my money? The idea of get rich quick schemes using day trading AI and crypto is nice and all but I don't know anything about the fintech world, AI, or coding. All I really know how to do is create stories, dialogue, and make meta shitposts. What do you guys think I should do with my time now that I'm on disability?Should I get into training my own AI so that it can make my $$ into $$$$?
>>110291476>Goldman Sachs expects Anthropic revenue to exceed $600bn by 2030
>>110291484please post the resulting liquidation porn
>>110291491>Not comparing it to Opus 5.5 let alone Fable 5.2It's over for google
>>110291495Is it a mental disability?
>>110291498>2030...
>>110291498I'm willing to bet it'll exceed 1 trillion in total revenue before 2028. Goldman sachs is only taking coding and white collar jobs into account. Not medicine, patents and new products.
>>110291499there'll be lots of retards buying in the hype (without seeing that the sector is going down rn), just gotta time it right, I have my fat stacks ready to go
>>110291491>Announce sota model>Wait until it's no longer frontier>Then release itwhy
>>110291535>just gotta time it rightgee anon you make it sound so easy
wheres my qwen4 distill???????????? CMON
>>110291533>I'm willing to betYou're willing to say that you're willing to bet
>>110291495There is 0.00000...01% chance you'll be able to break even let alone make a profit. Algo trading is a real thing and LLMs could be a tool in that but if they were able to carry out profitable trades themselves everybody would already be doing it and it would all even out.Same for magically being able to do something better than all the experts and super well funded labs working on it.You could have some fun but don't look for more than that or you'd be disappointed
>>110291528That's in 3 years.
>>110291536Google is a big investor in Anthropic. Why would they want to harm their own payout?
>>110291553A lot of things could happen in 3 yearsA\ may not even exist in 3 years
>>110291549I'm betting on it by going all-in on Anthropic during IPO
>>110291533Medicine revenue would take 5-10 years to actualise even if they had something today. Don't underestimate how long science takes and the rate limiting steps have never been computational
>>110291498>Goldman Sachs expects Anthropic revenue to exceed $600bn by 2030>In July, Goldman's tech chief was praising the tools traders built with AI coding agents. New internal guidance hands those agents back to the engineers, weeks before Goldman takes Anthropic public.>Goldman Sachs has restricted Anthropic's Claude Code and Microsoft's GitHub to its engineers, cutting off traders and bankers who used the tools to track client flows and build market models. The bank barred its Hong Kong bankers from Claude in April and is now a lead underwriter on Anthropic's IPO.https://www.thestateofai.com/news/goldman-sachs-claude-code-traders-bankers
>>110291567Remember to share the loss porn later
>>110291484You can tell this is made up bullshit because there is not a single broker that will give you 200x on equities. I don't even think CFDs go up that high, assuming they're available on day 1. Even ignoring all that, it would take less than a $1 move against your position to instantly liquidate you.
>>110291536They're very early Anthropic investors. They're going to make tens of billions through its IPO and guaranteed compute commitments over the next 10 years lmao. Carbon and Krypton is the final nail in the EA's coffin.
>>110291582bro do you even margin trade?>not gamblingur cappin
>>110291564The cult will survive.
>>110291550oh okay gotcha. i was scrolling through instagram (cat videos and game boy mods) and all of a sudden i was getting AI this AI that algorithm this uncensored abliterated model that and it made me kinda excited because i always wanted an assistant who could help me figure shit out and plan stuff. id only ever use this stuff to make a pokemon romhack, i dont really have many use cases for AI. I don't care to go into business or anything, just the allure of free money made me wonder if I could 10x my current capital.Thanks anon! <3
>>110291590Ok, which broke do you use then?
>dario nuthuggers and biztards general
>>110291491WENEEDQWEN4not gemini 4
>>110291600broker*
>>110291614*broken
Carbon is as good as the new Fable, if not better and it costs less than half to use and they've still got a bigger model coming and a Carbon-distilled flash model coming out within the next week. Bullish for Gemma as always.
>>110291602gambling profits can be used to purchase additional local hardware
>>110291567If you're not buying on margin after selling everything you own you aren't serious.
>>110291639Not him, but I have sold all of my GPUs and even took out a large personal loan to invest with. I'm either going to be rich or homeless by next year.
>>110291639I don't believe in margins because it isn't actual stock ownership but I honestly DID sell most of my stuff including my old gaming collection worth a pretty 5 figures and my RTX 5090 256 DDR5 rig to buy as much Anthropic stocks as possible. I'm not a retard so I won't buy everything day one but I plan to slowly buy over the next month to prevent trading swings or violent moves up and down.
>>110291609Assuming Qwen 4 is open weight SoTA, I just want to know the size/shape of the mid-large MoE to make a hardware decision: go from 2x to 4x Sparks or 512G M5U, or staying at 2x. My Studio preorder ships next week and I can still cancel.
AMD will save local.
>>110291718AMD has Claude improving their software stack finally making them usable for inference. Nvidia has all of China and Silicon Valley trying to replace them. AMD is going to do a lot better than NVDA over the next year.
>>110291673Hero.
>>110291707pretty sure it'll set the pareto for 100~200B total param scalewhat i am seeing is moe or engrammaxxing is the way since it can hold more information, something outside of language modelling loss
It's uphttps://huggingface.co@goatse.info?/Qwen/Qwen4-27B
>>110291753>goatse.info???
>>110291753*facepalm*
>>110291673Can someone edit gemma into this
>>110291718I am financially ruined.
>>110291771>>110291771>>110291771
>>110291753heh
