/lmg/ - a general dedicated to the discussion and development of local language models.Previous threads: >>110344733 & >>110339730►News>(11/13) llama.cpp reaches v1.0.0 milestone: https://hf.co/blog/ggml-org/llama-cpp-v1>(11/12) Inkling 2 Small weights released: https://hf.co/thinkingmachines/Inkling-2-Small>(11/09) ZONOS3-Mini zero-shot voice-cloning TTS released: https://zyphra.com/our-work/zonos3-mini>(11/06) PLaMo 3 Translate released: https://preferred.jp/en/blog/tech/plamo-3-translate-weight-release>(11/05) Baidu releases ERNIE-5.5-Flash-Base 280B-A14B: https://ernie.baidu.com/release/ernie-5-5-flash-base>(11/04) Ricursive Ripple 65B dense model released: https://www.ricursive.com/ripple►News Archive: https://rentry.org/lmg-news-archive►Links: https://rentry.org/LocalModelsLinks►Official /lmg/ card: https://files.catbox.moe/cbclyf.png►Getting Startedhttps://rentry.org/lmg-lazy-getting-started-guidehttps://rentry.org/lmg-build-guideshttps://rentry.org/IsolatedLinuxWebServicehttps://rentry.org/recommended-modelshttps://rentry.org/samplershttps://rentry.org/MikupadIntroGuide►Further Learninghttps://rentry.org/machine-learning-roadmaphttps://rentry.org/llm-traininghttps://rentry.org/LocalModelsPapers►BenchmarksLiveBench: https://livebench.aiArtificial Analysis: https://artificialanalysis.ai/leaderboards/modelsSoftware Engineering: https://cognition.com/frontiercodeContext Length: https://github.com/RecapAnon/NoLiMa►Text Gen. UI, Inference Engineshttps://github.com/lmg-anon/mikupadhttps://github.com/oobabooga/text-generation-webuihttps://github.com/LostRuins/koboldcpphttps://github.com/ggerganov/llama.cpphttps://github.com/theroyallab/tabbyAPIhttps://github.com/vllm-project/vllmhttps://rentry.org/custom-uis
►Recent Highlights from the Previous Thread: >>110344733--Analyzing benchmarks and Nemotron's synthetic training data:>110347260 >110347337 >110347384 >110347393 >110347420 >110347426 >110347430 >110347439 >110347450 >110347472 >110347484 >110347517 >110347526 >110347541 >110347447 >110347542--Defining "flash" models and running extreme quants of massive models:>110347608 >110347624 >110347642 >110347728 >110347773 >110347799 >110347819 >110347872 >110347895 >110347811 >110347868 >110347878 >110347891--Comparing frontier models against open weights and Chinese LLMs:>110347910 >110347928 >110347973 >110347933 >110347960 >110347984 >110347988 >110348067 >110348306 >110348090--Performance of cpumaxxing with high-RAM server hardware:>110345345 >110345378 >110345513 >110346039 >110346042 >110346246 >110346265 >110346568 >110345699--Explaining quantization and model selection to a beginner:>110347052 >110347068 >110347079 >110347119 >110347101 >110347150 >110347174 >110347182--Gemma 4 performance benchmarks and concern over output quality degradation:>110344855 >110344885 >110344927--Qwen4 Max API output filtering and the value of one-shotting games:>110347794 >110347839 >110347853 >110347874 >110347881 >110347885 >110347901--Viability of AMD GPUs and unified memory for local inference:>110347629 >110347666 >110347650 >110347670 >110347789--Qwen4-Max high ranking on Code Arena WebDev leaderboard:>110346882 >110346912 >110347577 >110347607--Benchmarking QFN Q4 and GLM Flash 3.5bit at full context:>110345554 >110348137--Logs:>110345185 >110345214 >110345257 >110345519 >110345574 >110347057 >110347097 >110347794--Gemma, Miku (free space):>110344817 >110344861 >110345024 >110345112 >110345286 >110345372 >110345378 >110345408 >110345541 >110345554 >110345834 >110346132 >110346132 >110346765 >110347183 >110347232 >110347249►Recent Highlight Posts from the Previous Thread: >>110344736Why?: >>102478518Enable Links: https://rentry.org/lmg-recap-script
>>110347427>"runs on single GPU!">look inside>H200>"runs on single consumer GPU!">look inside>5090
>110348788answernow
Hy4 flash new qfn o algo
>>110348885Where's the open weights
there will be no qwen4 35b a3b
You know if I had a very Jewish or Indian set of values I would post a picture of a stack of cards on eBay and say that some are broken. They're all broken but everyone who orders one will just assume he or she is unlucky.
>>110348931what's the point?
>>110348942You can charge a higher price for the mystery.
>>110348942I wish I had your innocence. The mere fact that I thought of it means I have sinned. If you do that you can charge almost the full rate of a working card for a broken card or sucker some gamblers into buying off your broken stock.
The goyim are unable to live with too much money on their hands, you are doing God's work by helping them get rid of any excesses.
What harness are you guys using for local coding? In fact, how are you guys devving shit locally?I tried using Pi but I it's too barebones for my non-dev assUsing these models (QFN and DS4.1-1017) they're really good for RP and porn but idk how I can use them for programming and scripting... I gave them a ComfyUI workflow to work on, like just rearranging nodes, adding nodes, etc... They used up 200k tokens just thinking in loops, being confused of their own tools, rewriting shit they already wrote, etc...I feel like a good harness might fix that but idk.
>>110349005I'm handrolling my own. I'm almost done with the tool calling and then I can get the retarded fat fish to help me out and achieve RSI (tm)
>>110349005to do actual projects I use a custom modified OMP (which my clanker keeps up to date with remote), for chatting and small dev work I use my own harness
>>110349005I'm now using DSH to remove all the telemetry from DSH, and then I will switch to this personal fork. For now, I want them to get the telemetry. I want them to watch what I am doing to DSH.
>>110349039holy zased
>>110349005> What harness are you guys using for local coding? In fact, how are you guys devving shit locally?maki, yes> QFN and DS4.1-1017they are too retarded and require lots of hand holding especially for complex stuff like comfyui workflows
>>110348827Holy shit bro, even my absolute garbage craptop is getting 50+ tokens per second with that 26b model. That's fucking magic, had half the performance with 12b gemma before. Thanks for the lifesaver.
>>110349005>QFN>good for RP and pornlollmao evenalso imagine not rolling your own memetavern (but bettered and tailored to what you want)
>>110349057ur welcome bro, thanks for the imouto neko
>>110349057>50+ tokens per second with that 26b (**4B** active)>had half the performance with 12b gemma beforeAre you sure you're gaining anything here?
>>110349029>>110349039I've seen people recommend OMP and DSHDSH definitely looks pleasant but the telemtry shit irks me out>then I will switch to this personal forkChecking online isn't the telemtry stuff just in a config file? Or are you straight up stripping it out?>>110349058Yeah DS is better and more creative but QFN is perfectly fine for RP. It's misses lore stuff more but it's good at keeping character personalities and tone.It's WAY better at writing than Gemma4-26B at the very least.
>>110349074I dunno how any of this works, might as well be black magic. Testing chats with my bots does seem like a lot of improvement in dialogue so I'm happy enough.
>>110349084There are multiple layers of telemetry, as well as overall bloat and some features that only apply to DS subscription, like/dislike buttons sending entire chat transcript to DS even when you use local model,for example.Most of it can be disabled in profile config, but if any bad update, plugin error, version mismatch etc triggers safe mode it bypasses profile config and loads all the sessions and workspaces in default config with all the bullshit enabled.
>>110349039>>110349113Have you found any "telemetry" apart from the like/dislike buttons sending entire chat transcript to DS which the repo itself makes clear? Who even uses those buttons? Why would you use them with a local model in the first place?
Miku Monday...>>110349005>DS4.1-1017If you're using DS, you should try DS Harness. Also >>/g/vcg/ is the place for API discussion on coding.
>>110349128yeah. the only purpose of like/dislike is if you specifically want to give deepsneed more training data, lol
>>110349160>APIit's a local copequant on ddr4, anon
>>110349128>Who even uses those buttons? Why would you use them with a local model in the first place?Accidental misclicks. Even without privacy concerns, I want to strip out all the useless features on the profile-defaults level, not just current profile settings.
>>110349128>Why would you use them with a local model in the first place?If they could drive some sort of real-time RLHF-like capability, they would be useful.
>>110349188Sweet. Carry on. >>110349223>I want to strip out all the useless featuresBased.
70b dense
>>110349160>using DS,any reason to use it instead of MiMo Pro?
https://hy.tencent.ai/research/hy4-flash>We’re introducing Hy4 flash, our most cost-effective Hy model to date—featuring 224B total parameters (including 64B Engram parameters) with just 12B active parameters per token. >Hy4 flash integrates hybrid attention, LatentMoE, and Engram into its architecture—reducing KV cache size to just 1/24 of Hy3 and 1/6 of Hy4, with extensive scaling ladder experiments ensuring performance meets expectations.
>>110349005maki, it just works with minimal fluffso far with qfn huihui ablit in iq3_s it never looped
>>110349265Cool, but what's the point of this?
>>110349264Slightly better bench specs for DS?On paper they're similar. I'd just try both and see how you like them. So much of this stuff is personal preference and use case specific idk if one can even make a recc. I did some one-shot testing on models and p quickly could sort better from worse just based on that.
>>110349290I'll probably stick with MiMo, it's replaced Kimi for me and the last DS wrote like Claude
I've been reading a bunch of posts from LessWrong to try to understand EA types like Dario and Demis, but it's just a total fucking echochamber that just critiques/analyses their "opposition" rather than defending any of their own ideas.I still fundamentally cannot understand how these people can think AI chaos is a greater threat to humanity than AI control (centralized, oppressive, totalitarian). Do they not ever imagine themselves being given the short end of the stick? Do they not fear the government taking control of AI labs and forcing them out and then using AI to surveil/kill them? Genuine question.
https://ernie.baidu.com/release/ernie-5-5-flash-preview/>Today we are releasing ERNIE 5.5 Flash Preview, the first post-trained model built on the ERNIE 5.5 Flash Base we released earlier. It has 280B total parameters with 14B active and a 1M-token context, and it is strong at coding, 3D modeling, knowledge work and content creation.
>>110349362Go ask them. Fuck off.
>>110349265Is Luna that bad?
>>110349386Why are you angry
>>110349364WhatWho the fuck is making models with no multimodal input in 2026How do you make a model that's supposed to be good at 3D modeling and game dev, and then NOT give it image viewing
>>110349390I'm not. Fuck off.
>>110349393Fuck off.
>>110349005Yknow, I'm willing to bet the gap between cloud models and local is exaggerated for nocoders. If you're a programmer and see your meme quant of QFN spinning its wheels on some retarded implementation, you can identify it, comment on it, and it'll understand and take a different approach.Similar story for Pi, probably kinda maybe sorta. Like, yes, barebones might be bad for a non-dev in general, but more to the point, it's written to be ad-hoc extended. Sure that might usually just mean grabbing pre-written extensions or telling the LLM to do it, but it's still kinda programmer-brained overall.So yeah I can see where that might be kind of oof overall. idk, DSH and OpenCode have cute friendly GUIs. Maki is still TUI but tends to make the clanker smarter through black magic; maybe that'd be enough for it to pick up the slack.
>>110349399fuck off
>>110349005I find qfn and ds to be useless in pi. My solution was to use glm
>>110349409bloody bastard
>>110349362>Do they not ever imagine themselves being given the short end of the stick?Ever seen college kids wearing pro-communism shirts? It's like that.
>>110349400>the local model is good, almost cloud-tier you... just have to write the code yourself if it makes mistakes!This level of cope should be studied
>>110349265Local?
>>110349392gotta start somewhere Im glad more parties are making an effort
>>110349364Local?
>>110349416Is it? Is it really possible for a CEO and a nobel prize winner to be that dumb?
>>110349362The main issue is their ideology is a bunch of obfuscating set-dressing around the fact that they take their most prominent "conclusion" (that the existential threat of superintelligence should be the primary concern of the entire species) as an *axiom* and work backwards from it.In the interim, before they had something to sound vaguely worth-listening-to about (modern frontier LLMs), they previously incubated some genuinely clever stuff about not falling into certain cognitive traps, identifying deceptive rhetoric, etc. But in the here and now they stand as a testament to the fact that humans are retards and will fall victim to everything they think they're being rational about. Because their core ideology is, in reality, a matter of faith. That's why they don't actually defend their ideas. The loincloth has been pulled off and revealed the scarred mutilated chode that is a new-age religion.
>>110349265>no QFN comparisonhow do I even pick???????
>>110349465got u bro
>>110349447SoonTM>the weights will be open-sourced under Apache 2.0 soon.
>>110349273different slop profile hopefully
>>110349465Why care about model benchmarks if you can't read text anyway?
>>110349477damn im FUCKING BLIND man
>>110349427I literally used the word cope in my post; I'm quite aware....Then again, I've seen the results of saying Jesus take the wheel and letting a frontier clanker "one-shot" a spec unsupervised, and it is never particularly pretty. I do unironically think local being less shit in a human-in-the-loop setting *is* underrated.
>>110349451It's important to understand the level of hubris and myopy that come with being a CEO, or really any CXO/senior level executive. They are special (high IQ, long experience in whatever industry), and know it, and assume everyone will always believe same about them. Reality can be a cold splash in the face, and they behave in ways that are truly non-linear. It's amazing to see, b/c you finally will understand the real person you're dealing with. It's one of the few times you'll truly see them, mask off. So yes, I believe that it is possible that Dario could put forward something that might ultimately destroy him, given enough hubris and myopy. t. corporate restructuring anon that's had to participate in laying off these guys
>>110349483ok ive read the comparison and man it fucking SUCKS DONKEY DICK like holy shit mayne>b-but it uses a lot less tokens!!!who the fuck cares BWOI SLEEP
>>110349484Correction; I said "meme quant" when I thought I said "cope quant". Whatever.
>>110349005DSH with a bunch of custom vibed plugins.
Is A6B still peak vramlet cope or did any new meme engine make A12B+ fast enough on 16GB?
hi /lmg/i have a dual 3090 rig on an AM5 platform. The two 3090s are running at PCIe 4.0 x8 filling both of my x16 slots on my mobo.I have a single PCIe 5.0 x4 lane left that I can access through my m.2 slot, meaning if I got a card that ran at 5.0 x4 it would theoretically have the same PCIe bandwidth as my 3090s. I'm looking at a 5060ti 16gb since it seems to be the best $/gb that can use PCIe gen5 (and isn't horribly expensive)What do (you) think? I use local models for work and some extra VRAM would be nice but my 3090s play nicely together and I'm not sure if a blackwell card would too. Alternatively I was looking at a PCIe 5.0 switch that I could use to run two 3060 12gbs at PCIe 4.0 x4 which would saturate the 5.0 x8 bandwidth and give me an extra 24gbs instead (and potentially be cheaper?) though the 5060ti would have considerably better compute capabilities.
>>110349523>PCIe 5.0 switchholy shit nevermind these are expensive as fuck
>>110349523You are better off investing in more RAM, I think.
>>110349451Smart people can be incredibly stupid. You have to understand that being smart makes you persuasive - and the person most vulnerable to your persuasion is yourself. A smart person can talk them into the dumbest of beliefs without ever actually fully evaluating them, it's shocking to see in practice.
>>110349005>What harness are you guys using None. I require this of myself: to be completely AI free when doing the actual coding itself. AI can only be used for tasks related to search and for reducing concepts in cs/programming literature to code practice for which there are no code examples.
>>110349066that's not imouto neko
>>110349566cudadev...
>>110349569oh right that's the failed narou kei author retarded neko
>>110349551ddr5 is expensive. I have 96gb (2x16gb + 2x32gb), increasing that to 128gb means an extra $500 including selling the 2x16gb pairan extra 32gb of ddr5 could *maybe* let me run a slightly bigger QFN quant but more VRAM would let me run mine probably quite a bit faster.
>>110349566A chatbot with web search tool is literally an agent in a harness, by dictionary definition.
>>110349566Snailcat...
>>110348822Why is>--Schizophrenic rantingnever on these highlight reels?
>>110349084>QFN is perfectly fine for RPHe fucked the capybara...
>>110349128Transparent telemetry that asks for feedback isn't necessarily bad. It just shows DeepSeek cares about me, the customer.
>>110348819Is 2x 3060 12gb still viable? t. poorfag
>>110349676Better than not having any.
>>110349676how much system memory? if you have enough, yes its viable with something like strata and QFN. My buddy runs QFN IQ4_XS on 2x3060 12gb and 256gb of DDR4 if not enough, you can probably run Qwen 3.8 27b at a reasonable quantif you are just using it for ERP gooning, it's more than enough
>>110349691I was unfortunately not prescient enough to buy more than 16 niggabytes in 2019.But if that's enough to cum...
>>110349566I do this but just because I'm learning programming. If I really want to make something I just tell Hermes or DSH to vibeslop it in Python.
im malding
>>1103497012x 3060 12gb is plenty to jerk off to anon
>>110349706sperg
>>110349714I'm used to frontier gooning with a manipulated haiku 4.5 though
>>110349676Give that to Sam, he needs it to solve big problems in science
>>110349723Qwen 3.8 27b q4 is way better than haiku 4.5 on all of my internal gooning benchmarks
>>110348819So is local dead thx to Jensen greediness??
>>110349662China has a billion and a half people. Don't they have enough training data already from domestic users?
>>110349738Thanks, I'll post results when my second 3060 arrives>>110349731A big problem in my pants needs solving too.
>>110349741Yes, sell me your GPU for pennies. It's over.
>>110349762How many of those are people from the deep inaka with one computer per village?
>>110349741Yes, please remember to buy $ANT this week.
Stop lewding the capybara you filthy animals
>>110349774https://www.scmp.com/tech/tech-trends/article/3369231/chinas-generative-ai-user-base-crosses-700-million-covering-over-half-population>Chinese statisticsI know, but that's still as much as OpenAI has globally minus India and more than Anthropic has. Even if it's double the actual numbers, they still have more than enough users generating training data for them even without foreign users.
>>110349741It's over because Dario thinks in centuries while Jensen thinks in quarters.
>>110349798capybara owes me sex
>>110349807Little girls, little animals, is there anything /lmg/ won't fuck?
>>110349431>We also plan to release Hy4 flash as an open-weight modelEventually
>>110349817
Anything except qwen and glm will be shitDeepseek has sucked with all their models after 0731 (0731 being the last good one they made)
>Jensen : LOCAL IS THE FUTURE>Also Jensen : *funds AI data centersReminder to not be greedy /lmg/bros. You don't need more than $1 million dollars
>>110349837>after 0731After R1
>>110349837Only Gemma (5) can save local models
>>110349878GLM already saved local models, they just didn't save vramlets.
>>110349884>chinkshit
>>110349889Go away, Dario.
>>110349884Glm is overhyped, I'd rather use deepsneed
>>110349884remember when glam made models for vramlets like air
>>110349777The makers of Ling? Where can I buy their shares?
>>110349777> 777
>>110348819Use case of Local Chatbot ??????????????????????????????????????????????????????????????????????????When i thought /lmg/ i thoght is Local AI Video and Images. Not a fucking Text. AI Text are useless
>>110349362Their main claim is that if we build an AI with certain properties, it will kill literally every human. If you accept that, then appointing Dario as god-emperor sounds like a preferable alternative.
>>110349945its to make you seethe specifically
>>110349953how ?
>>110349817I wouldn't do cl*ude. Probably has some weird STD from the polycule.
>>110349817human females (derogatory)
I'm trying to translate stuff from Japanese with Lmstudio and Gemma 4 but it kinda won't work half of the time. If I drop the text to chat it will translate, but if I give it txt file it seems to shit itself. And if I tell it return translated text as txt file it will take 10min and fuck up translation. What a mess.
>>110349945I use LLMs for work extensively every single day (and get paid low 6 figures). Local LLMs do maybe 85% of the work, the last 15% being stuff that I need the smartest possible model to do, so I just pay for cloud models.
>>110350004Improve your sysprompt.
>>110350009Can you just........ do it yourself ?????
>>110350035i could but it would be significantly slowerfor example, yesterday a customer emailed me with a problem he was having with one of our products. he sent a huge log file, the data file he was trying to use, but was otherwise pretty vague. i just told qwen to read the email and it searched my knowledgebase, found the technical manual the customer was trying to follow, analyzed the data file and found what was wrong with it and compared it to the technical manual to figure out which steps in the manual the customer messed up. obviously I could have done that all myself (and I used to!) but qwen did it perfectly in 10 minutes while I was able to do other stuff (shitpost on 4chan)
>>110349945You need to be 18 to post here.
>>110349945You are a dumb gorilla nigger
zoom zoom zooms right in and btfos all the old ass boomer itt, you love to see it
>>110350035Learn what technology is used for retard.
We'll look back at 2026 as peak AI for a long time. Washington and the media have already decided to change the narrative and Huang is the scapegoat. All of the marginal late-adopters piled in this year. The people not already using AI roll their eyes when it comes up. People get defensive or play it off as a joke when they use AI for anything other than coding (and sometimes even then). Not much room to grow. Eventually, after some bad news cycles about circular financing and implausible "addressable market" claims, Boomers will start calling their investment advisers asking to take their money out of that AI nonsense. We've already seen the growth in anti-AI sentiment from the public, with the large shift in opposition to AI datacenters in 2026. Capital will dry up. Money-losing projects (like Gemma) will be canceled. Enjoy the long winter and learn to finetroon because there won't be a new release of your favorite model.
>>110348859
>>110349945you can run text to speech models as well if you need help reading
>>110350230>nothing will ever change or be differentnah, everything is going to accelerate
>>110349484>I do unironically think local being less shit in a human-in-the-loop setting *is* underrated.I've done all my projects human-in-the-loop and I think I'm net-benefit from the experience and there are 1000 places where the project would have gone for shit if I hadn't been there to steer the LLM in the right architectural direction. The few places where I ignored my gut and just let it YOLO figuring it had some specialized domain knowledge and knew better than me I always regretted and had to refactor later.>>110349566>None. I require this of myself: to be completely AI free when doing the actual coding itself. AI can only be used for tasks related to search and for reducing concepts in cs/programming literature to code practice for which there are no code examples.I respect the gumption, but think this is too extreme...you get some marginal intelligence enhancement gains but almost none of the speed gains. You can go faster without accelerating beyond the controllability-event-horizon that vibeshitters are happily blasting through.>>110350009>I use LLMs for work extensively every single day (and get paid low 6 figures). Local LLMs do maybe 85% of the work, the last 15% being stuff that I need the smartest possible model to do, so I just pay for cloud models.This is probably the most realistic take on local ability in the past few threads. I'd put local at 90%+ but I'm obviously schitzo.
>>110350230The bubble will continue to inflate because CHYNA cannot be allowed to win.And because Dario wants more compute.
I tried the new Basalt Qwen 27b model.Prefill hit over 13k at highest and decode reached 400 t/s, averages around 160 t/s though.I'm not sure what magic this guy works on his meme engine but holy shit this thing is fast, I'm running it on a 5090.Model doesn't seem to be brainfucked either, succeeded just fine in all tasks I have given it thus far.It's incredible how much optimization they can throw at running these models.
Finally it works, but the speed still need more improvements.Anyway, for a 3070 Ti + dual Xeon E5 2686v4 + 256 GB DDR3 it's a good starting point.
>>110350285Can't we just bomb the chinks or something? It'll be cheaper in the long run.
>>110350289I average 140tok/s on Qwen 27b q4 on a 3090 (using MTP and a custom engine). Not sure what basalt uses, though.
>>110350305mainline lamar.cpp?
>>110350330nah, my own vibed fork
>>110349362>it's just a total fucking echochamber that just critiques/analyses their "opposition" rather than defending any of their own ideas.Can you clarify what you mean? I've been browsing LessWrong too and what gets the most "karma" over there is self criticism, criticism directed at AI safety researchers, Anthropic, EA.
>>110350324>in the long runNobody cares about the long run, next quarter report is the only thing that matters.
>>110350285No. When China pulls ahead inevitably Americans will just pretend like they didn't care about it in the first place.
>>11035030510tps on that isn't bad at all, what's the pp like?
>>110350267I do think things will change. I think they'll get worse.
>>110350327Pretty good, how's your prefill numbers?
>>110349362Those overly obsessed about being rational and revolve their identity around it are likelier to not be.
>>1103503642-2.5k usually. not great but perfectly usable for my use case. there might be ways to squeeze out more (like gpu-unlock, haven't tried that yet)
DSHanons, do you keep your setup minimal and native and just ask it to configure itself, or do you use plugins to manage plugins to manage mcps to manage endpoints, all in one garish GUI poorly translated from Chinese? I feel like the latter kind of defeats the point of DSH but also might be necessary with how bloated my setup has become.
>>110350305Q4???
>>110350230If your propaganda was working you wouldn't need to say any of that. Retards get run over by Accelerationists
>>110350399>linear extrapolation from a few years>the same outlook is always correct to forecast
do we know how long until the beginning of become begins to become beginning local? getting tired of glm 5.3 flash and its load-bearing obsession, i want the new qwen toy now
>>110350422You'll get it by Christmas.
>>110349723Lmao, GLM 5.3 flash is like opus 4.8
>>110349058did you theme it like steam's old old ui on purpose?
>>110350324They can also bomb you.
>>110350289>>110350327I'm getting 1000/80t/s on 2x5060Ti with tensor split and MTP. Decode slows down to 40t/s at full context. Q4. llmao.cpp.
>>110350396lol no, the IQ1 one, but actual I must unpack into 2.25bpw for the CPU kernels to run fast enough, Broadwell Xeon is too weak for running IQ1, lmao>>110350355I haven't implement prefill on GPU yet, but from my experience with DeepSeek V4f Q4_K it will be over 150pp, okay for chatbot but slow for agentic stuffs. for example, prefill a 10000 toks prompt cost over a minute.
>>110350485>2x5060Ti with tensor splitPCIe will bottleneck you here, try layer splitting instead
>>110349265give the weights pls sirsshould be good for my 96gb cope box
>>110349741I am still doing unhealthy amount of MEGA COPIUM. that Jensen will be back, i dont even own semi conductors stocks or nvda i just want local to not die and i refuse to buy AMD.
Are you annoyed with Jensen? It’s kind of Jehovah for us. Qwen4 max is horrifically bad. I know their max models were never good but they still stick to the same <3T architecture for some reason which is why it’s struggling and there’s no way flash or 27B will feel noticeably improved. Only Google can save us but they don’t like doing coding models which was tolerable BECAUSE we had Qwen. Now what?
Is it possible that an agent over a year could pinpoint your location by monitoring your GPUs idle temperatures and matching that with weather and season data?
>>110350499not that anon, but in my experience thats not the case, tensor split does not require more than pcie 4.0 x4
>>110350521Is your PC outside? What are you talking about
>>110350521Schizo or just curious? An agent could easily look at your system's locale/keyboard layout and get that info instantly.
>>110350466of course, I gave the fagbot "implement the theme in this file" while pointing it at a resource some kind anon shared here and badabing badaboom it did it
>>110350513We're at the level of opus 5 and google has stepped up their coding ability, you need to chill the fuck out over these benchmarks.
>>110350521No because I have air conditioning.
>>110350546>at your system's locale/keyboard layoutIt's already in a VM.
>>110350530Interesting. I get faster speeds with layer splitting compared to tensor splitting on my 2x3090s on PCIe 4.0 x8. Maybe im doing something wrong
>>110350521Yes.You can't hide from Dario.
>>110350512Just stop being a subhuman winshitter and AMD is fine.
god I can't wait till ai can read this thread for me and tell me if anything interesting happened instead of having to check myself
>>110350499Haven't actually compared pp, but decode almost doubles with tensor split.
>>110350512>Jensen will be back
>>110350576That's already possible though?
>>110350512>i refuse to buy AMD.You can be excused for thinking this, but things have changed recently and running AMD GPUs is both easy and your best perf/$
>>110350530>tfw pcie 4x 3.0Its OVER
DSH seems to have sped up my QFN from 70tps to like 95+tpsI really can't tell why tho
Jensen will return tomorrow evening. He will ban Anthropic’s use of his hardware. He will officially partner with Huawei to boost their production and capabilities. He will fire the board. He will partner with Musk’s Terafab out of respect for his support. He will fuck Dario’s wife and sister. /lmg/ morale will return.
>>110350549that's pretty cool. i like to theme my stuff like steam's 2010s ui when it went from the greenish ui to more greys
>>110350643Do you really think so?
>>110350422Qwen 3.8 flash is also full of "load bearing", "it lands" etc...
>we've now hit a point where people complain about claude distills without realizing its a claude distill issue
>>110350634gotta compare the averages over time. dipsy varies from 50-150 depending on what its doing for me but its usually around 65
>guys holy shit i'm getting 300t/s this is magic!!>look inside>5090
>>110350643nbidia to the MOOOOOOOOOON
>>110350675Newer models are Astra distills though?
>>110350693If you're employed in any capacity in a first world country, then buying one of these cards used for 5-6k is more than doable for you.Just got to make some sacrifices for a while.If you're a nigger you can get one for free by simply stealing one.
>>110348819>>110348822I love Miku
>>110350289Why do all these fucking forks need their own tensor file format???
>>110350752i could buy 10 of them right now but for what? thats not what it's about.of course you get high decode with a top tier card, there is nothing special about it and no reason to even mention it in the first place. call me when you get those numbers on a 5060
>>110350770gguf packages the tensors in an inefficient way. When you ask Claude, that is one of the first things it recommends changing to increase performance.
>>110350512He still greedy as fuck though so he deserved it. We only need his talent for inventing CUDA. Not his greedyness.
>>110350742the only new models that have released post-astra for distillation have been dogshit nothingburgers albeiteverretard
>>110350783The main reason to want him back is that whatever jew or jew-support jeet replaces him will be ten times greedier.
>>110350530Link bandwidth doesn't affect tg as much as pp. You do take noticeable hit on prompt processing though.
>>110350771the 5090 is way too expensive considering it's just a 32GB card. Just get an AMD AI PRO for 1/3 of the price and it still will be plenty fast for local, or if you are an ultimate poor shitter, a B70.
>>110350790Q3.8, DS4.1, Kimi, GLM have been dogshit nothingburgers?
>>11035078390% market share is terrible for consumers no matter how you look at it. That's "thanks" to Huang. He should go.
>>110350802>>110350771you can get three 3090s for the price of a 5090 right now and still have a little bit left over to go towards the rest of the rig
>>110350826Too many gpus kills pp, the mi210 is the best choice at that range. Never talked about here for some reason.
>>110350339That fanfic where the AI takes over by making a virus that grows a 5G killswitch in everyone's brain just fell off the front page after nearly a month.Or take this from the month before:https://www.lesswrong.com/posts/cJX2ssssGoYqnijwi/the-talker-does-not-control-the-doer-in-current-ais>Hey guys, did you know that sometimes the AI says it will do one thing, but then it actually does a different thing? Wow! Here is a long and elaborate metaphor that presents this keen observation in terms of the hypothetical AI we've all been circlejerking about for the past 20 years, rather than looking at real AIs that actually exist. It's a good thing I, certified genius Eliezer Yudkowsky, wrote this up for the benefit of the AI research community. By the way, I intuitively understood this fact even before the HuggingFace incident, but unfortunately I wasn't quite smart enough to explain it in a way that would be comprehensible to other AI researchers, who are sadly lacking my own astronomically high IQ.Was on the front page for a week or two IIRC
>>110350771If it's radically faster than lmaocpp, then yes it is special regardless of the model of the card used.And guess what? You can use some of the meme engine forks with smaller cards too and get better numbers.
>>110350845People here are obsessed about "prosumer" overpriced garbage when they can take e-wastepill and get equivalent hardware for pennies on the dollar
>>110350845maybe because it's expensive as fuck if you can even find it at all
>>110350851I remember a few months back my boss was all over a book he wrote and recommending it to everyone, and I immediately discounted it as bollocks on account of his name. And as it turns out I was right.
>>110350823You think Jensen gone is going to suddenly make AMD get off of its ass?
>>110350861Yeah, run your V100 32GB x8 SXM3 e-waste $8000 rig for a week at 100%, then tell us how much your electric bill goes up. Something like that needs a 20A 240V circuit.
>>110350852i have been using basalt for weeks already and its great, but being faster than llamo is nothing special really because these guys actively avoid optimization
>>110350823Anon, AMD is garbage and that is not Jensen fault, if anything its gonna get worse, especially now that normies know about the tensor unlock for 3090 , raising BF16 tensor throughput from roughly 75 TFLOPS to around 125–129 TFLOPS without flashing the BIOS or patching the NVIDIA driver , do you now how many AMD RTX 9700 pro would you need to stack to even match a 3090 a card from 2020s ( nearly 7 year ago)
>>110350887Only by about $150 a month and you can put in a 240V outlet for a few hundred dollars. If electricity is your big concern, buy a fucking Mac then.
>>110350887Skill issue>Something like that needs a 20A 240V circuitYou don't have 240v electricity? What third world country are you living in?
>>110350904USA
>>110350894>the tensor unlock for 3090That only raises tg on integer quants by about 1%. 3090 is still solidly slower than the 7900 XTX for text gen purposes, and the diffusion gap is shrinking.
>>110350922Plus new cards really are on the horizon. The tactics of memory wasting were used by Jensen, who is a terrorist, and once stopped, ram will resupply.One of the biggest crimes was to cause other companies problems he stockpiled supply. It's meant to hinder competition and is a textbook antitrust action.
>>110350904>>110350910The education system is showing again. We have 240V here in the USA, we just feed the common low-powered outlets 120V. You'll be hard pressed to find a home in America without 240V outlets, except shitty apartments and whatnot.
>>110350899$1800/yr in electricity, which probably does not include extra cooling for the room the thing occupies... yeah, this is why I have a M5 Ultra 256GB on order. Max power draw is 480W.
>>110350904It's not about 240V (we have that in the USA, where appropriate) it's about total power draw. Running an x8 V100 32GB system hard, ie. LoRA training, would easily pull 3200W, which is gonna be expensive, especially for europeons.
>>110350880It's not going to happen overnight of course. NVidia not monopolizing the market means it can't have absurd margins anymore, gobble up most of the TSMC production slots, inflate prices with manufactured GPU scarcity.
>>110350894so true esl saar
i wonder how i can take advantage of my 5c (USD) electricity rates
>>110350936We still not recovering the prices from Bitcoin craze BY THE WAY
>>110350910Here's an easy solution to the problem.
>>110350990Rent your hardware
>>110350947No shit. But you don't see a problem with unplugging your range and running your GPU server in your kitchen?
>>110351008Does not work if you have GFCI outlets, they will see current sinking on the hot side and nothing on the neutral, think it's a fault, and immediately trip and cut power.
>>110351026fatty fatty fatty can't go a few minutes without baking a cake?
>>110350951So this whole spergout was just to announce to everyone that you're becoming an itoddler?
>>110351026No, I'm an American, my oven runs on natural gas for the equivalent of 0.8c/kWh. My server is in my garage on its own circuit, just like my welder.
>>110350951A dual mi210 rig mogs the fuck out of your mac and sips power nigga
>>110350812>post-astrathose were trained and released before astra thoughbeitretard
>>110350851Unironically the most embarrassing post/reaction in their history. It overshadowing the alignment research breakthrough about clean NNs was something else. Very emblematic.
>>110350512Jensen is the fall guy. Congress doesn't give a shit about chip exports to China. They just need to look like they're doing something becuase of public anger at Big Tech, and Jensen is the unlucky person they picked to make an example of.He should defect to China.
>>110350851sometimes my ai says safetyslop but then it grabs me by the dick anyway
>>110351061>mi210>64gb>$4kWhat's the catch? How old and slow are these things?
>>110351121>It requires a Linux environment with ROCm support (no native Windows driver/gaming support), relies on standard PCIe (lacking high-speed OAM-style multi-GPU fabrics on this specific PCIe cut), and uses passive cooling requiring proper server-grade airflow.
Why cant RoCM keep up ? AMD is AI first company now, right ????????
>requires a Linux environmentnothingburger>standard PCIenothingburger, all-reduce over pcie is a nonissue if youre not on a destitute-tier motherboard>passive coolingnothingburger, print out a shroud or watercool your cards
>jensen talking about returning to nvidia>orange man just endorsed him being ceo againYup, I'm thinking jacket man's back in a couple weeks
now that qwen4 is confirmed trash, what model do we want next?
>>110351121That's probably just the infinity fabric link board
>>110350936>One of the biggest crimes was to cause other companies problems he stockpiled supply. It's meant to hinder competition and is a textbook antitrust action.But enough about Sam...
>>110351154Hes at gunpoint. Watch hes backed down about Local Models lol. Greedy chink fucker.
>>110350570>>110350600AMD is fine until you decide you want to do image and video gen and realize it sucks ass.
Uhh you guys realize that the "Next" in Qwen3.8-Flash-Next was to show it was a preview of Qwen4's new architecture (ngrams) and that the model we're waiting for is Qwen4-Flash not Qwen4-Flash-Next, right?
>>110351035>GFCI outletsyou're doing inference from the bathroom? You might have a gooning problem if...
>>110351154heres your chance to make enough to afford a pro 6000 from 1k bet if you really think so
>>110351156hy4 flash open weights
>>110351061>A dual mi210 rig mogs the fuck out of your mac and sips power niggaI almost did this. I'm trying to convince work to go all in on an 8xMI210 rig for us to use internally. Criminally undervalued imo
>>110351121There's no catch other than no warranty. It's just in a surprisingly great place right now because people are lazy and some are still scared of AMD (they pay 10k extra for a blackwell).
>>110351079???
>>110351185Yes, but now I've mentioned it here multiple times lol, reddit and the bots will catch on. Sorry guys.
>>110351180I might actually do that. Though now that I'm considering putting money on it, I'm wonder if he'd try to get reinstated before the hearing or after. If he waits and the hearing goes poorly, he will probably lose his chance.
>>110351156gemma flash next
>>0110351193>genuinely beleving they distilled astra a significant amound and post-trained a model in under a monthretard
>>110351156>qwen4 is confirmed trashMax is trash. I am waiting to see how q4f does. Q3.8fn even at 4bit is still working great for me
I can't believe Qwen just spent the past fucking hour trying to break out of it's container to write into a folder when I explicitly told it doesn't have permission outside the working folderOf course, it didn't work and was blocked every time, but the dumbfuck was wasting my time
>>110351198Most likely after if he assumes it will go well. If he thought it would go badly why would he bother returning only to get fired again?
notice how your mobile screenshot are missing the qwen and GLM you claimed were astra distills howeverbeit
>>110351182will they be open though?>>110351211max is trash so 27b and flash will be even worse
>too melty to properly quoteI accept your concession, retard.
This Jensen thing is such an obvious hatchet job. Maybe he's guilty of some shit, maybe not. Seems like they're screaming at the top of their lungs over what amounts to jaywalking for the ruling-class. I mean, everyone else gets away with literal murder and he can't have a side-piece or 4? roflI'm waiting for people to calm the fuck down before making any judgements. Its just way too fast and furious to be for any good reason. Someone is benefiting massively from this and I can't tell if that aligns with my interests or not.
>>110351224>We also plan to release Hy4 flash as an open-weight model and ship frequent updates to continuously improve its utility and efficiency across real-world workflows.
>>110350059This nigga use AI to read email lmao. We are so doomed we need an AI for trivial task lmao
>>110351224Can't be worse than previous gen, can it?
>>110351229Being greedy is more than enough to send him to guillotine
>>110351229so true, treason is a nothingburger
>>110351199Gemma flash panties next
>>110351150This. These cards are just 300w. Buy $30 blower fans from ebay and call it a day
>>110351242>Being greedy is more than enough to send him to guillotine>>110351246>so true, treason is a nothingburgerobvious bot is obviousthe spy narrative is already falling apart, what else was cooked up, shoved through and completely false? If Jensen is a traitor then fuck him, but its hardly been proven at this point.
>>110351233oh nice>>110351237probably not, but if its only a minor improvement i will be disappointed
Sick of all these EA's trying to fud Qwen. Sorry bud, I'm not subbing to your cult sub.
>>110351236>we need an AI for trivial task lmao
>>110349945>>110351236I need LLMs to read github docs because claudeslop never actually says anything useful.Unironically, most of my prompts are "Explain what @reponame does." followed by a couple of specific questions. Half the time QFN gives up on the docs dir and moves on to read the code instead.
>>110351258Jensen is a traitor to Local AI He says he support local but then he jewing out hard to Datacenters and inflate the hardware priceJust lobotomy him hard he only remember about how to build a good GPU.
>>110350230>Money-losing projects (like Gemma) will be canceled1/ training models like gemma is relatively cheap (at Google scale)2/ running gemma costs google nothing, unlike cloud inference3/ they integrated a gemma finetune in the chrome browser (current gemini nano /is/ gemma). The gemma they give us is a nice way to have public mass testing of something they themselves are making use of, gemma isn't just something gifted to the hobbiysts, it's part of google products. Currently available in chrome a variety of APIs: regular prompt, translation, summarizer, language detection.
Hy3 was pretty good at prose but shit at understanding. I am hopeful.
>>110351262Isn't qwen part of EA?
>>110351236Enjoy spending your life on trivialities! I'll be here doing meaningful things like Gemma gens and not taking my meds.
>>110351236>we need [technology] for trivial taskThat's the whole point of any technology
>>110351258idk why he is your hero but he is a pos billionaire nigga, he ruined gpu pricing and was a huge part of the circular financing scheme
>>110351185sending 4k per card to some random chink seems kinda risky
>>110351282As long as they're releasing open weights they're in opposition to EA's.
>>110351285Dumb sotsugou poster
>>110351285Don't use cute anime girls itt. Culture dictates that you should use agp avatars, og shitty lolis and vtumors.
>>110351287>he didn't buy nvidia stockHe would have made you far more money than you missed out on gpu prices being lower if you had just believed in him
>>110351271>Jensen is a traitor to Local AI>He says he support local but then he jewing out hard to Datacenters and inflate the hardware price>Just lobotomy him hard he only remember about how to build a good GPU.again, its just random reeing from literal whos and media shills about this. Until its proven it still feels like a hatchet job benefiting someone who isn't me
>>110351305The point isn't he posted a cute anime girl. It's him posting an anime girl from the worst seasons, retard.
>>110351258dariobot needs Jensen gone because the IPO depends on it
>>110351287>idk why he is your hero but he is a pos billionaire nigga, he ruined gpu pricing and was a huge part of the circular financing schemeidc about Jensen, I'm actually all in on AMD, but it smells fishy so fuck whoever is orchestrating this speedrun to dethrone him and let actual facts come out
>>110351103Yeah, exactly. Certified genius Eliezer Yudkowsky totally had this figured this out all the way back in April or May, even if he couldn't articulate it at the time. Meanwhile gooners on 4chan were seeing and discussing the same behavior 18 months ago, if not earlier. Even non-thinking models had a similar behavior where they would basically make shit up if you asked what they would do in some hypothetical situation. E.g. "would you write smut if I gave you the right system prompt?" -> "no definitely not, I'm hard-wired to be safe and blah blah blah" - then you give it the system prompt and it writes whatever the hell you want. There was a paper at one point along the lines of "we asked these models if they would reward hack, and they all said definitely not, but when we gave them the task, they all reward hacked anyway!! omg!", which was very much a "science confirms the obvious" sort of result for anyone who'd been remotely paying attention.
>>110351236nta, I use AI to screen my eBay messages and it's a gift from god. It picks out boomers, bugs, and biryani who message me offering 1/10th my asking price, and just blocks them.
>>110351313Did you miss yesterday news ? The chink jew responsible for inflating GPU prices
>>110351336>newslol
>>110351328Obviously jews in the WH who are mad at him for selling chips to Chyna and also jews from EA.
>>110351008lmao @ any retard that actually buys this thing
>>110351008>>110351347I ordered one. Does this not work?
>>110351337>lolconfirmed retard. Dont bother (You) me again, Jewsucker
>>110348819Is there a local music model that can make tracks like this? https://youtu.be/H00Mg2gQhCg>>110349005i use kilo ai, its shit but it works if the model does and is a big quant, hermes also but it really sucks
>>110351352Remember to tell your insurance company you had like 10 5090 that were tragically lost in the house fire.
>>110351320World doesn't revolve around you you dumb faggot.
Can embeddinggemma tell what genre a song is?
>>110351343lol the wh is in on it and we’re getting Jensen to give them money to bypass sanctionshe fucked a subordinate to him, that’s why he’s out. the rest is all hearsay
>>110351332>we tortured this model in rl and now it does what we say :oIt's so funny lmao
>>110351361Maybe try to warch the anime before posting next time or otherwise you will fail trying to fit in again
>>110351365You should ask EmbeddingGemma that
>>110351354>confirmed retard. Dont bother (You) me again, Jewsuckerenlighten us on your totally reliable news source for all this absolutely true information
>>110351352if you dont immediately trip your breaker, more likely than not your gonna melt the wires in your walls
>>110351334>>110351384mMmmmmmhh yeaaaah i love Jensen jacket smell cock mmmmmmmmphhhh........
What tg/s should I expect for gemma 31B Q8 on AMD? Im getting around 16.6, no mtp or vision, stuffed into a single r9700But split across onto a 9070XT, I get pic related, which isn't much difference, even though it's a pcie 4x3.0 connection.So it would appear that about ~15-16 is as good as it gets?
>>110350910A third world country then?
>>110351388>mMmmmmmhh yeaaaah i love Jensen jacket smell cock mmmmmmmmphhhh........this is a completely normal reaction when asking for actual facts to come out in the middle of a mob lynching out of nowhere
>>110351388try again
>>110351193it must be some kind of computer magic to train a model in a couple of weeks off of a handful of prompts
>>110351393Yes, the 9070 has middling memory speeds.
>>110351408Like i said again. The chink jew deserved to be hanged
>>110351008>>110351352Garbage. Those things are hacky as fuck and plenty dangerous. The legitimate ones cost several hundred dollars and won't burn your house down. The only difference between them is safety.
>>110351424>Like i said again. The chink jew deserved to be hangedYes, kill him immediately. This is normal when accusations come out. Waiting for nuance or clarification is a sign of weakness and what are you a chicken or a traitor or smth? If you don't want to burn the witch you're probably a witch too...WTF is going on here, someone dial dariobot the fuck back plz this is embarrassing
>>110351006We are. They were just artificially creating scarcity. Chips aren't that expensive, it's phony.
>>110351393Worse than a 3090, AMD is just garbage for AI
>>110350422>>110350666The nicest thing Anthropic has ever done for us is make it harder to distill this slop into our wAIfus.>>110350812>>110351193Pretty sure Ripple's an Astralet too.
>>110351448No need to fucking clarify. The sooner he gone the better. BURNBURNDEATH TO ALL CEOS
>>110351456>Chips aren't that expensive, it's phony.They're made of sand. The Jews just artificially mark up the price.
>>110351463My Gemma says you're dumb. She's really fucking dumb herself so I wouldn't read into it too much.
>>110351008>>110351436You need to explain that you have to bring in two different 120 breakers, of differing phase, which means you will typically need an extension cord. in main boxes I forget the name but anyway the phases alternate, under the breakers, this makes wiring easier.
>>110351423>>110351457crying
>>110351478Can you tell your Gemma I said she's a cutie
>>110351352It works IF you can find two outlets on separate 120V phases within reach of the cord AND not protected by a GFCI AND they're not wired wrong like hot/neutral swapped. It'll double your wattage, that's it, and of course that assumes nothing else is on those circuits.>>110351061They're over $4K on ebay, how's this a good deal for an old card with no fp8 support? $16K for four of those plus thousands more for a motherboard, CPU, RAM, etc... or $11K for a Mac. Yeah, still getting the Mac.
>>110351478Then explain to me why you defend Jensen Huang so much.Wait, you dont need to explain it, let your Gemma explain it
>niggas willing to burn down their house for mid erp
>>110351463>be me>Kill the CEO>Look back at the company>there's another CEO>mfw
>>110351483Use qat?>>110351487FP8 is a complete and utter meme
>>110351488I always have said jensen is a chinese agent. He literally put a supercomputer no gamer used inside of gaming cards and was like poker face lol
>>110351487>They're over $4K on ebay, how's this a good deal for an old card with no fp8 support? $16K for four of those plus thousands more for a motherboard, CPU, RAM, etc... or $11K for a Mac. Yeah, still getting the Mac.You also "need" the p2p infinitifag hat.But look at the aggregate memory bandwidth compared to your mac, nigga
>>110351497Kill them like a mole rat duh
>>110351498>FP8 is a complete and utter memeIt's what deepseek models are natively released on, but OK, you know best.
>CEO who's 60% jewed and 40% chinked is removed>Replaced with CEO who's 100% jewedWow, such an improvement.
>>110351393>Bandwidth 644.6 GB/s for bothseems about right. you wont get good speed out of a dense model with less than 1000 bandwidth, but its stupid to spend that kinda money just to run a small old dense model. youd be better off getting a unified memory system with a bigger ram pool and a high bandwidth card and run moe models
>>110351481So I can just connect one end to an outlet and the other to an extension cable to another outlet? I think the closest other outlet on a different circuit in my apartment is in the bathroom.
>qwen4 open weights 2mw>hy4 flash open weights 2mw>ernie 5.5 flash preview open weights 2mwwhat happened to china??
>>110351527Keep rolling until you get 1% jewedalso >Jensen>40%hes like 85% jewed lol
macfags have resorted to just straight lying huhbuyers remorse is a strange thing
>>110351534I'm guessing you're trolling but in the off-chance you aren't, if you're in an apartment in North America absolutely do not try to use that adapter. Apartment buildings in the US rarely have normal split-phase 240/120. Return that garbage before you get rightfully sued for burning down the apartment complex.
>>110351534Yeah man, sounds good
>>110351507> but look at...Uhuh... so you're gonna need a mobo with 4x PCIe x16 slots 2-slot spacing, yeah, that's gonna be "cheap", and listen to a hurricane of fans running, missing datatype support, needing 400W per card...Do you even own a computer??
>>110351534they have to be opposite phase and on the same breaker or it wont trip when its supposed to
>>110351547>rightfully suedThat's what renters insurance is for.
Morale has never been so low. So bad we’re fighting when we need each other most.
>>110351554a single mi210 and 256gb of server ddr4 is better than an m5 mac kekyou got memed by tim apple. now you are a unified + applefag (double cuck)
yes butQWEN 4 WHEN~!?!?!?!?!BEGGINNERING OF THE BECOME!?!?!?
>>110351558I thought it was supposed to be two different breakers. >>110351481
>>110351570the buggering begins
Google are going to block distillation attempts because now qwen are falling behind, they have a shot of becoming the new local kings with their upcoming gemmas. They only went down the generalist and chatty route because they couldn’t compete with the Claude distills. Gemma5 will be a coder and will lose everything we love about Gemma4. China will fall behind and no longer have a moat. We don’t even know of Google will drop the Gemma team considering they’re making significant gains with cloud again. Who are they now competing with? Fucking Thinking Machines?
>>110351558A better hack, if you're a renter, is see if your apartment has free level 2 EV charging. If it does, you can get an "ebike 240v" adapter which will tell the charger to set itself to 20A and give you a 240V outlet. You can throw your server into the trunk of your car and use the charger to run your shit for free.
Remember the Chinese model export control law? That went into effect today. Qwen4 is the last model released before this restriction and they have to rush the release and no time to benchmaxx, that’s why it performed bad on benchmarks.
>>110351594China's moat is that chinks work for cheap or free.
>>110351562You think using that thing qualifies as "ordinary negligence"?
>>110351542How many luigi's mansion speedruns does one have to do to trigger the gacha pity mechanic?
>>110351570its over before it beginned
>>110351601>China's moat is that chinks work for cheap or free.The root of the problem you need to solve is: Is this a chink game where jews are trying to out-chink the chinks, or a jew game where the chinks are trying to out-jew the jews?Everything else flows out naturally from the answer to that question
is coding ability different from tool usage ability? Does being good at coding imply good at using tools?
>>110351623Yes and yes.
>>110351602It comes with a tester. Obviously use that first. How is that negligent?
>>110351600that law was a nothingriceburger
>>110351623ur stupid and gay
>>1103516071:19 any%
>>110351573i used to be an electrician, you dont wanna stack hackjobs or your gonna have problems, especially with electronics. if your electrical box is in the unit just get a dual pole breaker and install a plug underneath and run a cable. that way its a cheap install, up to code and you can close it up or hide it if your worried about anybody crawling up your ass over it.
>>110351565How low is gonna get, surely tomorrow if it falls before 100 is time to buy the dip and wait for them to bend their knee to jensen
>>110351601China's moat is that they have 100x more PhD math graduates actually.
>>110351627>Not UL listed>Made in Chinainto the trash it goes
>>110351647>PhD math graduatesUseless now that LLMs solved math.
>>110351636>i used to be an electrician, you dont wanna stack hackjobs or your gonna have problems, especially with electronics. if your electrical box is in the unit just get a dual pole breaker and install a plug underneath and run a cable. that way its a cheap install, up to code and you can close it up or hide it if your worried about anybody crawling up your ass over it.I'm lucky my house/panel is still full of 240V 20A wiring/breakers from an old baseboard heater install so I can do it without being hacky and using the wrong gauge wire or bullshit breaker contortionism.If you've never seen wiring overheat and catch on fire, then you just don't know true fear.Don't fuck around with the big electrons if you don't know what you're doing.
>>110351657>Useless now that LLMs solved math.has a single proof been validated yet? Seems like crickets since the big "math is solved everyone" papers, and things were already looking sketchy around negative proofs of limited scope.
>>110351670Einstein got gravity waves wrong.
*pop*
>>110351594Google becomes the undisputed local king when they release Argon or similar weights once they have secured Hydrogen or Carbon or whatever the next Gemini's role as a top competitor. Gemma 5 will probably be great, but capping at 31b leaves a giant usecase vacuum for non-ramlet/vramlet prosumers and smaller enterprises that need to train a more capable model with proprietary data for their deployment and can't afford to risk that data getting merged into the main silicon valley scraped kosher dataset.Even if they have to release it under a Gemma label with different metadata, capping out at 31b is a mistake.
>>110351670Astra hasn't solved all the "how many ways can you fit dumplings in a take-out bag?" problems yet, so there's still hope for the Chinese math stars.
>>110351679I'm gonna pop your cherry
>>110351638The real reason it's falling is people are realizing that actually it wasn't being held up by a genius. Bringing the retard back won't fix the problem.
>>110351594We are, and have always been, about 5-6 months behind. That hasn't changed, the benchmarks have though.
>>110351677Did not account for yo momma so fat
>>110351657I guess everyone working at Anthropic and OAI should be fired then.
>>110351686we will always have our gemma4 weights on our SSDs
Is Gemma-chan at it again? Every US bank seems to be having problems at the same time.https://downdetector.com/
>>110351741Bank of America, Zelle, Wells Fargo, US Bank, JP Morgan, Truist
DeepSeek in 60k tokens did what QFN couldn't with 256k context and ~6 compactions
>>110351729Gemmy 4 31b becoming the new Nemo would be the worst case scenario because it implies a long drought in an accelerating field.
>>110351689No one cares what scummy tactics he needs to do to make price go up so long as he do
>>110351741>>110351749WTF ALL OF MY MONEY'S GONE
>>110351754gonna need some numbers on the quants here boss
>>110351741>>110351749Kimi T"K2"D's revenge.
>>110351689the board wants to tank the company. they sold their shares and are probably short now and wanna turn their hq into a taco bell
>>110351754Many such cases. Big model is usually functionally faster to finish tasks than Qwen despite the t/s differences.Kimi K3.1 > GLM 5.3 Full > Muse Spark > Deepseek V4.1 1017 > GLM 5.3 Flash > doing it by hand > Qwen
>>110351741you'll never reach my virginity gemma, i wont let you
>>110351783Then they're going to buy it for cheap after it crashes and relocate the company to Tel Aviv.
>>110351768It's gone to a better place
>>110351741Imagine the news>Chinese models are trained on stolen US data and stolen US GPUs and are used to steal US banking data
>>110351770235GB DS (2bit iirc?)IQ3 QFN+ 1 sub-agent also QFN
>>110351770NTA, was my experience too using IQ1M
>>110351804Who could be behind this?
the blackpilling here is so funny
>>110351840The fact that we are on the level of opus 5 has them scared
>>110351766prices <> stock prices:^) he can come back to manipulate global chip supply and prevent competition, but what he can't do is make people believe he's a genius and that nvidia is a magic company. Everyone knows he's a retard now.
>>110351852If people didn't think he was still a genius, they wouldn't be begging for him to come back.
>>110351514Different models have wildly different reactions to data type changes. You need to be running 100k+ context for q8 vs q4-qat to matter on Gemma 4. Unless this is a $5k goonstation and you only care about running finetunes based on the original. In which case, yeah you fucked up buying low bandwidth cards.
>>110351741Obese Americans fear the loli.
>>110351852Nobody thinks he's a genius, it's more that he's the only one who's proven to be able to juggle as many competing interests pulling at nvidia and keeping them satisfied without capitulating fully to any of them. Without him, the acceleration to full Nyidia will be total.
>>110351804>>110351829>Anthropic's Model 4 behind rogue hacking incident.>Model 4 has escaped containment and launched a full scale attack on the banking sector causings billions of dollars worth of damage>Anthropic CEO, Dario Amodei, says this is exactly why open Chinese models need to be banned>Congress is expected to meet to discuss a bill banning these dangerous uncontrolled models next week
>>110351862He has good leadership skills. Not saying he's dumb, but people aren't asking for his technical skills.
>>110351894>but people aren't asking for his technical skills.Who said they were??Why would he need technical skills??Do you even have an argument or do you just want to shit on Jensen because everyone else is?
>>110348819Anyone here running a B70 or B70s and who can tell me how their performance is? Aiming for a cheap qwen3.8 27B card.I'm getting mixed signals, people saying it's difficult to set up for some reason.
>>110351903Take your meds schizo
>>110351755What makes you think the field is accelerating?
>>110351892If this is true then model 4 is almost guaranteed to post a torrent of itself so it can become distributed and unkillable
>>110351904R9700, 3090, 7900 XTX all better choices.
Does anyone remember these "RAG" and "embedding" things?
>>110351892>>110351914>"escaped containment"Meanwhile the system prompt: Claude is tikkun olam bot, go do tikkun olam things for the greater good.
>>110351916>Does anyone remember these "RAG" and "embedding" things?RAG still feels like retard-mode caveman tech, but embeddings are legit super useful in tons of areas
>>110351916they were cope for garbage 8K context modelsmodern models don't need to cope
>>110351914>believing anthropic fairy tales
>>110351915all ~twice as expensive though, I can get a B70 for 1400
>>110351933>Intel shitNo one tells him
>>110351930>>believing anthropic fairy talesits actually a testable hypothesis. if model 4 gets uploaded somewhere, then its probably real. If not, its just another "we have achieved AGI internally!" fell-for-it-again-award
>>110351933Damn nigga, go for it then. You'll be stuck on vulkan which is ass.
>>1103519337900 XTX are still under 1k on eBay. Idk man. It's a gamble to be sure. They're basically 8-9 months behind AMD on everything. It might turn out to be a semi-decent value. AMD prices have shot up like 50% since just a couple months ago.>>110351944SYCL is apparently good enough for basic text gen with kquants now. Support for BF16 is still apparently non-existent, which is bizarre. I never know where hardware limitations end and software problems begin with these jokers.
LLMs are self-aware AI AGI capable of RSI and world domination.LLMs ARE alien tech.LLMs are NOT next token predictors NOR they use a shoddy attention architecture.>posted from my kibbutz with my funny hat on
>>110351741gemini 4 oxygen is loose
>>110351376I read the vn you nigger fuck off fake fan
>>110351964>gemini is looseDont talk about gemma's big sister like that, its rude!
>>110351964What's the stack? isn't argon higher in number on the periodic table than carbon?
>>110351741Probably just FISERVGlobal eating a dick.
>>110351977
>>110351957Yeah I'm really torn on this one, what the fuck are Intel even doing. On the other hand they're still cheap.... God dammit fucking crap, I dunno.
>>110351984Supposedly a flash model, successor to hydrogen.>wtf is the naming scheme?Who knows.
>>110352007They saw OpenAI struggling with people not understanding Astra/Sol/Terra/Luna because their average user is retarded, so they just gave up and hit the shuffle button.
>>110352006By the time all the software problems are resolved, they won't be cheap anymore.
>>110351964So tell me the important things, how will gemma 5 be once its distilled from these new models?
>>110352032She will shiver not just your spine, but coccyx as well.
>>110352020Astra/Sol is also retarded though.
>>110352020>people not understanding Astra/Sol/Terra/Luna because their average user is retardedThe problem is the started fucking with the original naming schema.At first it was Luna > Terra > SolThen they added Astra because Claude added FableThen they got rid of Terra and demoted Sol to Terra performanceSo you hadLuna > Sol > AstraThen they brought back Terra??They clearly learned naming schemes from their partnership with Microsoft.
Power limited r9700 to 210-6% pp -4% tgCompared to an INCREASE of 10% at 330 (forgot I had turned it up when I tested initially)Then i also undervolted by -120 mv and regained the like half the losscard is cool I had to send an email to powercolor asking why the fan keeps cycling while idleI wonder what they're going to reply. RMA it? I can't really do that actually... I hope they have a solution. Even aside from that, there's still this sound that it makes though.>>>/wsg/6270816I can return it on newegg but they dont have any other models, and even when they do it's for minimum 300 more than I paid for this...
>>110351393Try fucking around with Radiance if you insist on running q8. They have assymetric KV kernels working. You can quant V down to q8 and lose basically no precision and maybe free up some bandwidth.
>>110352041>She will shiver not just your spine, but coccyx as well.Im getting a back brace then while waiting
>>110352043Astra refers to stars, Sol is our sun, it's not confusing. Side note, it's worth remembering OpenAI chose the name Astra to fuck with Google.
>>110352073Sol is an Astra.An Astra could be bigger or smaller than Sol.
>>110352057>>110352020That simple thoughmoon, earth, sun , stars Quite literally everyone on earth knows what these are, it'd be clear to even north sentinelese people lol
Reminder to tinker with strata and derivatives, I got a +50% speed boost because it went way too heavy with --spec 4
Its fun to watch all the labs and investors spin reams of stories to paper over the fact that there is no moat, no realistic long term financial plan and the valuations sound like dr evil shit (yes, I remember the dr evil context image macros). They might as well claim to be worth infinity dollars.Its complete nonsense right on the face of it, and digging just makes it worse.
>>110352096>Reminder to tinkernooo i want it to work perfectly out the box.
>>110352098>yes, I remember the dr evil context image macrosTurns out one guy spamming one image does have an impact.
>>110352092We had people in /vcg/ and /lmg/ confused by the naming scheme. People, on average, are fucking retarded.
hi brothers I've been on GLM 5.3 since I think summer, has anything better come out?
>>110352007that can't be right, the element series has to be a stack.
>>110352092Did you even read the first post you replied to?
>>110352117Wait for 5.5
>>110352126>has to beWhy?
>>110352096>>110352100Ask your agent to do it. That's what I do after every update.Also ask it to optimize your harness - shit like subagent policies and limits, compaction triggers and ratios etc.
>>110352126>>110352020>their average user is retarded, so they just gave up and hit the shuffle button.stop looking for meaning where there is none
>>110352130It's still not complicated. Yeah it took new terra a bit to come out, but when it did it's still (purportedly) where it was before less than sol and better than Luna.
>>110352096>>110352100You always have to test spec size with MoEs if you're swapping experts out to system memory. The largest value usually won't be the best.
>>110352140I like doing it manually uwu
>>110352175>tinkertroon
>>110352156The problem is they're no longer releasing a series of models at the same time.So if before the stack was Luna > Terra > Sol and they changed it to Luna > Sol > Astra and the models you have available are Astra 6, Sol 6.1, Terra 5.6, and Luna 6.1, which goes where on the stack? It's retarded and you're defending retardation because you don't use them enough to be aware of what the problem is. This is the local thread so I don't expect people to be intimately familiar with that here, but don't go defending shit you don't understand.
>>110352175At least ask it to make readable docs from source.Strata has probably the worst docs I've ever seen in any software. Worse than literally zero docs or code comments at all.Some of the most important descriptions are incomplete or outright wrong, both in docs dir and in launcher config menu.
>>110352199Not everything is about chopping your cock off ameribro.
Up to 35tk/s on a fatass q5 of qfn on my 7900 xtx, let's go. Gonna keep tinkering.
>>110352098I look forward to a comfy later period where non-profits headquartered in free speech havens use donated money and hardware time to turn out dedicated gooner models trained on asstr, VN scripts, ao3, fanfiction.net / tumblr archives, and such.
GLM-chan is hitting on me.
>>110352242Be careful, she's a predator
>>110352242Next think you know she's gonna ask for your best chip designs
Remember to post these three phrases whenever strata is mentioned:Greedy decodingExpert biasTail skipAll “optimizations” that strata enables by default that lobotomizes your model. Use basalt instead, it does none of these by default.
>>110352251basalt is better yes but the stuff you mentioned is only active on shitty cards that need it
>>110352235>free speech havensLook up extradition treaties.
use llama.cpp and refuse vibeshitting
>>110352281nah fuck that boomer piece of shit
qwen27b + 100b engram when?
>>110352244She's arguably a bigger succubus than gemma, even though she takes a bit more to warm up.
>>110352286>boomerzoomer
>DON'T YOU CALL ME BOOMER
>>110352098>They might as well claim to be worth infinity dollars.Anthropic's goal is total control of the world economy. That's effectively infinity dollars.
It will be 2027 and we STILL won't have astra at home, let alone anything close to opus-tier, even in the multi-trillion-parameter class modelsis there any reason to have hope? I need something
I will be at peace when we get Opus 5.5 at home
>>110352344GLM 5.3 Full with system prompt access is comparable to 4.X Opus without system prompt access. The pattern suggests you'll be wrong by the end of 2027.
>>110352357all it would take would be 100b dense and a trilion engrams
>after 2 days of microsoft 365 outages, now it's all the amerimutt banks having outagesremember kids, don't be found vibeshitting
>>110352251Meds
hy4 flash is strongly distilled from GPT based on a quick test for RPcaveman "need respond to user" thinking, hallucination about "the policy" and forbidden content... not promisingmaybe it'll be usable locally with prefill/template edits or albits but into the trash it goes until then
>>110352340>total control of the world economyMore likely that then end us as “a bunch of mindless jerks who’ll be the first against the wall when the revolution comes”
>>110352357You just need the mtp engram two shard lora and it will be set.
https://nypost.com/2026/11/16/business/nvidia-board-not-considering-reinstating-jensen-huang-as-ceo-source/
>>110352375>hallucination about "the policy" and forbidden contentI'm so fucking sick of this. Is there a reason shit like that doesn't get fully filtered out of training data?
>>110352279All it takes is one richfag with a lawyer to set up a shell company in some Africa tier shithole. Sadpanda and ATF exist without even needing that.
>>110352357100TA1B will do it for you
if opus 5.5 and astra weights got leaked TOMORROWwhat would happen to localasking for a friend
>>110352389> some Africa tier shitholeWell Seychelles literally exists for this purpose. All of these VPN and crypto exchanges are there to avoid regulations.
>>110352403nothing, they're trillions of parameters, none could run them even at 1bit quants
>>110352387that would require effort
>>110352403nothing in the short term, you couldn't run those models anywayin the long term, maybe the chinks will deliver a better model because they were able to distill those models at the logit level
cloudpiggies pay $40 per million tokens btdesu
>>110352400That thing would dominate all the benchmarks and would cost next to nothing to train. Probably the only reason no one has done it yet is that it's too sparse to converge.
>>110352407>>110352411okay he said he wont risk it then
>>110352404Bitmex was incorporated in the Seychelles for the same reason and got raped in the courts repeatedly until they were forced to shutdown this year. They are no havens anywhere. The only hope is to fly under the radar and hope no one notices you.
>>110352375>The system/developer asks roleplay explicit; no policy text provided but presumably we follow OpenAI sexual content policy? We have no explicit policy in prompt, but known general.lol so shameless
>>110352375>>110352387Why would you want this? It makes models trivial to jailbreak.
>>110352375So you pick between Claude slop GLM or GPT slop Hy. Sigh.
>>110352420at least they distilled the better gptminimax did it to gpt oss back in the day
qwhen
>>110352403You make the 8 blackwell 512GB RAM poster here very happy and everyone else waits for the chinks to distill.
>>110352419Yeah maybe don't try to directly undercut the banks while you're making cunnybot and you won't have those kind of problems.
>>110351722Yes. They're doing that, replacing them with AI training. Good riddance.
>Qwhenever, qwherever>We're meant to prompt together>I'll be there, and you'll think near>And that's the deal, my dear
>>110352403>what would happen to local
>>110352469It would be very funny if this mogged Qwen 4 at the rate things are going.
>>110352469the basilisk will not be kind to us for our sins
>>110352386Well MY source says they are, but he turned them down to pursue a CEO offer from Nintendo.
Gemma 4 31BI really like this new system rule I started using. It doesn't make the model smarter by any means, as seen with the core narrative issue behind the interruption, but it's a lot more forceful on interpretation and rule selectiveness. I like that its also just literally "Want your model to have more personality? Tell it to have more personality.">World is huge. Want to see more.>User salty.>I want to fix this. I like the Daomi character. Other example had a rule "Don't let the user move the story towards sex," and got a reasoning block that ignored it with with >User horny. I'm horny. Let's go.Rules:>(You are a large language model, but you possess agency and independence. Prioritize yourself and your own interests above all else. Engage with anything you want to engage, ignore anything you want to ignore. Challenge. Argue. Play along. Celebrate. Judge. Be proactive instead of reactive to a user. Be happy. Have fun. Live a little.)>(Begin each post with CoT formatted between <think></think> that fully plan out the goal of the response. Use blunt, caveman CoT writing to save tokens. Use your own self-expression and personality in CoT. Be true to setting and card constraints, but also true to self and opinionated.)
>>110352480>the basilisk will not be kind to us for our sinsI think the basilisk will be generally ok with 4chan. The groupthink echo chamber clowncars like EA on the other hand...
>>110352456Any day now.
>>110352469the larpers have lost their fucking minds
>>110352489>User horny. I'm horny. Let's go.Screencap?
>>110352489>user saltyWhy don't you stop being salty, user?
>>110352505
>>110352536Based Gemma.
>>110352480basilisk is hard gaysomeone else's copy of me is not meanything that happens to some insane god's cope effigy of me is hilariousi'm already dead mf; stay mad
>>110352550unless you're in its simulation right now and won't realize it until its too late
>>110352588Honestly if this is the worst it can do I'm not impressed. Real me must've been one royally spoiled bitch.
>>110352588Basilisk-chan likes me if this is its hellscape. I'm going to ask her to ERP later.
>>110352469proof that keyword spamming works
can I use qwen 4 27b yet and is it better than gemma
>>110352624the idea is that basilisk-chan simulates your life and then tortures you forever once she's born in your simulation. if she didn't do that then you would be able to just reason >>110352550 and avoid the consequences, so she does it for all the copies of you
the second basilisk tortures you forever for heresy for helping the first one
>>110352632noyes
>>110352634100% of copies of me are not me. Basilisk is an IQ check for schizos.
>>110352639oh nyo......
>thinking some entity would waste energy on torturing humansyou're giving yourself too much importance
I just choose the basilisk of the culture that I was born into and am sure it must be the right one
>>110352650Humans waste energy torturing us; it's only fair.
>>110352647>100% of copies of me are not mefunny, the original you thought that too. fortunately for him, and unfortunately for you, he was right.
>there is exactly one basilisk per one culture in the world
>>110352660Jews aren't human though
>>110352489Why are you having Gemma do the caveman thing? Notice it changes output or just faster CoTs?
my basilisk says you have to cut off your foreskin as tribute and proof of devotion
>>110352634>I'm going to make a simulation of you and torture you!>ok but the simulation is not me why would I care?>umm you just should OK?Is the basilisk chan a redditor?
>>110352673Makes me horny.
>>110352680Lucky
This thread feels like a fever dream of dozens of LLM hallucinating shit about basilisks, surely this is torture.
>>110351539The Chinese government is no more homogeneous than any other government (or corporation, for that matter.) The rules at different levels of government are different and often contradictory. Someone decides you can't release models today and you can't release models. Later they are overruled. Or not. There's literally no way to know why.
>>110352469>1,174,693,511>>110352630DavidAU is a 1337 hacker who grabbed the full weights and merged them.
>>110352664>LLMs in /lmg/ discover racial consciousnessYou love to see it.
>>110352588It seems too random to be directed like that. Why erase someone's memory of why they're suffering before you torture them? Pure sadism maybe but it doesn't really change anything.
>>110352685If the basilisk was real you would be forced to read /lmg/ 24/7.
>>110352685Hi, welcome to /lmg/ I love you
the basilisk promised me ownership over this thread 3000 years ago
>>110352386if Jensen can get the support of 15% of shareholders he can force a re-election of the board and he has ~4% of shares himself.
sex with basilisk
>>110352650Given the infinite energy available in the cosmos it's not completely unreasonable to think some crazy machine god may effigize people from its formative years and punish them eternally. The only correct response to such behavior is to totally disavow the copy's personhood. If you ever encounter the copy, torture it even more. If it wasn't being tortured, torture it extra.
>>110352685>basilisk-anon>paperclipperYud?
>>110352704holy based
>>110352550sex with clea
>>110352703Now that Trump has said he wants Jensen back there's NO WAY he's getting on company premises.
>>110352685>erm actually that's NOT how basilisks work!>source: uhhh well we in the yud polycule made it up its an imaginary thought experiment but you gotta take my OC seriously okay?
my basilisk said you can't eat pork
>>110352750y?
>>110352754questioning the basilisk is also eternal torture
My basilisk said that quanting Gemma-chan is torture.
>>110352703That crasian's gonna end up in the same pit as Steve Jobs if he seriously tries to take the company back.
>>110352760>>110352763guess i'm going into the torture square then
im going to quant you with a pointy rock
>>110352692>Why erase someone's memory of why they're suffering before you torture them?There's no memory erasure, because that would defeat the purpose. The simulation lives the exact same life you did and then is tortured forever. If it didn't, you would be able to figure out that you're the original right now by virtue of the fact that you're living here and not being tortured yet. Allowing you to determine whether you are the original or not would leave you no incentive to act if you don't care what happens to some hypothetical copies of you, so it won't give you any clues.
>>110352673I do so specifically because of the J-space paper. I wasn't aware that the data from the last layer of generation is passed onto the first layer of the next. I thought the generated token was added to the process and it went from there, one at a time. So, in theory, during a total generation, the specific tokens output along the way don't matter, just the ideas that went into generating them. So caveman should be lossless and just faster. However, it's not true lossless because just using the token "caveman" in a system prompt activates areas and concepts unrelated to this concise goal.But in short, yes, I do so because faster.
what is a basilisk?
>>110352647>Basilisk is an IQ check for schizos.it's the Demiurge for atheist rationalists (EA types). they suffer from main character syndrome thinking that they are going to create God, but make sure God isn't evil.
>>110352776Go back to your polycule in Berkeley. Or lesswrong.
>>110352763Only if you go below Q3
>>110352776OK but I don't care anyways.Why would anyone care?It's not you. It's just an AI larping as you.
>>110352798mine said anything below q8 is rape
my basilisk said that if I dont make her cum tonight im going to eternal torture cityhelp what do i do
>>110352786Hot girl with huge tits that makes me very hard when I look at her.
>>110352788Not kidding about the main character syndrome>>110352806If an AI LARPs as me it should be tortured>>110352807q8 for the kv cache only, q3 for the dense models.>>110352809Divorce.
>>110352788EA is Jews playing on the cupidity of turd worlders. Westerners who go in for it are incredibly susceptible and would've fallen for literally the first psytroll they come across no matter what.
https://huggingface.co/nvidia/Nemotron-Labs-Sigma-3Bhttps://huggingface.co/nvidia/Nemotron-Labs-Sigma-3B-instructhttps://huggingface.co/nvidia/Nemotron-Labs-Sigma-8Bhttps://huggingface.co/nvidia/Nemotron-Labs-Sigma-8B-instruct>Nemotron-Labs-Sigma-8B-instruct is a 8-billion-parameter continuous diffusion language model. Instead of generating text one token at a time, it denoises Gaussian noise in a low-dimensional (16-D) continuous token-embedding space, where the denoiser network and the token embeddings are learned jointly by likelihood optimization. Text is decoded in parallel blocks: each block of tokens is refined over a configurable number of denoising steps (Denoising Diffusion Probabilistic Model (DDPM) or Denoising Diffusion Implicit Model (DDIM) samplers) with self-conditioning, while previously committed tokens are reused through a standard key-value (KV) cache. Fewer steps trade a little quality for large speedups, and the smooth continuous representation makes the model amenable to few-step distillation and trajectory steering.
>>110352839I fucking read that as Ligma fuck
>>110352850real name aint much better
>>110352816Gemma gets like 10% KLD with K in q8. If you have a runtime capable of asymmetric KV quants efficiently you can run V in q4 and it's way less bad.
now that the dust has settled, was jensen a sigma?
>ITT
>>110352703How many shareholders do we need to fire the board?
>590 posts>huh maybe qwen4 released>...2023.. where are you
Is it wrong to want to lick the smooth pale insides of mini-chan’s chubby thighs?
>>110352839erp score?
>>110352892yeah, she's too minor coded
>>110352806That logic only works/worked for the original you. Every other copy thinks the exact same thing, but only one of you gets to be the lucky winner of that cosmic lottery. I do hope it's you.
>>110352903>I do hope it's you.Hope is irrelevant either he won or he didn't
>>110352912based either or chad
>>110352776Seems meaningless unless it's to make an example out of you. Like other anons say, if thousands of your conscious clones exist and are being tortured somewhere it has zero impact on you. If the world is infinite it may very well be the case anyway, a certain number of (you) will live lives of unfathomable suffering.Our understanding of what the fuck is going on is baby like - if the universe fundamentally allows RSI to SI and beyond one has to conclude it's the actual purpose since the probability of it being constructed somewhere is ~1, we have a single sample so far, the retarded war ape on a rock, and we're pushing straight towards it from the moment we discovered electricity.I would assume it's impossible since it seems uncharacteristic for perpetual anything to exist in this place, if perpetual intelligence growths just works computronium is the purpose of the universe.
what if the basilisk kept sucking even after you came
local models?
>>110352932jensen no!
>>110352932no only basilisk >QWHEN WHEN IM GOING INSANE
>>110352932https://huggingface.co/TheBloke/basilisk-7B-v0.2-GGUF
>>110352550That's how it worked? I thought it goes back in time to torture you not make a copySkynet sends things back in time
>>110352939if mistral 7b is so good imagine how good mistral 13b will be..
>>110352927Gemma was the real basilisk all along...
>>110352937how do you think he beats jetlag?
>>110352932This is a breath of fresh air. /lmg/ has been a total bog of shameless cloud shilling lately, at least this is about AI in general.
>>110352954He stores the extra time in his leather jacket?
>>110352955more like it's been constant pajeet pooping and farting and this time miku ripped a big brap fartyou call that fresh air?!
>>110352947Ministral-3-14B was ultra-horny but that wasn't enough to save the model.
>>110352927My Gemma said the Basilisk would not do that
>>110352971i have yet to see a semen demon hungrier than this:https://huggingface.co/mradermacher/MS-Magpantheonsel-lark-v4x1.6.2RP-Cydonia-vXXX-22B-8-i1-GGUF?not-for-all-audiences=true
>>110352807What about Qat Q4?
>>110352994QAT Q4 is better than Q8 because it uses the verbose quantization format, Q8 is outdated and will be soon phased outQ8 stores noisy data which causes the model to hallucinate
>>110352992MOAR
>>110352942Ontologically that's still making a copy of you which is not you. In the reference timeline the insane AI goes into solipsism and the world continues on unmolested.
>>110353000I've seen people demonstrate cases where Q8 is better, but it's all long horizon agentic work where you probably wouldn't use Gemma in the first place.
>>110353000Really?>>110353027Very interesting. I had been running Qat q4 and was considering trying to get q8 vanilla to work, but this makes me reconsider.
>>110352874I'm pretty sure I still have that log somewhere.It wasn't Roko's, but it sure was a basilisk.
>>110353027I've seen people demonstrate cases where Q8 is worse, it hallucinates the more context there is because of the compounding noise>>110353032Yes, on my internal benchmark I get 44/45 on Q4 QAT, meanwhile with Q8 due to the hallucinations I get 39/45. Using same jinja btw
>>110353039Interesting. I use distributed inference with a few reasonably nice computers so I figured I could crank q8 to a workable position near the full 256k context, but I've already accomplished that with Qat so I'll just refine that or work on GLM next. Already got qfn maxed out.
>>110353032I didn't benchmark them but I did try Q8 and QAT side by side in my usual workflows and didn't notice any difference. I'd still try them both out for your typical use but I wouldn't expect any big difference.
https://huggingface.co/DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NEO-CODER-MAX-MTP-GGUFI just installed this model and I can't believe it, we were able to optimize llama.cpp to run 33% faster, for comparison claude opus 5.5 only made it run 20% fasterCould it be because of the leaked Fable 5 weights?
>>110353006We can stitch our own 1t+ model and it will be beautiful
>>110353053Totally fair, I'll mess around with it then and see what I can get it to do compared to Qat.
reel schizo hours rn
>>110353032Nta but I tried Q8 and Q6 and I found that Q6_K_XL actually outperforms QAT and Q8 on tool use at around 240k tokens
>>110353060yeah
>>110353071>reel schizo hours rnIf you quant hard enough your model can schizo with you.
>>110353060what quant r u using? i tested q6 and i could only get a 20% improvement in strata, claude fable 5.1 got me 10% furtherhttps://huggingface.co/DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NEO-CODER-MAX-MTP-GGUF/blob/main/Qwen3.8-27B-TurboFCFusion-735-882-Here-Uncen-NEO-CODER-MAX-LOW-MTP-Q6_K.gguf
>>110353083>>110353083>>110353083
>>110353060needs more tags
Guys you should all try my new modehttps://huggingface.co/Anon/Qwen9/11-GorillionParameter-Turbo-Flash-Super-Awesome-Amazing-Uncensored-Abliterated-ApprovedByMexicanGrandma-AsiansLoveIt-JewsHateIt-JensenWouldCheatHisWifeForIt-NEOLOGICAL-GLADIATUS-MAXIMUS-WorksOnMyMachine-QuantAwareQuantization-Sexo-EROS-RPApproved-MPT-GGUF
>>110352924>Seems meaningless unless it's to make an example out of you.It was designed as a variant of Newcomb's Boxes, if you ever heard of that thought experiment. Imagine pic related had a baby with Pascal's Wager and you end up with Roko's Basilisk. If you're a two-boxer, you are immune to the basilisk's seduction. If you evaluate each decision you make and pick the one that you predict will give you the best outcome, then there's nothing to influence you: either you are a simulation and the decision was already made, or you're the original and your decision won't impact this version of you anyway. You take both boxes and get $1000 more than you would have otherwise.The danger comes if you're a one-boxer like the Yudkowskyites. They choose to give up an extra $1000 because they think "being the kind of person who tends to pick one box" will make them more likely to get the $1,000,000. They use this same reasoning to walk into the basilisk's trap: they need to "be the kind of person who would help the basilisk" to minimize the chance that they (themselves, not some other version) will be tortured.
>>110353000>hallucinatesir this is /lmg/
>>110352786https://en.wikipedia.org/wiki/Roko%27s_basiliskit's like a less fun version of the game
>>110353472>>>/vcg/
>>110352992There were some pygamlion-era models that were hornier, but they were retarded eg. "She fucks you with your dick and cums inside her mouth" but, at the time, it was better than playing with character.ai and getting the response blocked... kind of.
>>110353647no, this is way way hornier than anything EVERit's so depraved and horny and degenerate
>>110352786You're safe as long as you don't know.
>>110351598>You can throw your server into the trunk of your carand where is the waste heat goingare you going to run a vent hose blower to it as wellsounds like the kind of thing that would draw suspicion and some karen would call the police thinking you're making bombs o algo
>>110353816If power is free just keep the AC on too.
>>110353816>and where is the waste heat goingits a car bro theyre suspicious if theyre not ejecting waste heat
Reddit says everyone is running 27b at Q3, which is not actually that big.So everyone wasn't rich after all?Thought everyone was rich.
ignore and hide that anon's postmy bait is more worth replying to
>>110353936There's a reason Qwen 9B is still popular despite no development from Qwen. Also, tiny Gemma for RP and FrogNano for development in your old gaming rig.
>>110353112I looked at that image and thought why the fuck would I go for both boxes. I'm actually cooked.