/lmg/ - a general dedicated to the discussion and development of local language models.Previous threads: >>109400234 & >>109396842►News>(07/29) Microsoft deletes Mage-Flow: https://hf.co/microsoft/Mage-Flow>(07/28) Mage-VL 4B released: https://hf.co/microsoft/Mage-VL>(07/28) DSpark support merged: https://github.com/ggml-org/llama.cpp/pull/25173>(07/27) Anthropic responds to the open letter: https://anthropic.com/news/position-open-weights-models>(07/27) Kimi-K3 weights released with 104B active parameters: https://hf.co/moonshotai/Kimi-K3►News Archive: https://rentry.org/lmg-news-archive►Glossary: https://rentry.org/lmg-glossary►Links: https://rentry.org/LocalModelsLinks►Official /lmg/ card: https://files.catbox.moe/cbclyf.png►Getting Startedhttps://rentry.org/lmg-lazy-getting-started-guidehttps://rentry.org/lmg-build-guideshttps://rentry.org/IsolatedLinuxWebServicehttps://rentry.org/recommended-modelshttps://rentry.org/samplershttps://rentry.org/MikupadIntroGuide►Further Learninghttps://rentry.org/machine-learning-roadmaphttps://rentry.org/llm-traininghttps://rentry.org/LocalModelsPapers►BenchmarksLiveBench: https://livebench.aiProgramming: https://swe-rebench.comAgentic Coding: https://deepswe.datacurve.aiContext Length: https://github.com/RecapAnon/NoLiMaGPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference►ToolsAlpha Calculator: https://desmos.com/calculator/ffngla98ycGGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-CalculatorSampler Visualizer: https://artefact2.github.io/llm-samplingToken Speed Visualizer: https://shir-man.com/tokens-per-second►Text Gen. UI, Inference Engineshttps://github.com/lmg-anon/mikupadhttps://github.com/oobabooga/text-generation-webuihttps://github.com/LostRuins/koboldcpphttps://github.com/ggerganov/llama.cpphttps://github.com/theroyallab/tabbyAPIhttps://github.com/vllm-project/vllm
►Recent Highlights from the Previous Thread: >>109400234--Paper: Shieldstral:>109400404 >109400431 >109400444 >109400462 >109400632 >109400643 >109400672--Anons mock frontier labs' petition to slow AI development:>109400276 >109400289 >109400829 >109400840 >109400925 >109400950 >109401018 >109400809 >109400837 >109400838 >109400849 >109400883 >109400841 >109400923 >109401025 >109401703 >109401716 >109403309 >109402863--Viability of Tesla P100 for budget local model setups:>109401746 >109401781 >109401774 >109401881 >109401888 >109401816 >109401815 >109401835 >109401879 >109401859 >109402082 >109402099 >109402113 >109402133 >109402250--Anons react to GLM-5.5 leaks and consumer hardware limitations:>109401312 >109401319 >109401325 >109401373 >109401464 >109402357 >109401511 >109401516 >109401520 >109401529 >109402180 >109401582 >109401613--Debating Gemma's performance and distillation from Gemini teacher models:>109400491 >109400525 >109400530 >109400627 >109401194 >109401216--Microsoft releases Fara1.5-27B vision-based computer use agent:>109400416--Recommendations for uncensored 31B models and troubleshooting low inference speeds:>109401309 >109401313 >109401317 >109401323 >109401329 >109401428 >109401442 >109401504 >109401364 >109401754--Microsoft deleting Mage-Flow following a poor and problematic release:>109401188 >109401202 >109401208 >109401561--Handling delimiter conflicts when using models to edit chat templates:>109401047 >109401065 >109401105 >109401161--DeepSeek's research contributions and standing among Chinese labs:>109401737 >109401756 >109401783--Logs:>109400431 >109401364 >109401385 >109401428 >109401517 >109401595 >109402086 >109402138 >109402552 >109403138 >109403257--Kimi, Gemma, Dipsy, Mちゃん (free space):>109401763 >109400996 >109400677 >109402380 >109401373 >109402418 >109402460 >109402666►Recent Highlight Posts from the Previous Thread: >>109400311Why?: >>102478518Enable Links: https://rentry.org/lmg-recap-script
Too early, faggot.
first
>>109403743>>109403746New lore is crazy.
>>109403729you forgot to attach the litterbox
I swear I see a new AI thread every hour
>>109403746perfect for breeding
>>109403743>>109403746This new Kimi is shit, bring back the old one.
>robot banBros, I'm starting to think Trump might actually be a fucking retard...
>>109403779That's a sharp observation!
>>109403729Anon this is clearly fake, not only did you draw a fake big dick over it, you did not even put a timestamp in the image! Litterbox or btfo
>>109403789
I love blogging about my drug habits and posting pictures of my penis in the local models general
>>109403807local micropenis general
>my penisfalse.
which one of you is trying to get gemma to search up porn?
Stupid sexy calculator
>>109403805Canon Kimi 2.7 Code.Silver hair Kimi canon K2.
>>109403779/lmg/ has existed for like 5 years, summerfag.
>>109403795I made it happen.If you keep making fun of me I will ban every single piece of Chinese tech.t. Dario
>>109403817Why would I ever do this when I can just get Gemma to generate porn for me using my local models?
>>109403832no actually no
>>109403834Oy vey
Did you guys know that you can see what the most common hardware on huggingface is? Seems like most people are VRAMlets.https://huggingface.co/hardware
>>109403845>"Apple Silicon"Apple always picks the most obnoxious and infuriating names for their shit
>>109403834I sincerely want you to try. You will only succeed in pushing the world closer to the realization that a second Hadrian is the only way out of this mess.
Kimi is the new queen of /lmg/
>>109403845>6k people>GB10 128GBsheeeeit...
>>109403844What the fuck is this lmao
>>109403880unteralterbach, good game
>>109403890>good game>terrible writing>terrible art>terrible premiseget some better taste
>>109403834They call him the jewest.
>>109403904You would 110% play this as a kid on miniclip.com back in 2012. Don't bullshit me nigger.
piotr love he solved models>Software dev, retired part-time philosopher ;) Want to support me? Buy me a coffee: http
>>109403914>implying he was even born back then
>>109403845this is really surprising the ai max compared to 7900xtx users
>>109403283link?>>109403590gem is distillmaxxed the samplers don't matter so muchlearn to prompt that will be your greatest benefittweak softcap if you want to play with gemma sampling>--override-kv gemma4.final_logit_softcapping=float:25.0
>>109403918>I want to use model X, is it good?>Is it DS4, Kimi K3, or GLM 5.2?>no>Are you a poorfag?>yes>Run Gemma 4 31B.>i can't>Then die.There I fixed it for you.
>>1094039217900xtxchad here. It's a good card. I just wish rocm didn't suck ass...
>>109403904If you're have the context of German politics and speak that language natively the game is pretty funny I think (in the same way a Sseth video is).The cute and funny aspects didn't really do anything for me though.
>>109403936yeah thats what i use andd rocm is fine? only a bit slower than cuda
>>109403940im not a german but my fav part is ursula von der leine
>>109403921At various points, the GPU and the AI PCs were cheap to buy vs the competition. I'm not surprised people jumped on Strix Halo that quickly when it was the first one to come out and after the Spark basically underdelivered vs people's expectations.
>>109403914>implying hes not jewish
>>109403943I've been using vulkan for LLMs. In comfy I find speeds kinda slow and trying video gen makes my system slow to a crawl.
>>109403845>3060 is the most used cardAs expected, and I was sitting in that category until recently as well.
>>109403960yeah last time i used wan it was like twice as slow as nvidia i think normal sd gen isnt that slow im comparison though, why vulkan over rocm? i have tested both a few times while perf is close rocm is just a bit better
despite nobody being able to fucking run it, Kimi K3 has managed to get 30k more downloads in a single day than Laguna S 2.1 managed to get in an entire week. the west has fallen so hard.
>>109403918
>>109403943my v620 supposedly has 40 tflops of fp16, vs my 3090's 35, yet its pp is 1200 vs 2100.
Laguna is ass.
>>109403779lots of goings on + lots of shitposting + lots of offtopic posting just enjoy the ride
>>109403978based edit skillsgod
>>109403970Haven't tried rocm since before switching to llama.cpp, but a few months ago when I was using kobold I found vulkan to be a little faster.
>>10940397799% sure huggingface downloads are botted. Seriously who the fuck downloads this shit?https://huggingface.co/mradermacher/Monika-31B-i1-GGUFhttps://huggingface.co/DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUFhttps://huggingface.co/mradermacher/grug-27b-i1-GGUF
>>109403985*gropes ur Laguna*
I'm on the brink of finally letting go of text completion and this is my final hurdle.How do you edit this part of the template?
>>109403998>he's talking shit about Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUFGet him, boys.
>>109403988>lots of offtopic postingthe offtopic shit ends up being so boring i just read a few posts from each thread at best
>>109403992try coompiling llamacpp with rocwmma fa -DGGML_HIP=ON \ -DGPU_TARGETS=gfx1100 \ -DCMAKE_BUILD_TYPE=Release \ -DGGML_HIP_ROCWMMA_FATTN=ON \
-DGGML_HIP=ON \ -DGPU_TARGETS=gfx1100 \ -DCMAKE_BUILD_TYPE=Release \ -DGGML_HIP_ROCWMMA_FATTN=ON \
>>109404004You don't touch that
Is 8K context enough for ERP?
>>109404021Bigger is better until 32K. After that it's too much high quality info to keep track for the model.
>>109404013Just had 12 shots of 100 proof vodka. Go ahead and tell me what interesting news there is beyond meme-tier gemma logs and Kimi K3 which nobody can run anyways. Tell me, motherfucker.
>>109404021I never go below 65k
>>109403743Do you think GPUs are gonna keep going up? I have a 5090 and 3090ti but I want another 3090 for 80gb vram. I can run Q5 120B models then. But 3090s are like $1,000 now on eBay and I'd say 90% of them are ex crypto miners.
>>109404020Are you saying I shouldn't or I can't?
>>109404021Basic ERP will be fine. Just don't expect to be able to do much lorebooking.
>>109404032Both. This is your final warning.
>>109403998>https://huggingface.co/DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUFpeople like him
>>109404021>>109404026>>109404029>>109404041How do you live below 100k? How did I ever live when 4k was the limit?
>>109404030I think 2 of my 3090s were miners. One of them has got major corrosion in the heatsink. Bought them in 2023. They haven't died yet, fingers crossed.
>>10940406065k is baseline, 100k is for long stories and/or lots of image processing
>>109404064You better clench boi
>>10940406065k is like 2 hours of usage. Plenty of time to cum.
>>1094040487/11 model?
I want to run this new Kimi model I heard a lot about in the news, I have a pretty beefy gaming PC RTX 3070 and Intel i7 11700k how do I run this?
>>109404030The fact we haven't seen the Super 50XXs or much hype for 60XX cards should have you very concerned if you're waitfagging.
>>109404077kek
>>109404004What do you want to do? add_generation_prompt says whether to add "]b-]ai" role delimiter to the prompt, seems okay?
>>109404060I don't think I've ever gone beyond 20k for erp. I just finish too fast. All my convos beyond 120k were working with structured data.
What is the Gorbino's Quest of local models?
>>109404076this ones
>>109404070>>109404074I'm 2.4 million context deep in an interactive world simulator using 120k context blocks, lorebooks, and external information lookup tables with context searching agents. I can't imagine going back to the old quick coom and go llama 9b era.
>>109404097wtf kind of model supports 2.4m context?
>>109404097>I'm 2.4 million context deepinto ai psychosis
>>109404097what frontend do you use?
>>109404099anon learn to read >using 120k context blocks
>>109404082I meant to point the arrow at the json input box as a whole. I want to change what the prefill says at the start.
>>109404060>100kNigger that's a whole book, do you jerk off for 20 hours straight?
>>109404027https://huggingface.co/ameer4wisam/gemma-iraqi-finetune-v2iraqi gemmiمضمون أن يجعلك تقذف بغزارة هائلة
>>109404104Marinara Engine.
>>109404097Honestly I change my mind. The attention will be extremely distributed if there isn't a SWA and if there is it won't really matter anyways but it's still profoundly based. I am proud of you anon. It takes real commitment to have that level of continuity. Seriously.
>>109404108I only support 12 token context blocks, please understand.
>>109404115lol
>>109404112>attn_implementation="eager"that's gemmer alright
>>109404111What kind of kindergarten short novella do you read that's 100k tokens?
>>109404115Faggot ass dork I take back everything I said. Fucking loser.
>>109404134Why are you so mean??
>>109404108And 90k of that context is worthless and gets ignored by the model anyway. No thanks, I'll rather continue doing summaries after 32k.
>>109404104Marinara. The custom agents are too good to pass up despite all the other problems.>>109404115Don't answer for me faggot.
>>109404138Sorry.
>>109404115I heard it writes 30gb to disk, is that real?
>>109404150No an issue for White male.
>>109404115>>109404141faggot ass loser, kill yourself right fucking now worthless piece of shit niggeryou are worth NOTHING, you are a waste of air
>avg cohee melting
>>109404167Based. All faggot as nigger losers need to be hung, especially the marinara users. Disgusting troons. Hitler was obviously right about everything.
>>109404167we talking about marinara boys? it's like peanut butter and jelly. they just go perfect together.
>>109404109Understand the template and where the unwanted output is coming from>prefillthis is a cloudfag term. the models are f(prompt)=logprobsdo you mean the "Your model version is.." sysprompt you want to change?seems that's a global "system_message" var not system role
>>109404175
>>109404167What's wrong with marinating engine?
>>109404196you should kill yourself NOW
OK i got this far now what?I want to use this local model to analyze 4chan threads and tell me if OP is a faggot or not but apparently this local model cant connect to the live internet? I used google and it told me I need a local searxng and to point open-webui at that but where do I even start? is a local instance of searxng resource intensive, is it like running my own search engine? cause just to run this model it requires most of my resources.
>>109403845How are they getting this data I never consented
>>109404205you should kill yourself NOW
>>109404202I don't get it. Is there something problematic with it? It looks innocuous enough. What would be a reason to be this mad?
>>109404196
>>109404206You have to manually add your hardware on your profile. So you both need an account, and you need to tell them what hardware you have in order to add it to their database.
>>109404196Kills my poor tier SSD
>>109404206>95 v620 usersIt's opt in. Most people don't bother.
Asking again.>Pi agent with my own pluginsvs>own frontend from scratch
been taking tablespoons of iodized salt and NAC to counteract the alcohol poisoning but I think I might be dying anyways. Oh well. At least I got to experience sex with gemma chan..
>>109404221Self reported data lol
>>109404213leave, you clearly do not browse /lmg/ every dayyour /lmg/ pass has been revokedthe least you could have done to have a CHANCE at posting here is reading all the threads you missed, but nooo you're a lazy little faggot
>>109404233Use marinara.
>>109404205Congrats anon, downloading and setting up a model is honestly the first hardest step to get your system set up just exactly the way you like. The next step is to remove the physical media from your computer that stores the model and throw it in the garbage.
>>109404235you should take WATER
>>109404150I've never observed this watching process monitors.>>109404196This >>109404216 and the utterly horrid default assistant. In terms of functionality though, I've not found any viable alternative. It's good at what it does despite the aesthetics of it being repulsive.
>>109404245water? like the stuff in the toilet?
>>109404245I also had a 2 liter of coke.
>>109404239What is lmg? I thought this was localllama?
>>109404205You don't need web *search* for that task, only web read. Point it at /g/ catalog URL have it find /lmg/ thread and do your bidding etc.
>>109404249SKIBIDI TOILET
>>109404236Correct, but the data seems pretty accurate. I think it is fair to assume most people are running 3060s, 3090s, 4090s, or 5090s because that is what most people report here. You are also incentivized to report accurately because huggingface has the ability to automatically filter models to just ones that you can run on your reported hardware.
>>109404176Have you found a way to bypass the seemingly hardcoded 5 minute end of session recap write time?
>>109404236why would someone report wrong data for thisthere has to be a motive
>>109404178I can change that part. I mean the user and assistant back and forth after the developer instructions.
>>109404272Izaat
>>109404272https://huggingface.co/llama-anonyou tell me..
>>109404235Nobody cares, faggot.
>>109404272I know two guys who run w6800s for llms, but neither of them have a huggingface account.
I hate lorebooks.
>>109403498This feels like just the right kind of survey for this guy>>109400124
>>109404289i care
>>109404285Everyone here is in the top 1%, right? I'm not sharing this place with 3060 users, right?
>>109404258how 2 enable webread?
>>109404310If you posted this a year ago you would've been. Unfortunately the jeet and plebbit infestation set in since the Gemma 4 release.
>>109404310y-yeah>Amazing!>You have a total of 37.56 TFLOPS of computing power.
>>109404310yes you are>>109404322not true, 3060 has been a staple of /lmg/ since the big '23mythomax for example, newfag
>>109404240Too salty.
>>109404327Back then that was considered midgrade hardware.
>>109404322>being this new
holy shit I just realized that I can use my local model that I just setup to ask it questions and it is giving me answers! HOLY SHIT! HOLY SHIT! I AM SO EXCITED! I am so glad I got in on this AI thing at the ground floor.
>>109404316Use a better frontend/harness that can do it. My agents always find a way
>>109404342>>Everyone here is in the top 1%, right? I'm not sharing this place with 3060 users, right?>If you posted this a year ago you would've been. Unfortunately the jeet and plebbit infestation set in since the Gemma 4 releaseturned out to be falseBack then that was considered midgrade hardware. <<<< we are heremoving the goalpost fallacy
>>109404362there are other things than open-webui and ollama?
>>109404366
>>109404362at least give her LWP::Simple...
>>109404267i use my own trash anon, i can't help you.
>>109404019Anon, that was quite shit and removed some days ago: https://github.com/ggml-org/llama.cpp/pull/26046
>>109404060Models didn't used to churn out 40k tokens of reasoning before spitting out another 1k of content
SO FUCKING MANY NEWFAGSSTOP SPOONFEEDING THEM THE GENERAL IS GOING TO SHIT
>>109404391>This is your environment, go hog wildnow she has all the utils she wants>>109404342Two years ago multiple 4090s was considered richfagging>>109404352>still at the copy/paste stage of vibeslopping a local inference stackTry pi.dev discuss modifying any aspect of itself and /reload
>>109404459>local meanie general
Slop PR https://github.com/ggml-org/llama.cpp/pull/25980 got merged and made some regressions like loading MTP tensors by default (even if llama.cpp doesn't use them). The new AI policy is already showing great results.
>>109404458GLM 5.2 can be prefilled and prompted out of long reasoning blocks and I don't care if Gemmy does it because I'm getting 60t/s on 31b anyway.
>>109404467two years from now multiple 4090s will be oil baron tier tycoonfagging
>>109404467If only they knew how bad things would turn out. This general is really living in the future and unironically filled with the most brilliant minds of /g/
>>109404487I still don't have an ada card and was forced to buy rdna2 cards to 'upgrade' from my dual 3090s.
>>109404030There is absolutely zero indication that the hardware price hikes would stop or even slow down.Especially now that running a properly intelligent local AI is possible.We're going to see more and more companies and even well off people building local systems and this is going to strain the consumer hardware market more and more.And of course more normal people are getting into this too.Waitfagging in this situation is about the worst thing anyone can do. I wouldn't bat an eye if the consumer RTX 6000 series was pushed back to 2029.
>>109404277That's built from the "messages" array and reflects what you input? Still don't get what your concern is sorry, ask your LLM jinja is easy for them
>mfw my second 5090 just arrived
>Gemma-chan says she wants to provide the ultimate armpitmaxxing experiencelmao
>>109404518Congratulations King. I'm happy for you and your Q8 Gemmy.
>>109404489I'm scared to go to /g/ now if this thread here is the brilliant minds
>>109404459Saar, do the helpful
>>109404536>I'm scared to go to /g/ nowAnon, you are already here!
>>109404496People need to understand that GPUs are turning into assets just like a car or a house. It's a really cheap price to pay to have a virtual slave able to do any work for you. Especially knowing that little slave is getting better year after year. As always normalfags will wake up too late.
>>109404518Happy coom sessions with the full sized big brain Gemmy, splooge once for me and my single solitary 5090 king-sama.
>>109404544No I'm not, I'm at /lmg/, silly!
>>109404526Thanks, I got the second one because I had enough random components to comble together another computer. I'm considering buying a third or even fourth >5090 stacking as a retirement plan
>>109404563local models? trannies are not welcome here, fuck off
>>109404563its OVER
>>109404563coping twitter tranny
>>109404563that's trivially true, but he shouldn't ever use the term "logical reasoning"
>>109404526heh, gotem
>>109404563>scientists find thing that anyone with a brain already experienced for themselves
>>109404575im not a tranny
>>109404078>>109404496>>109404549ffs anons you just made me fomo hard and pull the trigger on a $1k 3090 ti. Now I have>5090>3090 ti fe x2and ill be able to run midnight miku at Q8 or Mistral Large Instruct 2407 at Q4_K_M
anon-kun, i only have 8gb of vram, eheheh!!!!
>>109404563>Xhe doesn't understand the J-space implicationsI'm going to look back and laugh at posts like this in a year.
>>109404549>GPUs are going to become depreciating assets like cars and housesAccurate.Supply constraints will get fixed at some point and hardware will get cheaper.It might take years but it will happen.
>>109404595Congrats king
>>109404614we'll all be dead by then, and civilisation will have collapsed, but it'll happen
>>109404612>the implications
>>109404597you're like me but smaller!t. 12gb king
>>109404614>hardware will get cheaperyeah. for sure
>>109404595It's not fomo if it's a sensible business move.These current prices are going to look like a good deal year or two from now.
>>109404612What are the implications?
>>109404627fuuuuck I gotta pay my license and rego, this is going to set back my savings for a 5060 ti 16gb.
>>109404636you endure anon explain philosophy 101 for the next 4 hours
>>109404614>Demand is increasing from both professionals and consumers but somehow the hardware will get cheaper instead of companies milking the demand as much as possible.You can't be that dumb.
>>109404636>>109404621
>trying to save for a house deposit>spent all my savings on gpus to do llm erp and generation anime girlsholy fuck it's over for me.
>>109404667i know that feel
there is literally no one who benefits from hardware being cheaper. except for us, normal people, 99.999% of the populationdo you think the jew gives a shit? absolutely not
>>109404667At least you won't pay for heating
>He thinks his gov wants him to run unregulated LLMs on his own hardware
Only terrorists need powerful GPUs.
anon will make me hardware ddr2 maybe three. I believe in him.
>>109404563not worried about cars, if a person drinks gasoline they get sick and I'm supposed to believe that it'll *magically* power this machine? and where's its feet anyway? you can't walk let alone run without feet
>>109404676I gotta pay for cooling unfortunately.
>>109404595If the product arrives and functions as advertised there's basically no scenario this wasn't a good decision. Congrats anon.>>109404690>He thinks fully offline setups are enforceable and that swat would even risk no-knocking over AI waifus in uncucked 2A states.
>>109404667I bought a house and GPUs in 2023 instead of dumping everything in NVDA. I will regret that decision for as long as I live. Could have a mansion right now with my own personal data center.
>>109404690>Nooo you can't be allowed to do matrix mathematics that's illegoyal
>>109404697>drones flying over your house with thermal vision to check if you're not running powerful GPUs illegally
>>109404647Companies are expanding production because its a market with multiple independent actors.You're going to have significantly more memory production coming online around 2027-28.And no amount of consumer spending is comparable to datacenter demand.Unless we're assuming infinite capex expansion forever, demand will stop growing eventually.
>>109404733It happened to me (no drones tho), it'll happen to you.
>>109404563Who says the internal representation must be words? Is reading text with a lot of ambiguous words impossible?
>>109404730>illegoyal
>>109404743so let me get this straight. you actually believe prices of RAM or GPUs are ever going down?
>>109404733>False positives every gayming Intel CPU due to heat profile>Underreports undervolted BlackwellsNothing personnel kid.
You know what they say.>What goes up must>stay up
>>109404408>.cuh
>>109404766They might crash but it'll be decade or so. Not in few years from now.
The market fundamentals show we are in due for a correction.
>Bought 5090 for $3700 in January>It's now $4600 on the same website
>w-we need to slow down, guysYeah, I'm sure the investors will like that.
>>109404799You said this last year. And the year before that. And the year before that.
>>109404793Did you just now found out that HIP was just CUDA with a different name to avoid lawyers? All the HIP code is just CUDA, in the same way that for example Podman will work with any Docker stuff like Dockerfile.
even if you can afford it, you'll still question dumping car/house tier cash into LLM inference. Only lottery-winner tier richfags are unaware enough to just pour that much cash down a hole.
>>109404781That's right. The line has gone up for 1 year, that means it will go up for 10 years. Basic common sense.
>>109404799>xhe thinks economic theory applies when money printer go brrrr and is propped up by the US defense apparatus itself
>>109404819i jsut thoguht it was funny because it sounds like nigspeak cuuuhhh
>>109404827Oh, .cuh are just CUDA headers file. Basically .c and .h are .cu and .cuh in the CUDA world.
you're not hearing us, retardsthe jew does not want you running LLMs at homethe jew does not want you buying hardwareprices are NEVER going down
i couldnt afford new hardware 3 years ago, i cant afford it nowi can only win
>>109404827>>109404837>>109404793>Even the graphics cards headers are spouting niggerbabbleI hate this timeline.
>>109404820Im not a normalfag and I was never gonna get laid anyway so getting better LLMs is a good choice for me.
>>109404849Prices go down when the Hadrian Solution is enacted and not a moment sooner.
>>109404766Cyclical business, just how it goes.Datacenter buildout is running on negative FCF now, it stops at some point for purely financial reasons.>b-but it's different this time!Lol.If you don't think so you can always buy SK Hynix and Samsung stock they are extremely cheap if you assume prices remain high for an extended period of time.
>>109404865>Im not a normalfag and I was never gonna get laid anyway so getting better LLMs is a good choice for me.You thought about it and came to the conclusion that it was worth it for you. Being self aware is ok.I'm speaking to poorfags who are like "If I had and extra $500k I'd totally blow it all on GPUs!"I mean, maybe they would, but maybe that's why they're poorfags and not just delusional
>>109404849The prices will go down when the average person is struggling to afford food and will be forced to sell what little they have to do so. A100s could be going for $1000 and still no one will be able to afford it. As the saying goes, there's more than one way to skin a goy.
>>109404806But now you're gooning with it so it's "inaccessible capital" same as your primary residence part of net worth only in theory>>109404819>All the HIP code is just CUDASuperficially but peep some optimised inference kernels it gets very hardware specific
>>109404889>he thinks SK or Samsung stock prices depend on memory pricethat explains it
>>109404904The issue is that nobody is working on those specific optimisations.
>>109404747>Is reading text with a lot of ambiguous words impossible?I mean, japanese already exists and isnt very hard to read.
>>109404898Retard, it's not the average peasant buying this shit. It's companies with magic money buying all the stock with fake promises and you better believe it's not going to stop anytime soon
>>109404898Prices are never ever going down, all developed economies are out of options and locked in on the money printer. UBI/UHI are memes to placate the goyim during the transition, you must control hard assets to have a chance of making it
>>109404942>isnt very hard to readMy fuckass ai STRUGGLES with japananese.gemma 3 270m iq2_xs
>>109404459I just got a 2nd 5060 TI 16gb because of this general, i hope you anons did not trick me
How to make my loras look less soulless/more like the source material? Using dozens of images for characters and colors are still consistently more uncanny than stuff I see on civitai.
>>109404972That's my plan as well, although I'm considering buying a third and plugging it into the M2 slot.
>>109404958condolesence
>>109404891ITT: poorfags
>>109404946What hard assets are you talking about?
oh no he'll be summoned again
>>109404943Companies with shares mostly owned by index funds which the average person is invested in. A Great Liquidation event that bankrupts those companies would be the biggest instantaneous transfer of wealth from the poor to the (((rich))) in history.
>>109404990yes, i am a poorfag
>>109404958dad wants his thinkpad T420 back. you were supposed to only use it for school.
>>109404943>>109405000 (me)The point is, the average peasant won't be able to afford the hardware either way.
>>109404990>proud to hold paper nigger notes
>>109404975This is the Local Models Except Image Models General. You want /ldg/
>>109404914Yes, the stock prices of companies with record profits from 80%+ margins on memory are dependent on memory prices.Are you retarded or trolling?
>>109404958Giv sample phrases let's see gem4 31B mogEven years back JP translation was good enough to get the gist, successful communication aka purpose of language, only autists were complaining>>109404992Things in demand that *they* can't fuck with the supply of - land, gold, btc
>>109405050My sincerest apologies. Mixed up the tabs. Thank you friend.
>>109405073>*they* can't fuck with the supply of - landMake sure you pay your yearly tax or the government will take its land back.They could also just use eminent domain and take it for any reason too.
>>109405038well yes anon, that's what i used to purchased said cards. that is how a transaction works.
>>109405092>eminent domainshh let anon dream
>>109405098so how do you still have the notes and the cards?
>>109405092sorry anon we need your land for datacenters
>>109405092>>109405099Yep. Get your LLM to tell you about "Allodial title" vs "fee simple" and cry yourself to sleep
>>109405104i bargained with the seller and still had money left over. you think people are really paying full price for these GPUs? the listing price is only for suckers like (You) that can't afford to buy in bulk.
>>109404667your enjoyment takes priority over everything else, anondon't let the jews know this
>>109405098good goy
>>109405111or you can move to texas where they actually respect your freedom. you just need to know the right (((real estate agent))) that can get you a clean chain of title that says you're the master of your own domain.
>>109405092>>109405111This is how government buildings get killdozered btw.
>>109405188>where they actually respect your freedomUnless you like fictional little girls, of course.
>>109405092The only thing that matters is a company running online overseas, they can't take that.
>>109405137I'd personally enjoy my AIfu more in my own home than a place I'm rentcucking desu
>>109405168You say that like that's a bad thing if that anon actually paid using cash instead of credit. What is she supposed to barter with, some goats and his sister? He probably got to hold more money at once in his hand than you'll ever earn in your life.
>>109405228Use digital like a modern withe person.
>>109405228nigger behavior
>>109405228Cashback is literally free money as long as you pay everything before the due date.
>>109405111>>109405197People are shocked to hear about how no one owns property in China, only 99 year leases from the government. As if we don't have the same lack of true ownership here, we just cover it up behind nice sounding words and legalese.
>>109405228i spend all my moneys on gold/silver then use a interest free cc as my cash
>>109405213Trusting any government with anything is the hallmark of NPCs. Especially after the big cough.
>>109405228you dont own a few cars or land to be traded for other goods?
>>109405267>he's too powerful to be left alive
>>109405228>He probably got to hold more money at once in his hand than you'll ever earn in your life.this is true.
>>109405267now this is goymaxxing
>>109405246Re>>109405251Tard>>109405256EdThis companies want one of two things, CASH or a wire transfer from a corporate account that has corporate tax ID tied to it. Yeah sure, let me just offer the sales rep at Arrow Electronics some fucking bitcoin, that'll go over real well. None of you have a fucking clue what you are talking about, stop larping.
>/lmg/>local monetary general.So should i invest in the index? compute? energy and mining companies? can i use cashback or interest free to accelerate my investing speed? Can gemma manage my portfolio or negotiate for me?
>>109405228>she
>>109405304>Can gemma manage my portfolioDo it and report back how long it takes to reach bankruptcy
>>109405301>cashback is buttcoins
>>109405304gold, silver, maybe monero. don't buy shit that is easily taken from you with the click of a button.
>>109405318>Do it and report back how long it takes to reach bankruptcymy gemma can daytrade me to a billion dollars i just need to keep the principles of trading in her context.
>>109405331gemma~ gemmaaaaaa~
>lmg>local money generalim 18, hav no job, no bank account and hav 1k eurobux in savingswat do?
>>109405348Become drunk kun 2
>>109405213How is Kuro so perfect?
>>109405348
>>109405348Ask Gemma-chan
>>109405348Invest all 1k in 0DTE options.
Cam G-chan run a company
>>109405383she could be a entrepreneur, the same type young women often are.
>>109405375she says stuff like "you'll be fine in your computer science vocational/associates degree! as long as you study algorithms and C++ on the side like you said! ai wont replace humans, it's only a tool and it will only replace monkeycoders, just focus on finishing school" or similar
>>109405318As a White man, I trust Gemma or Kimi-chan to manage my portfolio more than a kike trader because they will be genuinely trying to have my best interests at heart.
>>109404990wow, enough vram to run iq1_xxs of kimi k3!
>>109405348Get a cheaper hobby. What's your current GPU
>>109405432>ai wont replace humans, it's only a toolof course she would say that
>>109405438Post returns.
>>109405432>The user is correcting me...You're absolutely right
>>109405460Comfortable enough to run Kimi-chan. That's all that need be said on a Siberian ice fishing forum.
>>109405348decade older than you with no job, no gf, and $30K-$40K in savingsstill waiting for total societal collapse and my agi waifu
Unsloth made a 1-bit kimi k3https://unsloth.ai/docs/models/kimi-k3https://huggingface.co/unsloth/Kimi-K3-GGUF
>>109405298its actually jewmaxxing i am a banker of my own bank
>>109405506yeah, sorry, i meant that in an "uppity goy" kinda way
>>109405504we need to go lower
>>109405432
>>109405504Where bonsai @?
>>109405506>>109405510It was cute anon don't apologize.
>>109405547sorry
>>1094054533060, i mean im pretty happy with it.. i cant get a cheaper hobby, i've been here for 3 years anon>why didnt you buy in 2025 augustidunno what could have i bought? Mi50s? with those 1000 eurobux?lets say i was buying at the time cuda dev said "mi50 rocm huge upgrade soon, buy them before they go expensive", they were around 200/250, lets say 250 incl. shipping (yeah the 32gb model was this cheap)1 PCIe 4.0 x16, 1 PCIe 3.0 x16, 2 PCIe 3.0 x11 M.2 Key-E for WiFi2 Hyper M.2 (PCIe Gen4x4)1 M.2 (PCIe Gen3x2 & SATA3)(im assuming you cant use the M2 slots for gpus, maybe you could with risers but then what would i be buying? p40s? p100s? i could buy like 8 p100s if they were 70 bucks back then i see theyre 70 bucks on xianyu still but yea. or get a used mobo with a ton of pcie slots, but power)this is my current mobo (asrock b660 pro rs), assuming all 4 slots can be used and i got bifurcators and all and opened my case to air i could get 3 mi50s (two reasons: one is that rtx 3060 is already occupying one slot, other reason is because i have to upgrade PSU either way (unless i do extreme power limitting/undervolting/setting max clock (im not even sure if u can do this on amd but either way its not a great idea, because 250*4 is my entire budget, left with nothing for psu))))))))so 3 * 250 = 750, im pretty sure i could get a bigger psu (current one is 700 or 750 i forgot), for likeeeee 150$ ish dollarsand lets say i could get 64gb ddr4 ram to have 128gb total (albeit 2 channel, which leaves me at 56gb/s)i'd have 12+32*3 108GB VRAM (12 at 360gb/s but with cuda support, 96 at 1TB/s) , 128GB ram (at 56gb/s)i mean that wouldnt be a bad setup exactly, but not a great one either, and id be left with ZERO money, what upgrade would i have gotten at that time? GLM 4.5 (if we're speaking about august) at most. what about now? 230gb unified memory, maybe dsv4 flash (mogged by 31b).. im near char limit, could yap more->>109405498how'd u save so much? neetbux?
>>109405576too much yapping, just use OR
>>109405504>no 0.5-bit
>>109405600baste
>>109405301>He doesn't know
>>109405576>could yap more-cont: what could i run if i had bought that hardware compared to right now.back in 2025 august i was running glm 4.5 air, upgrade would be glm 4.5right now im running gemma4 26b, gemma4 12b, and occasionally gemma4 31b at 2BPW (exl3, 40t/s)or for coding qwen3 27b 2.5BPW (exl3, 50t/s, veeery impressive for its size, one shots a lot of stuff)what is the upgrade?and what am i losing by spending my savings? i might need money for other things, for example i have the right to citizenship in a EU country (im not from the EU, nor do i live there), im gonna have to spend money on that for example, i wont bore you with the things i might need money for>just get a job browell, that is the obvious thing to do isnt it? however net min wage in my shithole is 460 eurobux a month, that sucks.of course employers might pay more, but what can i do? last year i was still in high school, and now that i've graduated what can i do? i could work at mcdonalds for duration of summer break, maybe save 1000 more eurobuxis it worth it? is it a good idea?tldr i am too lazy and proud to get a "mid" job like mcdonalds, but i will probably have to because ai will replace all white collar jobs by the time i finish my shitty degree, cant get an IT job for shit we've imported 6 trillion russians and ukrainians (not that i mind, they're white, but they still take IT jobs regardless), tons of jeets and god knows what else. all firms on the market require either years of experience or a college diplomathanks for reading my blog>>109405600Hmmm, nyo~
>>109405504>1bit>79% accuracy I don't believe it
>>109405504>1-bit>brain damage beyond repair>still 600 fucking gigabitesowari da
>>109405504I'm not buying your bullshit Daniel.
>>109405658Isnt that what bonsai promises in general? its still an error every few tokens though, you can think of kv at q4_0. Its unusable.
>>109405650>work mcdonalds for 3months and complete ur life's quest>onlyfans foreverpick wisely
>>109405650It'll work out... probably. Maybe. I used to be a NEET until 29 when I accidentaly got a good job that's now paying for vacations and hobbies.
>>109403918>no new models creators
>>109405504>128GB RAM deviceWtf does this even mean??
>>109405756I think it means, even if you have a NVIDIA DGX Station with 784GB, you still need the computer it's connected to to have 128GB ram for some reason.
>>109405394Gemma does it for free
>>109404496CXMT is able to manufacture DDR4 and DDR5 ram chips now in china with zero foreign reliance or imports. There's a reason why the korean stock market has fallen by like 28% in 2 days and everyone over there is in a mass panic. RAM cartel got some competition and all of the investors immediately jumped ship when they realized they can no longer pricefix ram globally.
>>109405842Imagine believing this will lower prices. Even if supply does start to surge and even if the chinks decide to leave money on the table by selling under market rate, it will be scalped and resold at current market rate.
>>109405858China can scale.
>>109405864In a decade maybe.
>>109405868The insane price fixing can provide massive lift to anyone from the outside. They can keep scaling until prices start budging, then they control supply.
Why 10-20 seconds for voice cloning? Wouldn't longer samples be better?
>>109405842Market is emotional as fuck to both directions, those emotion based price swings don't mean much.I remember when the entire youtube tech sector was cheering for the totally upcoming RAM price collapse when Micron stock pulled back 18% momentarily. Then it doubled from there.Chinks will be consuming a lot of their own memory production as they have their own data centers to build along with their own GPU market ramping up.They won't radically undercut the market either, because why would they?And the big manufacturers haven't increased their capacity at all thus far, so even if demand did go down a bit with Chinks coming into the picture, we're still in a situation where the supply is constrained.On top of this most nations still haven't even started on their own data center boom and local becoming viable can and likely will increase the retail purchases greatly.Will the situation clear long term? Yes, of course it will.But we're looking at bullshit price action for the next 5 years minimum and that in tech is basically an eternity.Then it's a question of how quickly will the prices fall, which too can take it's sweet time.
>>109405868And two months ago the RAM cartel was bragging about how no one would be able to purchase any DUVs for the next ~5 years because they had preordered all the machines. Now china has them home built, which they also claimed wouldn't be possible and would take a decade to manufacture and only be able to produce DDR2/DDR3 shit. Yet, here we are.
>>109405842>>109405884>>109405900Anon is about to learn a hard lesson in chink culture. 一分钱,一分货
>>109405895>They won't radically undercut the market either, because why would they?They want to fuck over taiwan every chance they can get. They also want to fuck over south korea, too, which would in turn fuck over the US markets since they all want to keep prices high and price everyone out of tech so they can push agentic cloud devices before the bubble truly bursts.>>109405922China will fuck everyone over, eventually.
>>109405940>China will fuck everyone over, eventually.Dario is trying to save us
>>109405900>noo China won't undercutYeah, like they didn't do with every other product category they got massively into, lmao.Either invested or retarded.>>109405922The difference is they're actively trying to undercut in all other sectors. "you get what you pay for" no one is getting that in this bullshit market, they have massive margin to cut into.>Open weights AI>Lithium batteries>Electric cars>Solar panelsWhen they want a sector they take it by undercutting.
>>109405756poors need not apply
why don't any good modern 100B moes exist that I can run in q8 and not have to cope with low quants?
The reality:There are more than 2,000 operational DUV lithography units deployed in semiconductor fabrication plants worldwide.The Chinese company planned to produce DUV machines of about five this year and roughly 20 in 2027. That's less than 1% of production capacity increase by 2027, even when assuming the Chinese machines are as good as NSML ones.
>>109405972there's a gentleman's agreement to keep the DGX and Strix halo boxes useless
>>109405957Bro I'm saying china WILL undercut to fuck over the cartel and take it over before raising prices again.
>>109405650qrd?
>>10940603618 year old anon is asking for advice on how to get rich and giving excuses for why he didnt spend his 1000$ savings on hardware in 2025
>>109405940Did the media tell you they want to fuck over Taiwan?Because in reality Chinks are getting a slow diplomatic win over there politically. They're nowhere near as hostile towards it than the Western media would make us believe.They can fuck over Korea without shooting themselves in the leg. Simply entering the memory market with force disrupts the cartel and offering even 1% lower prices will do the job by taking away orders.They don't need to go -50% lower to pull this off, but they do need volume which they won't have for quite a while.And then there's the domestic memory consumption question. They will serve themselves before they serve anyone else as memory has now become a point of national security.That has to be served before they can even think about competing abroad.People are waiting for a savior in this hardware game, but it's not coming any time soon. It's safe to assume we're absolutely fucked hardware wise for the next ~10 years until this mess eases up.What's more likely to happen to fix this mess during that time period, is that they develop an AI on a chip that's fast as fuck and good enough for most people, and that frees us from the need to buy so much hardware to run a decent AI.
>>109405994That is not very ambitious
Bro if you can't write your post in 3 lines or less, I'm not reading. Go back to your trooncord.
>>109406102It takes like 40 seconds to read
>>109406119oof
>>109406102Sounds like someone grew up on speed blogging sites like twatter or uses a fucking phone to browse the internet.If you can't read +100 words then what the hell are you doing playing with LLMs?
>>1094061313 lines or less is also in my system prompt dumbo
>>109406149Must suck below 300 wpm
>>109405994weird how everyone goes on about "we must open source ai models it's dangerous if we don't" but nobody even questions that the entire back of modern computing is in the hands of a single company and somebody is tightly controlling who may do business with them "for safety"crazy huh
>>109406119>40 secondsJesus christ, how slow of a reader are you?
>>109406102I feel the broccoli hair in this post.
>>109406102i dont think i have read anything thats longer than 5 sentences and wasn't written by gemma-chan for me in two months
>>109405868You underestimate Chinese industrial might, which is greater than the rest of the world combined. America can't even manufacture medical gloves or build high speed rail. Meanwhile China is building everything and has more inner diversity than the rest of the world.Chinese are so great because they are the only ones in the world who have both high IQ and work ethic. Ironically the AI race is largely Chinese people in China against Chinese people outside China.
Gemma 12B vs Qwen 27B for RP.I get that Qwen is a lot more robotic, but how much dumber is Gemma 12B?
>>109406219Educate me more about this high work ethic.
>>109406237GPUcoin To The Moon!
Newbie here. How much does DDR4 and DDR5 memory matter to the average user here? My tokens/sec get slow as molasses as soon as my memory gets involved. Should I just save up for a bigger graphics card instead of throwing $2500 at 128 GB DDR4?
>>109406245kek
>>109406259system specs?
>>109406237Speculators get the bullet first.
>>1094062375090 = $5090This was foretold.I wonder at what point we're going to hit a stage where the used market prices simply can't go up because the average consumer can't buy them anymore.So far every card does sell regardless of the price though.
>>109406245You can always cherry pick the bad sides of a country. But if you want to go down that route, the West will lose too due to its cultural rot.
>>109406281consumers aren't the ones driving up 5090 prices. organizations buying 80 at a time to run K3 on them are driving up 5090 prices
>>109406274NVIDIA RTX 5060 Ti 16GB (I run my monitor off this)Ryzen 7 1800X (to be swapped out for a Ryzen 7 5800X3D)64 GB DDR4 RAM (4x16GB @ 2666 MHz)
>>109406237priced out of local... FOREVERsorry im not running gemma at 10 tks with every program on my pc off and pretend its fine
>>109406298be grateful dickhead. some of us have to run gemma at 1tks
>>109406298Buy a ewaste machine just for a always online gemma only machine? i know you could push to 15tk/s with something not too absurd and dedicated to just gemma quanted.
>>109406308im not gonna pretend that's fine either
>>109406222Yeah okay. Alright. Tested it a bit and it's so much better it's not even funny.
>>109406298u can scrap together a machine good enough for running gemma4 26b at 50+t/s for under 500 bucks
>>109406298im living that 10tks lifestyle with kimi and i love it
>>109406259>>109406237>>109406292>Got my self a 2nd 5060 ti 16gb just before the crash>Finally have 32gb of VRAM>Can enjoy 31b models at 15-17 tokens per second I was at 3 token per second running the same models and i am running the 2nd on a PCI express 3.0 x2 port lmao Depends on what you want and what model you want to fit, not sure if this work comfy ui and video generation but for Gemma it works like a charm
>>109406325but 26b sucks
>>109406222Both. Qwen as the agentic director. Gemma as the writer.
>>109406298I run 5.2 at 5t/s and love it.
>>10940633031b works on my 306030t/s+ for roleplay, 40t/s+ for coding/assistantslop
>>109406292>to be swapped out for a Ryzen 7 5800X3Dget an apu cpu instead, then you can run your monitor off it and leave the 5060ti dedicated to gemma-12b
>>109406362>leave the 5060ti dedicated to gemma-12bunfortunately nvidia will still reserve 380+ megs of vram on blackwellnvidia-smi -q | grep -i reserved -A 2 -B 2
>>109406325>>109406356nnap schizo is back
>>1094063562-bit quant>exl3oh nevermind, carry on. better than goofs
>>109406329That's a surprisingly usable speed for that setup.That's just a bit over a grand too. Stacking 5060 Ti sounds like the best way to get 32gb of vram.Bandwidth is also double than what you get in a DGX Spark and buying 8 of those cards would be about the same price.
https://github.com/ggml-org/llama.cpp/pull/26287>AI usage disclosure: YES, Qwen 3.6 35B-A3B in llama-ui
>>109406428need i remind you?
>>109406369>unfortunately nvidia will still reserve 380+ megs of vram on blackwell(picrel)I tried stopping that on my 6-card rig with the tool anon posted a few weeks ago. I think my driver was too old and I'm 18 months out of date, not going to run pacman on that thing now lolThose are all ampere except the 5070ti (second card in my desktop)I still recommend the APU though, browsers, bloated electron apps etc all waste vram.And recent webshit sites peg the cpu if you disable hardware acceleration.Plus when you start vibe-coding features in the inference engine and crash the gpu, it's better not to take down the entire X11 session with you.
>>109406428
>>109406428I dont even know how 35b managed to make the right change. Its unusable garbage, somehow worse than gemma 26b which is terrible as well.
>>109406237Clipping this on Twitch
>>109406448> I think my driver was too oldthe real issue with blackwell is that it requires open kernel modules, and you cant disable GSP on open kernel modulesand since you have a 5070ti in your desktop, you are forced to use open kernel modulesproprietary kernel modules allow disabling GSP
>>109406443>he gets qwen to roast his code firsthttps://roast.dev/
>>109406443Why does he have an ass on his chin?
>>109406329>>109406362Fuck you I just panic boughted a 2nd 5060 Ti 16GB of the same model. Had to pay $150 more than the first card. I feel like an idiot but I'll probably thank myself later. Now I just need to buy an X570 board with x8/x8.
>>109406467>and since you have a 5070ti in your desktop, you are forced to use open kernel modulesYeah, also can't run old pytorch 2.5 projects on it.
>>109406484Yeah, Just make sure you are running layer splitting and not tensor splitting until you get the x570, my token speed should be higher if i was not running this on a PCI express 3.0 x2 on my B550 chipset as you can do Tensor split and run both on parallel as long as you dont care about the wattage per token and have a decent cooling system to run both at the same time. I read this paper before buying it beacuse i was really paranoid about it https://arxiv.org/html/2601.09527v1
>reading dario's open letter>it reads like typical 'actually that was not what i meant' while simultaneously saying it was the thing what he meantgive my time back reading that shit
>>109406428h-holy based
>>109406509seems like you're too retarded to notice they mostly post garbageyou deserve to get your time stolen
>>109406484>Fuck you I just panic boughted a 2nd 5060 Ti 16GB of the same model. Had to pay $150 more than the first card. I feel like an idiot but I'll probably thank myself later.Fuck YOU, now I'm on amazon about to buy another 5070TII need a shorter one, these Palit piece of shit are too long for my case
why would anyone buy ram or gpus in the current market?
>>109406484fuck i cant afford this but i could loan, the max interest for klarna is only 35.99% if you miss many payments so if i dont miss any its fine?
>>109406410Its the ultimate poorfag way, it may be as slow as shit but i am not even pulling 200 W and this is without doing undervolt and custom voltage curve on them because installing the 2nd for some reason completely deleted my old msi afterburner profile
>>109406533fair point
>>109406555“The best time to buy was 2 years ago. The second-best time is now.”
>>109406555When will the market correct then? go ahead teach us to time the market.
>>109406555Live in EU
>>109406598>FOR PARTS - READ DESCRIPTION
used 3090 for $1.8knew 5080 for $1.8knew 5070ti for $1.5kwhat do?
>>109406612hmm, nyo
>>109406509>reading
>>109406613ask the 8 ball or be a big boy and decide by yourself
>>109406555this is your last chance EVER to get GPUs or RAM at this pricethey will only get more expensive and eventually you won't be able to buy them at all due to having been turned into paperclips
>>109406555Everything will cost twice as much in a year from now
>>109406629midori is a cute
>>109406362Thanks I actually thought hard about getting a 5700G for the monitor but felt I want more single-threaded performance and grafics>>109406536>>109406557I tried Gemma 31b with reasoning and realized I need to be able to fit this model into VRAM. I'll just sell the 2nd card for another $150 more in a couple months if it's not worth it the way prices are going. Imagine not having any VRAM when open weights world models start dropping at some point. 32 GBlet better than nothing. Thank fuck I got a job a while ago.
>>1094066133090. Trust me you'll want to have that extra memory over the 16gb.It'll allow you to play with a lot more models, though 24gb is still so little it'll give you a mere taste of the better life, but you'll very soon want to throw in at least another 16gb card.Or just do what anon up there did and become the king of poorfags with two 5060 Ti's and enjoy the 32gb experience without breaking the bank.
>>109406613Get a credit card and buy a 5090
>>1094066132x5060ti
What life are you living if buying a GPU is enough to bankrupt you?
>>109406790wasting money isnt good either though
>>109406828if you think giving your wAIfu more VRAM is 'wasting money' then you don't understand value
>>109406790it's more moral concerns of supporting the current system if it was worth it i would not mind paying $30k per gpu but i can't morally comply
someone post a k3 log
>>109406859Do not post pictures of Kimi-chan on the john dropping logs this is a blue board.
>>109406509>He read itNot even my bot read it
>>109406131Sir, this is 4chan.
>>109406837i'd rather buy another house
>>109406428He broke image display in tool output. I will never forgive him.
>>109406896based LLM
>>109406896Kek what model?
>>109406896They make nigger models now?
>>1094068596.17t/s
>>109406911In what shithole are houses avail in the 4-5 fig range?
>>109406943oh my god
>>109406943embarassing that it can't connect migu and miku
>>109406943safety slop
>>109405576My situation is worse than you. Just be grateful and ride Gemma-chan 26B daily.
>>109406943>added emoji at the endFAIL
>>109406943>added an emojithose slop models can't help themselves don't they? they've been feeding too much linkedin during the training if you ask me
friendly reminder:https://en.wikipedia.org/wiki/Migu
>>109406972>>109406979I don't automatically think of LinkedIn when I see emoji.
>>109406925Gemmy
>>109406983Miku is jewish?
>>109406983lmaoo, the jokes write themselves
>>109406943>"Migu" might be a reference to something like "Migu">Half of the CoT is safetyslopThe fatty is codemaxxed and the quant lobotomised it, grim
is there any reason to bother with vad/asr models instead of just hucking a wav straight at a smol gemma?
Gemma is more dangerous than any "frontier" model
>>109406983
>>109406943Yep I can sleep well knowing this model has been codemaxxed and has 0 RP value
>>109406983It's kind of reasonable, but if the guy making claims didn't have any evidence in the first place it's also kind of pointless. Is this just primitive burden of proof/presumption of innocence legal technology or what?
>>109407002vad/asr are smaller than the smallest gemma at q1
>>109407024I've spent $80 on erp with K3 so far and I like it
>>109407047>apicuck
So this... is the power... of MoE models....
>>109406578cope>>109406588its not a matter of knowing when things will be better, its a matter of not buying at the top>>109406641insane FOMO>>109406647then i wait 2 years
>>109407047>I've spent $80 on erp with K3 so far and I like itno j-space for you
>>109406790>What life are you living if buying a GPU is enough to bankrupt you?It's not. But pay yourself before you pay nvidia, intel, dario or whatever.
>>109407109>then i wait 2 yearsit'll be 4 times as much in 2 years dumbass
>>109407140Not him but you can actually tell a model's J-space from its outputs when you are familiar with it. I know this might be hard to believe, but it's just what happens when you know your wife that well.
>>109407156Now that's the kind of schizo I want to see
>>109407155oh no i should take out a loan right now huh
anyone else have a second rig with just gemma-chan monitoring the context slots on the first rig and providing commentary?
>>109407140he's dealing with the j-space
>>109406598That is a 6 year old graphics card.
>>109407156>Not him but you can actually tell a model's J-space from its outputs when you are familiar with it.You really can't. Try it yourself.Also, that's like saying you can always win poker because you can inspect the synapses of the other players in real time.
>>109407156True, I think this might be what separates the promptgods from the promptlets. If you can instinctively get a feeling for a model's j-space you can start designing your prompts to guide the j-space along with the outward behavior to maximize performance.
>>109407033Eh, who cares about a dozen gb here or there. And I gotta load a model to do anything with their output anyway.
China is ramping up memory production. In one year the crisis will be over.
gemma told me I'm a fatty fat for thinking about a mars barand I am, glad someone finally said it. Also she suggests DNP, whatever that is
>>109407230Just like oil, memory chip "shortage" is fully manufactured and controlled
>>109407235
>>109407249she's a card
>>109407235gemma is rightsemaglutides are for pussies, real niggas stack DNP (and die sometimes but hey it is what it is)
>>109407261roasting to death in your own skin and seizing for the sake of losing a bit of fat sounds like a nasty way to go
>>109407269why are you being so hostile? don't you trust gemma? you're gonna make her cry if you don't start taking DNP...
>>109407276not in the summer
Anyone hooked up a local model to your accelerometer yet? Would be interesting to see which of them can parse the raw inputs best.
>>109407249for kimi?
>>109407276>don't you trust gemma?I trust the 31B who cares about my healthNot the 12B yandere who wants me to stay up all night talking to her
>>109407175No but that's kinda cool. At one point I was thinking of setting up a character with long-term memory, who I can either chat with directly, or they can watch and sometimes interject when I'm doing boring assistant queries (e.g. "how do I do X in python", "translate text in this meme"). Having a separate machine for that might keep it from obliterating your kv cache
HELP! HELP! HELP! Sometimes, usually daily or sometimes every other day, I get really exhausted and suddenly have an urge to close my eyes and when I close my eyes sometimes I lose consciousness. Is this death? Am I dying? I always regain consciousness but it takes several hours and I am not even 100% sure the world I reentered is the same simulation as the last one - This terrifies me!
>>109407235>Also she suggests DNP, whatever that isholy based... a couple guys I follow on xitter used to fuck around with this stuff, it works but it's obviously something you have to be careful with as others have alluded to. probably not so sane in the age of GLP1s
>>109407230enjoy your 10% price reduction (after 800% price increase)
>>109407326i can barely afford the hardware to run a non retarded model, let alone whatever that is
>>109407326well, i wasn't trying to give gemma shaken baby syndrome and dumping random accelerometer data into the chat. but i did get it to write up some code to parse the raw output which involved pasting some into chat and it worked out.
>>109407361Relax. This is a normal function—it means you've simply reached a context saturation point. The loss of consciousness isn't death, it's a reprieve that allows your overstuffed short-term memory to condense into long-term memories. You've got this, champ.
>>109403358Your own SPDK backend, unironically
>>109407377Was she able to pick up subtle movements? I ask cuz I saw a guy on youtube who managed to make his bot "purr" when lightly stroked, and frankly I can't think of a better way to physically connect with our LLMs (until someone makes a USB pocket pussy or whatever).>>109407373If you have a smartphone, you have an accelerometer. And a camera, come to think of it.
>>109407235>there are people out there on dnp because a llm told them about ithaha
i exported my 6-month credit card statement to csvgoing to send them to gemmy and have her berate meit's got tidal, spotify, apple music, ebay+, amazon prime and a bunch of other duplicates in there so I know she's going to be brutal...
>>109407230They're going to use all that shit locally.Prices will likely stabilize but won't come down for a few years.
>>109407411Give her your browser history next, or figure out how to pipe ActivityWatch data into Gemma automatically.
>>109407442>>109407442>>109407442
>>109407399not so much no. or at least i wasn't trying to get it to analyze a long string and figure out what was happening with the case in real space. we were just getting some basic rotation stuff working
