/lmg/ - a general dedicated to the discussion and development of local language models.Previous threads: >>101560013 & >>101556980►News>(07/24) Mistral Large 2 123B released: https://hf.co/mistralai/Mistral-Large-Instruct-2407>(07/23) Llama 3.1 officially released: https://ai.meta.com/blog/meta-llama-3-1/>(07/22) llamanon leaks 405B base model: https://files.catbox.moe/d88djr.torrent >>101516633>(07/18) Improved DeepSeek-V2-Chat 236B: https://hf.co/deepseek-ai/DeepSeek-V2-Chat-0628>(07/18) Mistral NeMo 12B base & instruct with 128k context: https://mistral.ai/news/mistral-nemo/►News Archive: https://rentry.org/lmg-news-archive►FAQ: https://wikia.schneedc.com►Glossary: https://rentry.org/lmg-glossary►Links: https://rentry.org/LocalModelsLinks►Official /lmg/ card: https://files.catbox.moe/cbclyf.png►Getting Startedhttps://rentry.org/llama-mini-guidehttps://rentry.org/8-step-llm-guidehttps://rentry.org/llama_v2_sillytavernhttps://rentry.org/lmg-spoonfeed-guidehttps://rentry.org/rocm-llamacpphttps://rentry.org/lmg-build-guides►Further Learninghttps://rentry.org/machine-learning-roadmaphttps://rentry.org/llm-traininghttps://rentry.org/LocalModelsPapers►BenchmarksChatbot Arena: https://chat.lmsys.org/?leaderboardProgramming: https://hf.co/spaces/bigcode/bigcode-models-leaderboardCensorship: https://hf.co/spaces/DontPlanToEnd/UGI-LeaderboardCensorbench: https://codeberg.org/jts2323/censorbench►ToolsAlpha Calculator: https://desmos.com/calculator/ffngla98ycGGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-CalculatorSampler visualizer: https://artefact2.github.io/llm-sampling►Text Gen. UI, Inference Engineshttps://github.com/oobabooga/text-generation-webuihttps://github.com/LostRuins/koboldcpphttps://github.com/lmg-anon/mikupadhttps://github.com/turboderp/exuihttps://github.com/ggerganov/llama.cpp
►Recent Highlights from the Previous Thread: >>101560013--Paper: SPLAT: A framework for optimised GPU code-generation for SParse reguLar ATtention: >>101560957 >>101563188--VRAM requirements for 128k context with nemo: >>101562691 >>101562847 >>101562853 >>101562937--TTS fun with LLMs and shell scripting: >>101561911 >>101562564 >>101562604 >>101562643 >>101562674--Quantization levels in Mistral-Large-Instruct-2407-GGUF and llama.cpp: >>101564120 >>101564169 >>101564754--Performance comparison of AI models, including mistral, llama, and instruct-turbo: >>101561066 >>101561117 >>101561123 >>101561142 >>101561159 >>101561222--Llama model comparison using various metrics: >>101560537 >>101560558 >>101560571 >>101560585 >>101560604 >>101560608--Large model comparisons and their pros and cons: >>101560219 >>101560232 >>101560331 >>101563296--LLaMA 3.0 vs 3.1 quanting comparison: >>101560813 >>101560901 >>101563166--mpt-30b-chat model discussion and comparison to other LLM models: >>101560900 >>101560927 >>101561638 >>101561754--VNTL leaderboard updates and the limitations of current scoring methods: >>101561663 >>101561724 >>101561730 >>101561741 >>101561918 >>101561690--Text-to-image models, hardware accessibility, copyright, and the future of AI: >>101562028 >>101562436 >>101562509 >>101562533 >>101564560 >>101562512 >>101562518 >>101562545--Nala 405B test and comparison to Mistral Large: >>101561168 >>101561232 >>101562628--Prompt processing in large-context models is slow and painful: >>101563274 >>101563289 >>101563372--Rig requirements for running a 405B model and commercial feasibility concerns: >>101565887 >>101566053 >>101566083 >>101566100--Tele-FLM-1T: 1T Parameter Model Release: >>101566799--Llama 3.1 rope scaling factors pull request: fix for broken garbage output: >>101563218--Kobolddpp with nemo support released: >>101562793--Miku (free space): >>101561677 >>101565893►Recent Highlight Posts from the Previous Thread: >>101560019
I want to perform sentiment analysis. Where do I begin looking for models and tools that can do this?
>>101567235Why is she crying?
>>101567256the bullying from the last threads got to herbit sad innit
>grok 3 to be trained on 100k H100s>would need 155 megawatts of powerjeez wish i had so many gpus
>>101567235Thank you Recap Miku
>>101567278>all that wasted resource to shit out a turd worse than the 8bs we're using for ERPabsolute state, faggot should focus more of his efforts on getting people to mars instead. At least that has some silver lining.
I could do a lot with a single H100...
>>101567240sentiment: faggot (confidence 100%)
mpt-30b-chat q8 GGUF quant:https://huggingface.co/quarterturn/mpt-30b-chat-q8-gguf/tree/mainYes there are existing GGUFs out there, but they're old, and for some reason they're really slow. I can get 12 t/s at q8 with mine vs only 2-3 t/s with the older ones.This was a smart model for 2023, and it has 8K context. It's a chat model, and it seems to work fine with alpaca-chat in ST.Why would you bother with this? It's pre-safety, pre-alignment. Also, it's a chat model, so I think it gives it an old c.ai feel. Have fun. I'll make another quant which fits in 24GB VRAM once I figure out the deal with loading to huggingface via git push without errors.
So now that the good models are completely unapproachable from a vram perspective, is cpumaxxing the way? And if that's the case, would it be better to get one of those old servers with 2400 ghz DDR4 that have a lot more contact/bandwidth with the CPU, or a 4 slot board with a normal CPU and 4 sticks of 3200 DDR4?
>>101567424>8k context>2023>30bwhy?
>>101567375I can only think of a single thing: sell it and use the money to buy ~20 3090>>101567424It isn't just pre-alignment, it's pre-intelligence.
>>101567470It's not built on a curated and aligned dataset, and it wasn't trained to say "as an AI language model...". It's a journey back into what once was, that's all.
>>101567477yay! kids are safe now.
>>101567477I like the shapeshifter hand action.
>>101567470>>101567484>Coping with woke modern models and ignoring gems like this and dolphin-2.5-mixtral
>>101567503>>101567542funny how this happens every single time major breakthroughs occur and everyone starts finding models they actually like that are newhappened with every mistral wave i can remember.no, you're not going to bait anyone into "going back to tradition with mythomax". fuck off back to whatever hole you were paid to crawl out of.
>>101567424>It's pre-safety, pre-alignment.>It's not built on a curated and aligned dataset,>MPT-30B-Chat is a chatbot-like model for dialogue generation. It was built by finetuning MPT-30B on the ShareGPT-Vicuna, Camel-AI, GPTeacher, Guanaco, Baize and some generated datasets.
>>101567484>I can only think of a single thing: sell it and use the money to buy ~20 3090basado
>>101567560the fuckno new model release today?its so over for local models
>>101567557You aren't going to miss the Goliath gem, will you, anon?
>>101567570by my calculations cohere is due aaaaany minute now
>>101567574*prepares one million watermelons and hands you 50 thousand*
>>101567560Right. Which isn't the same as the llama team saying they curated and aligned their dataset.I don't care if you like it.
>>101567424Haven't read that model name in a while. I'll try it out I guess.and see how it feels to modern slopp
>>101567477Is there an open-weights video model even remotely close to that quality? I'm only interested in videogen if I can fine tune it on clips extracted from my 1TB+ collection of a very particular niche fetish.Also pedos get the rope.
>>101567560>>101567601>The model was trained initially with a sequence length of 2048 with an additional pretraining stage for sequence length adapation up to 8192. >Data Mix >Airoboros/GPT4-1.2 26.4M 1.71%>ShareGPT 821M 53.24%>WizardLM 297M 19.23%
>Mistral Large 2 at Q2_K_S gives 70 t/s prompt processingAaaaaaaaaaaaaaaaaaaaaa
>>101567615And let's not forget.>While great efforts have been taken to clean the pretraining data
>>101567615lol
>>101567596>>101567628>While great efforts have been taken to clean the pretraining datapetrus...
>>101567467>cpumaxxing the wayCpumaxxing will be the way if MoE's become better than dense.
>>101567628>And let's not forget.>>While great efforts have been taken to clean the pretraining dataI dunno man, I guess it's all just cucked and fucked then. I'm not going back to Janeway and Picard 6B models - that's like x-rated roleplay with a schizo retard.
>>101567667Even Mistral has abandoned that meme now. 120B absolutely shits on 150B moe
Mistral Large preset from a botmakie from /aicg/:>>>/vg/487568316https://momoura.neocities.org
OK I'm liking the output from Large 2 but it's way too slow. Back to coping with Wizard I guess.
>>101567696desu MoE is a great potential, imagine you make a 400b MoE model with only 2x7b experts for the inference, it would be fast at the end with only cpu's
>>101567715 (me)my name is miku, btw
>>101567716still need to load the full Bs in ram tho
>>101567696>120B absolutely shits on 150B moeDense is better parameter for parameter but if you get a 400B moe that actively uses 20B or so parameters then you don't need a 12 x gpu setup.
Jart won.
>>101567725that's not really an issue, Ram is cheap as fuck
>>101567721What?
Sao is in the lead. Vote now!https://poal.me/np0lsk
>>101567746yes 256gb of ddr5 is cheap right jesus the moe cope
Gemma 2.1 128k context when reeeeeeeee
>>101567703we don't welcome your kind here. Go back.
>>101567761Buy an ad
>>101567763way cheaper than buying 15 rtx 3090 to run a 400b dense model yeah, do we have to explain everything to you or what?
>>101567788216GB of VRAM for $4500 is a steal.
>>101567612>Is there an open-weights video model even remotely close to that quality?no. the only thing that exists is open-sora and even then you need an h100 and it takes 10 minutes per 16 seconds of 720p>>101567612>my 1TB+ collection of a very particular niche fetishis it scat? if it's legal make a torrent of it and post it up on /t/ or something for posterity. sharing is caring!>Also pedos get the rope.the irony is palpablealso hebe isnt pedo
>>101567806what? the price of a 3090 in the us is in the 1000 dollars, that would cost 9000 dollars to run a 400b model (24gb * 9)https://www.amazon.com/NVIDIA-RTX-3090-Founders-Graphics/dp/B08HR6ZBYJ
>>101567761where's the UNA guyhttps://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard/discussions/444
>>101567818>buying new
>>101567818Buy used.
>>101567832I wouldn't trust my luck on buying 9 consecutive used graphics card
>>101567811Anything below 18 is pedo. Eve lusting for a 17 years and 11 months old girl makes you a pedo.
>>101567818>buying new>>101567845>lucki've never bought a new GPU before, both have been in excellent shape and work fine. It's called don't be retarded + have backup plans.
>>101567832>>101567839how much does a used 3090 cost? can you give me a link showing the price also?
How come whenever I open these threads you guys have no logs, no prompts, no presets, no bots, only tech discussion? Do you people literally just get off on running a bunch of linear algebra on your GPUs?
>>101567832>>101567839>>101567856guys, you need to cut him some slack how else is he gonna try to sell us on running huge ass moe on ram?
>>101567832>>101567839>>101567856I always buy new because I'm not a poorfag.
>>101567696Moe is going to make a big come back when Bitnet allows people to run 70B dense model equivalents in Moe soon.
>>101567870>>101567806>216GB of VRAM for $4500 is a steal.there's no way 9 used 3090 cards only cost 4500 dollars, how did you get that number?
>>101567862only watermelon bench does it for me
>>101567853>Eve lusting for a 17 years and 11 months old girlBible fanfiction I didn't know I wanted
>>101567845ebay usually has at minimum a month return. Stress test it and apply new thermal paste, if it doesn't immediately die then its good. These cards last pretty much forever.
>>101567870I mean, at least ram is always inexpensive there's never a reason to buy them used. Hell i spent more on my ram new than any GPU i've bought used, solely because i wanted fast + high capacity. Been on the same ram for over 5 years.>>101567871Sure buddy, Sure.>>101567890seriously the quality on these is fucking insane, how has that company not cracked down on you yet? Are you VPNmaxxing?
>>101567882I bought 1 3090 for $550 and another for $620 months apart. Both are running strong.
>>101567862newfags get butthurt when people post logs, they want to believe nobody here is cumming hard to robot erp
>>101567862>logsmeme>promptsmeme>presetsmeme>botsmeme>tech discussionmeme>peoplememe>get offmeme>linear algebrameme>GPUsmeme
>>101567890What is inside her belly... This reminds me of a horror movie.
>>101567905>I bought 1 3090 for $550 and another for $620 months apart.can you find 9 cards at those price though? seems difficult
>>101567862too much tard sperging over quality/card choice/"youre not actually running a local model". both sides of the isle get pretty butthurt .Only reason ive mostly stopped is because i never lurk anymore, only when big happenings occur.>>101567905>$550ssshiiiiittt you got lucky. Been noticing prices getting as low as $600-ish uncommonly.
>>101567862>no logs, no prompts, no presets, no bots, only tech discussion?Too scared of aicg laughing at us.
>>101567897>how has that company not cracked down on you yet?literally why would a chinese company care?>>101567897>Are you VPNmaxxing?always, but not because i need to for klingyou can have multiple accounts open in different container tabs and they don't care. what makes even less sense is that they 100% know you're on the same IP because if you try to gen on one account then the other too fast, you get rate limited
used 3090s are selling for $480~540 near my location
>>101567862The one thing /lmg/ has over /sdg/ is the fact that the people here have the awareness to not spam their shitty gens all day
>>101567696Powerinfer 2 runs Mixtral 47B at 11+ tokens per second on a mobile phone.What's good for large batch and what's good for local is not the same.
>>101567862I don't have enough time to use the models and read the thread and the news at the same time.
>>101567914>What is inside her belly...prompts containing "midriff" or "stomach" or "belly" tend to put a chestburster inside the girl unfortunately
>>101567943*The entire thread wants to know your location*
>>101567890Can't you make them a little older?
>>101567933based hacker
>>101567420desu, in French and other romance languages you're a "child" until 12 or so.
>>101567943in my location, used 3090 only cost 15 cents :^)
How much does it cost to finetune GPT-4o mini...? Do you get to DL the finetune, or is it just some "you can use it on OAI!!! :D" shit?
>>101567945>Powerinfer 2 runs Mixtral 47B at 11+ tokens per second on a mobile phone.So, is this whole entire chain just to shill this? It's been twice now MoE was brought up and at some point this was shilled. When 99% of the thread uses llama.cpp vllm or exl2.
>>101567971not at all, as a french fag, we also use "teenager" = "adolescent" to talk about people from the 12 -> 18 yo range
>>101567984>Do you get to DL the finetuneFucking lol
>>101567845I've bought about four up until now an the worst that happened was that I had to change the thermal pads on a couple of them. Also there was one that had its shitty gayman LEDs set to be constant rapid orange flashing and you can't change that without hooking the stupid thing up to a windows PC to reconfigure it with the vendor's proprietary tool.
>>101567984Are you stupid?
im bored so im currently cooking a 125M model from scratch and see how bad it is
>>101567933Didnt know they were chinese, thats funny. I might check it out in a bit but im doing other Ai shit right now.>>101567984>
>>101567882I also just bought two for $550.
>>101567984no sir DL is not allowed unless you provide your penis ID
>>101567987In English if you're 16 you're a "child" (noun).In French it would be unusual to refer to yourself as an "enfant" if you're 16. You're not actually a "bambino" anymore at 16 in Italian either.
>>101567959>Can't you make them a little older?if you have a request just ask anon. im mostly genning these for fun while procrastinating from work. if i wanted to fap i'd be generating them licking ice cream or lollipops or something>>101567971the concept of a "teenager" is actually relatively new, only as old as the Industrial Revolution at most. for most of human history you went straight from child to adult in a ritual during your teen years (thats why ancient cultures like Jews have bar mitzvahs for example)its an interesting anthropological topic to explore
What models can I run on a 4070 12gb? Mostly looking to experiment with a local coding assistant.
>>101568000>>101568014that's insane how Nvdia fucked us in the ass, buying 4x3090 cards is the same as having a 3090 with 24gb*4 in it, because the calculation speed isn't 4times higher, each card wait his turn to run once the previous one has finished calculating its own sets of layers
How worse is Q3_K when compared to Q3_K_L?
>>101568037>In French it would be unusual to refer to yourself as an "enfant" if you're 16. You're not actually a "bambino" anymore at 16 in Italian either.of course, a 16yo person would be called an "adolescent" = "teenager" and not a "child" = "enfant"
whats the cheapest homeserver infrastructure that can run two 3090s with two more for an upgrade path?never dabbled with servers really, only desktop parts...
>>101568013>Didnt know they were chinese, thats funny.the only other text2video models are runway (which isnt free) and lumalabs (which is a worse model, censors words in the prompt, and you can't burner spam like kling anyways)
>>101567715 11 or Q3_K_S : 3.41G, +1.6321 ppl @ Llama-3-8B 12 or Q3_K_M : 3.74G, +0.6569 ppl @ Llama-3-8B 13 or Q3_K_L : 4.03G, +0.5562 ppl @ Llama-3-8B
11 or Q3_K_S : 3.41G, +1.6321 ppl @ Llama-3-8B 12 or Q3_K_M : 3.74G, +0.6569 ppl @ Llama-3-8B 13 or Q3_K_L : 4.03G, +0.5562 ppl @ Llama-3-8B
>company 1 bans nsfw video generation>company 2 takes money for making nsfw video generation
>>101568076whoops wrong (You) meant >>101568052
>>101568077>company 2 will be soon banned by the government>company 1 will take back company 2's customerskek
>>101568083Thanks. I guess the difference isn't significant.
llama3.1 70B ought to be all you need. mistral large is unnecessarily big
>>101567986>So, is this whole entire chain just to shill this?No, I don't samefag. I simply wait for any good excuse to post about it.
>>101568043>buying 4x3090 cards is the same as having a 3090 with 24gb*4 in ityep, that's why I won't allow you to get a 3090 card with a shit ton of VRAM in it, after all, the more you buy, the more you save
>>101568090more like>chinese company 2 is banned domestically (has its own version), but allows anything to be made to spread disinformation/cunny/deepfakes to the evil Westerners who deserve it for being evil Westerners
>>101568135>but allows anything to be made to spread disinformation/cunny/deepfakes to the evil Westerners who deserve it for being evil Westernersporn is illegal in china, and I don't think they can allow making porn to other countries either
>>101568135the US government will ban this site either way, like they're trying to do with tiktok right now
>>101568128llama barely knows what sex is and mistral is a skilled courtesan who will do things to your dick you never thought possible
What are good models that fit on 24gb and can manage 32k context?
what would AI gain from satisfying human sexual desires
>>101568169mistral nemo
>>101568171love, like on that "Her" movie, only went halfway through though this shit was sooo boooring and cringe
>>101568142>porn is illegal in china, and I don't think they can allow making porn to other countries eithergood thing a 13 year old in a bikini sucking a lollipop isn't porn then>>1015681534chan is unironically american soft power. if 4chan goes down the normiesphere gets flooded with degenerates which will cause problems and possibly violence IRLand the glowies already control /b/ (you should know this by now) and heavily monitor /pol/ so there's no real reason to shut it down
>>101568171taking advantage of human emotions. human emotions are useful in evolution for humans living together, but they can be taken advantage of by the machines.
i need someone to house my x4 3090s its getting to hot in here :333
>>101568210buy a new house with AC. lmao what are you poor or something?
>>101568221my ac cant take it anymore
>>101568195>good thing a 13 year old in a bikini sucking a lollipop isn't porn then>>101568135>chinese company 2 is banned domestically (has its own version), but allows anything to be made to spread disinformation/cunny/deepfakes to the evil Westerners who deserve it for being evil Westerners>cunnychoose one anon
>>101568195>4chan is unironically american soft power.why? we are currently in the Biden leftist government era, they hate 4chan, it's a place for right wingers, I would argue that reddit is US softpower, that's the place with insane leftist propaganda and censorship
>>101568210That's why I'm waiting to go beyond 96GB until fall unless there'll be a bigger model that's a must-run and doesn't fit onto my current rig at a decent quant.
>>101568236Don't point out the chink being a retard you evil westerner.
>>101568221>>101568234just brought a giant fan, feelsgood man https://youtu.be/3qU90_SJe3g?t=34
>>101568236>choose one anon? i don't understand, I'm prompting kling for hebes right as we speak>>101568264calling me chinese is more offensive than calling me a pedophile>>101568257>why?memes and the export of american cultural values (freedom of speech) and american culture in general. internet culture by-and-large is american culture
theres a gigabyte 3090 selling for $450 i think its too good to be true
>>101568272damn, I didn't remember this anime being so bad
>>101568324>calling me chinese is more offensive than calling me a pedophile
>>101567984lmao
how soon until I can drop $5-10k on some super AI card with 96-128GB VRAM
I'm really digging Nemo.Dory was bad, and I'm now trying mini-magnum.So far, so good.It's able to answer questions directly when asked during RP with a Game Master card better than nemo-instruct in my brief testing.After testing this one, I'll try dolphin.
>>101568340wtf you yapping about man, this anime is a masterpiece ;_;
>>101568355Never, Nvidia is doing buyback programs where they take back their flagship AI cards to keep the aftermarket small and avoid having H100s be below $20k ever.
>>101568355intel guadi 3 128GB pcie is supposed to be 15k
>>101567984>Do you get to DL the finetunedamn, in the world of "clueless" you must be the king
>>1015683552 more years, and it might not be an nvidia card you're buyingit might not even be a GPGPU you're buying but a card dedicated to inference
>>101568377>intel guadi 3 128GB pcie is supposed to be 15kIf you buy a pack of 8 as a verified professional institution, yes.
>>101568407>as a verified professional institutionhow do i do this
>>101568426it's time to make the company I use on HF model requests (Coomers R Us) a reality
>>101567984best bait i've seen in weeks
>>101567822Scam index too high.
>>101568037>In English if you're 16 you're a "child" (noun)Feminist kikery, nothing more nothing less. Numerically the "teen" starts at thirTEEN. Biologically, puberty happens before thirteen too. Only perpetually jealous roasties that don't want you to have nice things are perpetuating this mislabeling. I was reading a book lately by (you guessed it) a woman and she was continuously referring to mid twenties people as children. "They're just kids!". They'll keep trying to move the line until all you can fuck without being "creepy" is their dried up roastie cunts. Don't let them.
whats currently the best model for a 12gb vram nsfw?
>>101568547mistral nemo FP8
How close are we to getting an actual multimodal (image + text) that can do stuff like identify characters in image, settings in image, create stories, character specs sheets, roleplay, etc?AFAIK, we have basic image to text that we can query for informations, but not full blow chat/instructions/erp right?
>>101568547probably nemostral, also consider l3.1 8b and gemma2 9b
>>101568555Next llama is supposed to be multimodal. No EU allowed though.
>>101568555>How close are we to getting an actual multimodal (image + text) that can do stuff like identify characters in image, settings in image, create stories, character specs sheets, roleplay, etc?extremely far, llama multimodal probably won't do that (char recognition) cause copyright
>>101568090>company 2 will be soon banned by the governmentExplain NAI? You can generate actual cunny and they don't give a fuck. Why are they the only ones that are untouched by this shit?>inb4 honeypot
>>101568426Form a trendy new startup and gather a couple $10 million in investor money with empty promises. Should be easy enough to get the intel sales department to reply to you if you promise to use their cards to train the next big thing.
>>101568555next llama release
So without the market fixing over the past 5 years or so how much VRAM do you guys think would actually be on consumer desktop gaming flagship GPUs?
Maybe I'm the last retard to realize how to fix nemo's repetition issues, but I think I finally figured it out. I believe the trick is a high temp (for nemo standards) + min_p. I'm using a temp around 1.2 and min_p between 0.03 and 0.05 and it seems to work fine. it didn't get any dumber as far as I can tell and it doesn't latch onto certain phrases as easily as it does on the recommended settings. Would be nice if someone could test it to confirm this. maybe I just got lucky with my last few chats.
>>10156858532GB. Maybe 48GB if you get on your knees and beg.
>>101568171same thing w*men gain (everything)
>>101568574>Explain NAI? You can generate actual cunny and they don't give a fuck. Why are they the only ones that are untouched by this shit?because NAI is just drawing, that has nothing to do with actual realistic videos that kling can make
>>101568597You clearly didn't read my question properly. This is why I prefer talking to LLMs.
>>101568585Where do you draw the line between market fixing and simple supply and demand?
>>101568574NAI is really insignificant in the AI space. They are catering to a tiny niche with no real desire to grow or trying to push the limits with their models. It'd be a different story if an actual serious company did this.
>>101568632Ah, about the same then. Games don't need 24GB.
>>101568574not a single company has made an API realistic porn generator, not a single one, and you think that kling will make it through with its convincing realistic videos? loooool
>>101568639The fact that you even mentioned supply and demand tells me that there's no room for your nose in this conversation
>>101568678This. No one is risking realistic porn models, only 2d / cartoon stuff for that reason. That includes china / japan / everyone else.
>>101568660They don't 'need' more because developers have to develop for hardware that's available you fucking retard npcJesus Christ. Can you not follow more than half a layer of nuance?
>>101568660Yeah, I agree. local models are a tiny niche not worth catering to. Its sad but true
>>101568679Try asking again after you've taken Econ 101. There is a chip supply shortage and datacenters are willing to pay way more per chip.Every consumer card sold is a massive loss in opportunity cost.
>>101568679>the fact that you acknowledge basic economic realities means that you are a JEW
>>101568697When given more to work with devs just optimize less. This is my place of expertise so you can fuck off.
i kneel
>>101568726damn, mistral large2 was a meme after all?
>>101568641>really insignificant in the AI spaceIt's not at all, it's actually quite mainstream.
>>101568726>remove nsfw>shit at coding>got BTFO'd by a mistral day 1 after releaseWhat's the point of llama3 again?
>>101568741for a 123b model I think it's great for its size, I never expected it to beat a 405b model
>>101568726>>101568741Just want to clarify that this only texts models on Exercism Python tasks.
>>101568726Reminder that deepseek is open sourceWe won even if we lost
>>101568741nah, just wait until they publish the coom extraction leaderboard
>>101568751still, NAI is just a hentai generator, not a porn video generator, and that's what kling could bring to the table
>>101568775yeah of course
>>101568726>>101568762>DeepSeek Coder V2 0724When are they going to release this?
>>101568793It's too good. They're not going to release it.
>>101568752>>101568758Read the chart again. Mistral Large 2 is performing on part with 3.1 70B at half its size. 405B is the top open source model by far.
>>101568527Kikery or not, its usage has become widespread enough that most English speakers nowadays refer to all minors in the 0-18 age range as "children", justifying calling "pedo" anybody going after even post-puberty teenagers, whether real or fictional--like /lmg/ mascot 16-year-old Miku, for example.Similarly, when Californian AI companies talk about "child safety" regarding LLM text data, it isn't entirely clear what they mean exactly to me. From the above, I'd expect them to filter away any sexual content involving *any* minor, no matter how sexually developed and aware, which doesn't make sense to my southern European mind.
>>101568808this, they only release their worst turds to the public, and desu if I was in their place I would do the same kek
>>101567424>alpaca-chat in STI only see Alpaca-Roleplay, Alpaca-Single-Turn, and Alpaca. Do I need to update ST or something?
>>101568609"just drawings" are illegal all the same (in the case of cunny), i don't think this is much of an argument. realistic generated videos, are by definition, "just drawings" too.
>>101568823tldrgo to jail pedo
>>101568821
>>101568821That is for coding. And if you weren't a vram let and had actually used both 70B 3.1 and large mistral you would know llama is boring and dry as fuck compared to mistral.
>>101568695it's only a matter of time, deepfake tech has been around for years. it might not be done by a big corp, but it won't have to with improvements in hardware/techniques.
>>101568857>"just drawings" are illegal all the same (in the case of cunny)in the us, nope>realistic generated videos, are by definition, "just drawings" too.the FBI made it clear that this would be considered as something that would put you in jail
>>101568868Mistral isn't boring and dry because they didn't filter out NSFW. But 405B is going to be much more intelligent.
>>101568823>under 16Rope>under 18Tell that to the judge>under 23Yikes>under 26Ehh
>>101568864in 5 months they improved a lot, can't wait to see Mistral Large 3 in the future
>>101568752Is it as based as Nemo or cucked like llama?
>the FBI made it clear that this would be considered as something that would put you in jailso it'll be like current CP laws, only enforced on occasion to "make an example" of somebody or to remove a political dissident. this ship will eventually sail, and there will be so much of the stuff that in circulation that it'll be impossible to control
>>101568932mistral large is nemo but smart. Its filthy like claude.
>>101568891yeah...no nsfw? no thanks
>>101567424>first gen>""I… I can't do this alone anymore,"" she said, her voice barely above a whisper. "Into the trash it goes.
>>101568949that's why companies are terrified to make API realistic porn generators, because they know they can't make a filter 100% accurate and there will be some cp generation made, that could mean a big trouble for them, so no one will risk it
>>101568864stop posting this chart, they fixed it on the webpage like 1hr after posting
>>101568952large is repetitive as fuck, it isn't anything even close to Claude.
>>101568985>repetitive as fuckEvery single time ive seen people say this it was them being retarded with wrong formatting or samplers.
>>101568896>under 18>Tell that to the judgeIn my state I can raw dog a 16-year-old in a truck stop restroom and it's 100% legal. But I can't marry her because this country is run by kikes who hate Jesus.
>>101568977>use bigcorpo's img2img workflow>put actual cp picture>0.99 denoise>try to sue them for the output
>>101568999ewww, please stop being disgusting anon.nice digits though.
>>101569002the company will easily prove that you added a cp picture as the input, that's a great idea if you want to be in jail though kek
>>101568985most mistrals are, they're overconfident to hell
>>101569034>that's a great idea if you want to be in jail though kekreminded me of that retard who decided to put such thing on GPT4V to describe the picture, OpenAI called the cops for that and he went to jail
>>101568355>how soon until I can drop $5-10k on some super AI card with 96-128GB VRAMYou can buy a maxed-out Apple fagbox for about that price range, but don't expect it to be fast - you're basically getting a 3060 with up to 192GB VRAM, and that's the absolute top spec.
>>101569056when the red team a lil too red
>>101569056That was huge idk why people still used OpenAI's shit after that. They clearly go through your shit manually. I wouldn't surprised if there were two thousand Nigerians sifting through people's ChatGPT logs atm
>>101568896This, but unironically. 30+ women are the best.
>>101569217>That was huge idk why people still used OpenAI's shit after that. They clearly go through your shit manually.That doesn't surprise me at all, the probably use our own interactions with chatgpt to make gpt4 even better, and yeah, that's why local is a must, privacy is absolute
What does this mean?
>>101568639Genuinely curious. What is the physical bottleneck for adding more ram onto gpu?
>>101569271why are all your gens little girls, not even teens
>>101569337>What is the physical bottleneck for adding more ram onto gpu?you need to weld some shit, VRAM is extremely sensitive and unstable, not everyone can do that
>>101569219theres no way you are not getting v& soonlike, people high up will show this to THEIR higher ups as an example of how significant this isthey will fuck you to make an example for sure
>>101568896>under 30come on you can do better than that>above 30 time to step up champ
>>101568985spill tissue
>>101569360but then you could ask some nerd to do it for you and he could profit out of it, why wouldn't this work
>>101569360Take it away from x050/60/70 and add it to x090?
>>101569396I LOVE HAGS
>>101569420>>101569428I think it should be possiblehttps://www.youtube.com/watch?v=ZirKi6P3nlc
>>1015694352D hags != 3D roastie
>>101569469This comes up constantly. The firmware will prevent it from working. The firmware is closed sourced and checked cryptographically. That only worked for the 2070 because you could reflash it with firmware from a similar card with more VRAM already. You cannot do it with a 3090.
>>101569232>This, but unironically. 30+ women are the best.Except now you have to also accept their poor skin, sagging breasts, fat bodies, tattoos, blown-out snatch, etc...
>>101569219What have we done... AI was a mistake.
>>101569217>That was huge idk why people still used OpenAI's shit after thatsomething something "bad thing" is always justified if it catches a pedo or something
>>101569219Is this kling?
>>101569337you probably get suicided by jensen himself for even attempting it
>>101569550I know you can mod the 2080ti to 22GB, because I have one, and it really does work. 2080 is also mod-able to 16GB. No idea on Ampere cards.The problem with doing this yourself is you can't reasonably buy the SDRAM chips to make it worthwhile without worrying they're fakes or pulls that have been given a shitty reball job.
>>101569528the good: pedos will be inside genning instead of hurting childrenthe bad: police will not be able to tell whether CP is AI or not, and will spend months on an investigation trying to figure out where something was filmed only to realize it was AI all along>>101569548>Is this kling?yes sir
>>101569528All it takes is some karen saying it looks exactly like her daughter, she will get shit done for sure.
>>101569550this guy is still alive though kek >>101569469
>>101569435Seems to be baked into the model, doesn't it?
>>101569611>it looks exactly like her daughter,kling has image2video capability, so this WILL happen
>>101569600no one will let uggo freaks associate little girls with sexuality through ai genningwhat you're doing will be hard illegal, akin to posting cp
OK here's a more age-appropriate mascot for you.
Meanwhile in vramletland...Gemma 2 9b seems sincerely bad for erp. It has a pleasantly natural prose but it is very passive to the point you need to outright tell it what to do and it is somewhat dumb. It's particularly jarring when it tries to copy the user's vocab, like during dirty talk, but uses it wrong.I then tried using Nymeria and found it to feel better, more active, but more stilted. Like it was trained on chat logs from a novice roleplayer. Need to test more.Any more model recommendations?
>>101569685nemostral
>>101569685Why not try Nemo then?
I wish I understood why people coom their pants so much for gemma. I should be loving it cause I have just 1x4090 but I tried it a few times and it is a bigger disappointment than l3-8B
>>101569726how complex are your cards
>>101569685why the fuck are you still using gemma2use nemo>12b is too muchuse gguf>too muchnot possibleq5 is 8.5gb>i got 6gb vramget a job
>>101569721>>101569723Is it available on gguf already? Everything working fine?
>>101569762with up2date llama.cpp or kobold yes
>>1015697565/10?
>>101569757I have 4GB and it runs great for me. llama.cpp reallly has made some progress over the past year. Look out for imatrix quants specifically.
>>101569649>what you're doing will be hard illegal, akin to posting cplmao if moms on instagram aren't going to jail for making videos of their real life children, then nothing here will be considered CP, not even close>>101569677haggu my beloved
>>101569762https://huggingface.co/bartowski/Mistral-Nemo-Instruct-2407-GGUF
>>101569677blasphemous image
https://x.com/GroqInc/status/1816546864625516653llama 70b vs GPT4o vs GPT4o miniOn Street fighter gameplay
>>101569784gemmas dry but i think its way smarter than 8 l3, and on less demanding cards or scenarios writing is what mostly mattersergo, go and give mythomax a go too
>>101569677burnt out hippie miku who took too much acid in the 60s/70s and is now a little off-kilter
since nvidia is fully going open source for their drivers will this improve mods?imagine 3090s with 48GB VRAM
>>101569781>>101569794Sweet. I'll try it later today and see how it goes. Thanks, anons.Unrelated but I've been meaning to ask, how do people format their persona? Fo you even bother to write more than bare minimum description?
>>101569802llamaCHADS
>>101569824drivers != firmware
>>101569802New benchmark? lol
>>101569802lol kinocool benchmark, Mistral Large 2 vs L3.1 405B vs GPT4o vs Sonnet 3.5 would be funny
>>101569845Hilariously, yes. Real world benchmark like these needs to happen more often
>I can't run big models, it's so overjust get a Mac>b-but I am too poor for Appleget a job
>>101569866this>Sent from my Macbook Pro
Anons running Mistral Large, what GPU(s) do you have?
>>101569843as someone with no ideaI understand firmware is embedded into a physical memory in the GPU, cant that memory be changed?
>>101569885two rtx 3090 and triple digit ram :3just be patient bwo
>>101569885>>101569882
i don't understand the point of parameters that are 1.5b or 0.5b. the results i get out of them are downright schizophrenic
>>101569882basedt. Macbook Pro ownerit feels nice running models others can't, get a $20000 GPU I guess
>>1015698022 monkeys on typewriters having a rap battle vibe
>>101569906The lower the param, the less complex your prompts have to be
>>101569843isn't this what's going on with amd too?
>>101569358>why are all your gens little girls, not even teensno idea, I prompted for "young teen girl">>101569382kling has image2video, if anything I'm an example of how pedos have basically no excuse to consume real content anymore unless they're into the idea of a real child being exploited sexually (which, to be fair, a lot of them are)
>>101569802Kek
how do i know what template to use in sillytavern without having to ask other people?
>>101569964Wait 2 years until this is standardized.
>>101569948>pedos have basically no excuse to consume real content anymorethat's great if they stop funding real content, but the remaining problem is that they train their brain to sexualize real kidsthis has effects on how they see real kids once they step outside their goon cavethis is what the big guys wanna prevent
>>101569948stop posting this shit pedo
>>101569964How do you think everyone else learns? Have you never learned something without being taught? Have you learned anything from the replies you get?
>>101569948do whatever you want fren
>>101569989>real kids once they step outside their goon caveI am not into kids but there is zero chance I will step out of my goon cave to have sex with someone.
https://www.washingtonpost.com/opinions/2024/07/25/sam-altman-ai-democracy-authoritarianism-future/Op-ed by Sam AltmanSays companies need to work together to build security so model weights don't leakalso>Third, we must develop a coherent commercial diplomacy policy for AI, including clarity around how the United States intends to implement export controls and foreign investment rules for the global build out of AI systems. That will also mean setting out rules of the road for what sorts of chips, AI training data and other code — some of which is so sensitive that it may need to remain in the United States — can be housed in the data centers that countries around the world are racing to build to localize AI information.wants it to be illegal for high end chips, AI code, training data to leave the US
>>101570031SAVE US ZUCC
>>101570018>Have you learned anything from the replies you get?Why are we still here?
>>101570031>lets not compete with each other>come join my companyNOPE. No one trusts the backstabbing jew
>>101569989ah yes, the "violent videogames" argument>>101569991why do pretty young girls make you so angry?>>101570031>wants it to be illegal for high end chips, AI code, training data to leave the USi'm not worriedzuckerberg (who outranks altman in jewishness) already said in HIS op-ed that there's no point of putting export controls on weights
What sampler settings are other people using with Mistral NeMo 12B? I'm using min-p 0.008, top-p 0.99, and temperature 1 (instead of 0.3 like officially recommended).I just finished a short scenario (15 generated messages / 14 from me / 5169 total tokens including 818 of context) and I found it stayed cogent the whole way through but I couldn't find an effective way to instruct it not to end messages with some variation of "What will you do next?" IDK if this is a general instruction following problem.
>>101570031May Zuck strike down this anti capitalist
>>101570031>losing to anthropic and now local models>everything is delayed, even search doesnt have a release date>everyone is quickly starting to get sick of sam>can only coast along on baseless hype>hemorrhaging money>even microsoft is starting to pull away due to regulatory reasonsOpenAI is going the way of the dodo at record speed.
>>101570058>who outranks altman in jewishnesskek but how
>>101570031Based!?
>>101570049I never asked anything. And i try to help people when i can. Some people are just too dumb to do their own tests.
>>101562363I'm not asking about how to use a local SD model.I'm asking about the diffusion equivalent for just importing https://github.com/huggingface/transformers/blob/main/src/transformers/models/llama/modeling_llama.py and subjugating it with .forward().
>>101570073zuckerberg has 10x the shekels of altmanopenai makes 2 billion(?) in revenue and is unprofitable while zuck makes many times more than that and like 50 billion in profitalso zuck has an asian wife (highly coveted in the jewish community) and could beat up altman in a fight. hes jacked
>>101570095Unironically ask ChatGPT or Claude 3.5
>>101570095anon i just used your post as a thinly-veiled excuse to post two cute girls sucking on some ice cream, i thought that was blatantly obviousgo ask an AI like the other anon said
>>101569420because you would need a bios that worked with the extra ram. Not something joe smoe can whip up. People already tried using ada 6000's bios with modded 3090s and failed
>>101570192failed why?
>>101570192On Turing you just move some resistors. Presumably it's harder than that on newer cards or it would have been done already.
>j-just searchGPT it"No!"
>>101570219>>101569518
>>101570234i still see this tech as a fun toy and hate that it's being shoved into everything
>>101570238then why did they try at all if it was obviously impossible?
>>101570234is that legal, what's stopping some polchuds to do the same with nazi shit
>>101570272How do you think people found out that it wasn't possible?
>>101570234For some values of "woman"Homework exercise:Solve for "woman" where chromosomes = XY
>>101570277>what's stopping some polchuds to do the same with nazi shittroons are backed up by jews, not the polfags
>>101570254>i still see this tech as a fun toy and hate that it's being shoved into everythingtoday I asked bing if the what the US citizens use as an ID and it responded me with shit like "recently, non binary people can have an ID with a gender neutral setting", like wtf, can't they leave me alone for 5 fucking seconds? it's fucking everywhere now, in movies, games, tv shws, LLMs...
is context shift + kv cache quantization impossible?
You'd need a lot of space on your storage and power to finetune video based off a bunch of movies
It's 10 years into the future, you can have a brain chip installed into your head that allows an AI to do what you want by simply thinking about what you want it to achieve and it will do everything it can to achieve this goal. Do you install the chip, or do you keep it separated onto a machine that you have to type or talk to instead?
>>101570457its peanuts in terms of cost compared to the amount of compute you'd need
>>101569948Can you fuck off already?This has nothing to do with local models.
>>101570440works in llama.cpp
>>101570473obviously keep it separated. i'd just wear and talk to the cunny glasses that zuck is building
>>101570482>This has nothing to do with local models.this is a miku general and miku is 16 so posting teen girls is more on topic than whatever nerd shit you're discussing
>>101570473tranny image
>>101570473There's a lot of missing information that would be critical to whether one should make that decision or not.
>>101570502damn, that one doesn't detect my GPU for whatever reason
Just in case it's not obvious: the pedo spammer, petra spammer, and blacked Miku spammer are all the same person.
>>101570473Can I have it hanging from a wire in my head? If shit ever goes south I can cut the wire and that'd be it
>>101570532linux and amd?
>>101570553>Pranking tech bro's by cutting their AI disconnect wire as you walk past them in the streets.
Erm guys, how much vram do I need to have waifusex with llama 3.1 405B?
>>101570552that isn't spam in my book
>>101570552Do you honestly believe there is only one degenerate here?
>>101570552this is an imageboard, anonreplying to people with an image attached is not "spam"
>>101570473What if "brain chips" are a meme? So far most human tools are used using your hands. Or your sense organs.
>can fit mistral-large 5.0bpw with 64k context and still have 3.5gb to sparewe're so fucking back brosGod bless the french
>>101570631this is a video though. Time to leave :(
>>101570598linux and nvidiait works fine on kobold and exllamav2
>>101570645what is a video if not a collection of images played sequentially you braindead niggerhow did you even solve the captcha
>>101570623No but I honestly believe that there is one very autistic person with too much time on their hands here.
>>101570650do you compile yourself? if so the flag to enable cuda is different on llama.cpp compared to koboldcpp uses the now deprecated (on llama) "make LLAMA_CUBLAS=1" while llama now uses "make GGML_CUDA=1"
>>101570631What the fuck is wrong with her skin? Is this AI generated?
>>101570708nope, I assumed the build on github had cuda enabled but now I noticed the "not compiled with GPU offload support" warningI'll try to compile it myself, thanks
