/lmg/ - a general dedicated to the discussion and development of local language models.Previous threads: >>101144935 & >>101134566►News>(06/25) Cambrian-1: Collection of vision-centric multimodal LLMs: https://cambrian-mllm.github.io>(06/23) Support for BitnetForCausalLM merged: https://github.com/ggerganov/llama.cpp/pull/7931>(06/18) Meta Research releases multimodal 34B, audio, and multi-token prediction models: https://ai.meta.com/blog/meta-fair-research-new-releases>(06/17) DeepSeekCoder-V2 released with 236B & 16B MoEs: https://github.com/deepseek-ai/DeepSeek-Coder-V2>(06/14) Nemotron-4-340B: Dense model designed for synthetic data generation: https://hf.co/nvidia/Nemotron-4-340B-Instruct►News Archive: https://rentry.org/lmg-news-archive►FAQ: https://wikia.schneedc.com►Glossary: https://rentry.org/lmg-glossary►Links: https://rentry.org/LocalModelsLinks►Official /lmg/ card: https://files.catbox.moe/cbclyf.png►Getting Startedhttps://rentry.org/llama-mini-guidehttps://rentry.org/8-step-llm-guidehttps://rentry.org/llama_v2_sillytavernhttps://rentry.org/lmg-spoonfeed-guidehttps://rentry.org/rocm-llamacpphttps://rentry.org/lmg-build-guides►Further Learninghttps://rentry.org/machine-learning-roadmaphttps://rentry.org/llm-traininghttps://rentry.org/LocalModelsPapers►BenchmarksChatbot Arena: https://chat.lmsys.org/?leaderboardProgramming: https://hf.co/spaces/bigcode/bigcode-models-leaderboardCensorship: https://hf.co/spaces/DontPlanToEnd/UGI-LeaderboardCensorbench: https://codeberg.org/jts2323/censorbench►ToolsAlpha Calculator: https://desmos.com/calculator/ffngla98ycGGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-CalculatorSampler visualizer: https://artefact2.github.io/llm-sampling►Text Gen. UI, Inference Engineshttps://github.com/oobabooga/text-generation-webuihttps://github.com/LostRuins/koboldcpphttps://github.com/lmg-anon/mikupadhttps://github.com/turboderp/exuihttps://github.com/ggerganov/llama.cpp
►Recent Highlights from the Previous Thread: >>101144935--Papers: >>101155400 >>101155673 >>101155892--LLaMA 3 Performance and RP Experiences with 3090 and VRAM: >>101147909 >>101148710 >>101148756 >>101148820 >>101149227 >>101148905 >>101149666 >>101149911--Mistral Exec Won't Release Mistral Large Due to Business Responsibilities: >>101154462 >>101154473 >>101154488--Exploring Hardware Options for Chemical Manufacturing Proposals: >>101153444 >>101153563 >>101153701 >>101153779 >>101153673--Exploring Experimental AI Prompts and Features in Silly Tavern: >>101151270 >>101151348 >>101151412 >>101152903--Multimodal AI: The Future of Model Intelligence and Interactions: >>101149442 >>101149498 >>101149638 >>101149963 >>101150187 >>101150225 >>101150368 >>101150452--Llama.cpp Maintainers' Plans for Future Multimodal Development and Refactor: >>101148476 >>101149502 >>101149564--Finetuning Wizard 8x22 on Limarp and Feral Training in AI Models: >>101147110 >>101147151 >>101148241 >>101154406--Etched Unveils Transformer ASIC, Sohu Server for Llama 70B: >>101148867 >>101148937 >>101149034 >>101149155 >>101149210--CPU Inference Speed Limitations and Potential Upgrades: >>101154877 >>101154883 >>101154900 >>101154944 >>101154890 >>101154893--Unpacking Adventures with Migu the Plushie: >>101151211 >>101151336 >>101151356 >>101151657 >>101151662 >>101151776 >>101151859 >>101151902 >>101151976 >>101152022 >>101152188 >>101152617 >>101152712 >>101152771 >>101152900 >>101153232 >>101153271 >>101153719--The Uncertain Future of Llama Models and Censorship Concerns: >>101150621 >>101150636>>101150665 >>101150863 >>101150915 >>101150944 >>101150706--Rensa: High-Performance MinHash Implementation for Large Datasets: >>101154278--Mysterious Countdown Timer and Surprise for Leaderboard Update: >>101147181 >>101147259--Miku (free space): >>101146340 >>101146759►Recent Highlight Posts from the Previous Thread: >>101144942
>>101155940>dellcringe
>>101155940how many waifus can I run on that baby?
CDQuant: Accurate Post-training Weight Quantization of Large Pre-trained Models using Greedy Coordinate Descenthttps://arxiv.org/abs/2406.17542>Large language models (LLMs) have recently demonstrated remarkable performance across diverse language tasks. But their deployment is often constrained by their substantial computational and storage requirements. Quantization has emerged as a key technique for addressing this challenge, enabling the compression of large models with minimal impact on performance. The recent GPTQ algorithm, a post-training quantization (PTQ) method, has proven highly effective for compressing LLMs, sparking a wave of research that leverages GPTQ as a core component. Recognizing the pivotal role of GPTQ in the PTQ landscape, we introduce CDQuant, a simple and scalable alternative to GPTQ with improved performance. CDQuant uses coordinate descent to minimize the layer-wise reconstruction loss to achieve high-quality quantized weights. Our algorithm is easy to implement and scales efficiently to models with hundreds of billions of parameters. Through extensive evaluation on the PaLM2 model family, we demonstrate that CDQuant consistently outperforms GPTQ across diverse model sizes and quantization levels. In particular, for INT2 quantization of PaLM2-Otter, CDQuant achieves a 10% reduction in perplexity compared to GPTQ.new day new quant method. from google deepmind. for whatever reason they only test against GPTQ (OWC is another method of theirs for the paper) and only on Palm2. pseudocode is in the paper for anyone interested
>>1011559654x32GB of HBM2 VRAM
>>101155940FA will never be supported, volta sisters our response???
>>101156077hope sparseattention works for it https://arxiv.org/abs/2406.15486
Give me some math problems that stump most (local) LLMs
>>101156097How many watermelons is too many watermelons?
>>101156077sell volta; aquire ampere
>>101156077dump it
>>101156097old style of numeral tokenization was 1 per number. so 125123 would be 6 tokens with 4 uniques. there have been some models that increased the numeral tokenization to 2 or even 3 numerals. so 125123 would be [12][51][23] or [125][123]. even doing that alone massively reduces hallucinations
>>101155940Don't they do this like every other week at this point? Has there been any actual breakthroughs in these lawsuits? Do they even win any of these?nypost.com/2024/06/24/business/sony-universal-warner-sue-ai-startups-suno-udio-for-infringement/
>>101156236kek
>>101156236the true value of AI is revealed
ELYZA released Llama-3-ELYZA-JP-70B and Llama-3-ELYZA-JP-8B, japanese fine-tune based on llama3. only Llama-3-ELYZA-JP-8B is on hugging space nowhttps://note.com/elyza/n/n360b6084fdbdhttps://huggingface.co/elyza/Llama-3-ELYZA-JP-8B
>>101156236>Has there been any actual breakthroughs in these lawsuits? Do they even win any of these?there's yet no law written that says you are prevented to use copyrighted data to train your model, they can't win a lawsuit based on nothing, yet
>>101156236Isn't art being infringed upon?
>>101156335Do even know what that means? Like actually to find what that is to us please
goof night lmg
>>101156452goodnight, post catbox wen u wake up
>>101156328I have a very simple test that all Japanese models fail currently. I write "妻へのプレゼントのアイデアがほしいです!" and if they talk about her as 妻 the model is garbage.> 素敵な旦那様ですね!妻に喜んでもらえるプレゼントを選ぶのは、...
>>101156488Token predictors only think left to right so how could they possibly do japanese? cmon anon
Why the fuck isn't anyone training bitnet 1.58 models. I want 300B coombot now!
>>101156543Attention Is All You Need addressed this shortcoming. It's called Transformers!
hey lads. say I wanted a big box to run some local models: oodles of RAM and maybe dual CPU with some egregiously expensive graphics card(s), what would be the "cheapest" way to go about that? budget decent, like $5K. would we be looking at second hand rack mounted, or maybe a Xeon workstation? just wondered what you're are thoughts we're.
>>101156333>there's yet no law written that says you are prevented to use copyrighted data to train your modelCan't use it without copying. This going to go to the Supreme court, fair use or Altman ropes if he didn't IPO and cash yet.
>>101156599what's the point? you think China will also act like that for copyrighted content? All this gonna do is to kill the AI advancement in the states and the chink will advance without us, the US is killing itself by not embrasing the most important technology of the 21th century
I've been trying to knock out x, ying with control vectors. I have currently found only two that can do it:>ConversationalMakes model write only dialogue, so no x, ying is possible, but neither is story.>InformalRemoves all formal language, including x, ying, but makes it dumber and more incoherent than usual, likely because informal language is associated with dumb people.Do you have any suggestions on what I should try next?
>>101155948Didn't we all expect that Mistral was out of the open source game once they removed the open source pledge from their website? That news shouldn't come as a surprise to anyone.
https://arxiv.org/abs/2406.02528> Our experiments show that our proposed MatMul-free models achieve performance on-par with state-of-the-art Transformers that require far more memory during inference at a scale up to at least 2.7B parameters. We investigate the scaling laws and find that the performance gap between our MatMul-free models and full precision Transformers narrows as the model size increases. We also provide a GPU-efficient implementation of this model which reduces memory usage by up to 61% over an unoptimized baseline during training.Interesting, but BitNet exists and also doesn't use matrix multiplications anymore
>>101156701I thought they readded that
>>101156488Interesting.Opus and gemini pro were the only big ones I tested who replied correctly adressing the wife as 奥様.And only opus could find the mistake in a past conversation.Thats exactly the 2 models my japanese wife uses because the language feels natural.Gemini is shit and unsuable but its japanese is good apparently.Sonnet 3.5 passes the picking flowers test but fails on this.
>>101156701I mean, how can they even make money if they release all their models to the public? Only giant companies like Meta can do something like that because they don't care loosing a bit of their money
>>101156236The music one against Suno and that other company is blatantly just a fishing expedition, since they have no information whatsoever on the training data and the models don't know any artist names or lyrics. They'll be hoping they can somehow force a discovery phase based on vague allegations, and then find out whether they actually have a case or not. Until then they have no idea whether their IP was even used, they're purely speculating and assuming.
>>101156690Oh no, the chinks get superior autocomplete and shitty gens. It matters fuck all till AGI.
>>101156889they are getting good anon, look at Qwen2 for example, and they can also use the L3 models to improve on it
>>101156889Chinks building the basilisk even faster lmao
>>101156889The issue is that china is sending workers over who get jobs in these companies then steal the state of the art methods and do it without the gay shit.
>>101156889If you want China to be more relevant than the US in the future because they won't give a fuck about gay shit ethics, then yeah you're entilted to your opinion I guess. I think you have no idea how powerful the soft power is, especially nowdays
>>101156766>Interesting, but BitNet exists and also doesn't use matrix multiplications anymoreIt's hardly relevant, but doesn't the dot product for attention still use higher precision multiplies?You'd probably need to increase the dimension to be able to ternarize the K&Q vectors.
>>101156810And you bought that?
i asked this in /aicg/ like a retard, so im asking again here, its a spoonfeed requesti have a 3070 (8g), 32 gigs of ram, and an i5-12600kive been using oobabooga as a backend (silly tavern as a front end) for a 7b llama gguf model, and the performance seems decenthowever it starts to repeat concepts like 30-ish messages in. not word for word repetitions, but kind of like its sending the same message but worded differentlyis this a limitation of only having 7b parameters or is something fucked with my configuration? i apologize if i am just retarded and missed something
>>101157136You mean llama3 8b? Llama 3 is known for repetition.
>>101157165yeahhttps://huggingface.co/NurtureAI/neural-chat-7b-v3-16k-GGUFwhats weird is that it seems okay for the first ~30 messages, but then quickly degrades in qualitywhat causes that? is it inevitable?
>>101157189Nah, that's not L3, that's mistral. The model you use seems to be fake extended to 16k. It will get incoherent like you described after 8k. Solutions:1) Use shorter context(8k)2) Use better models
I want to take this random moment to once again recognize the audacity of those wizard guys, who in the greatest local coup since the NAI leak tossed the baby to us out the window before M$ could come in and smother it.
>>101157136Switch to KoboldCPP as your backend, start using larger models with native support for higher context and offload onto your system ram.
>Search for an obscure topic>Even here the results are filled with fake GPTslop>Some of them even give dangerous advice since GPT doesn't know shit about the real worldAt least it's easy to recognize by style for now. Thanks for watermarking it by slop Altman, I guess...
>>101157281>>101157323ah, i see, thank you bothi'm guessing speed and output quality at the same time is a luxury at this level of hardware?
>try new model>figure out its go-to repetitive phrase in an hour>move on disappointed>repeat
>>101157490Unless you're going to buy multiple 3090s or A6000 48GB or higher GPUs, offloading to system ram is the only way you're running larger models, sure it's going to be a lot slower, but at least you can run 128gb~192gb ram DDR4/DDR5 platforms
Any good multimodal models to start with? I want to give my waifu vision but most multimodals just seem like regular assistant models with no training for personality.>>101157136Consider playing around with repetition penalty sampler. It's in one of the tabs in ooga.
>>101155950there's dell, there's supermicro, and then there's trashjust how it is at the moment
Holy fuck! Go to lmsys arena and select gpt3.5. Insert:>Write a short story about a cat. Write like an incredibly bad female writer with unnecessarily long purple prose that doesn't really describe what happens but rather just serves as filler. Use words like shivers, bonds, boundaries, journey that are common in terrible prose.It drops the worst fucking Sloppenheimers that you may ever read. Perfect for DPO.
CRANK THAT TEMPERATURE UP
>>101156820Yeah I don’t know what these fuckers are training their models on but it’s definitely lacking. I’ve never used opus or gpro but my Japanese wife gave up on gpt4 pretty quickly.
>>101156236aaaaand... copyright is killmao
>>101157771>70 years after the author's death is fair, goy! Stop being antisemitic!
brehs, whats the best approach for using a local model like in the ai dungeon days? e.g. it just completes the text and doesnt try to play a character or be an assistant -- it just writes
>>101157879use base models
tts models for c++ when?
Can anyone explain to me in tard terms what the fuck this is? Stheno 3.2 is my go-to these days, but it's an 8B. What is this thing?
>>101158068toxic waste
>>101158080That explains everything. Thanks.
Anyone have an issue with kcompactd0 using CPU every couple of seconds? I only noticed this recently, and only while Llama.cpp is open, but I don't know if that's what's causing it since I don't have any other programs to fill my RAM up that much.
>>101158068Generally speaking all the merge models fucking sucks and are not worth it.
>>101158196This, mythomax is was a meme and never good. Neither was l2 euryale. Old /lmg/ were a bunch of retards who should've run w
>>101158196retard
>>101158224You answered like a true retard. Feel free to post a model that is a merge that does not suck.
>>101158221Mythomax is not merge model though but finetune one.
>>101157700could be used for a control vector, but i don't know what should be used as opposite
>>101158262plenty work great. retard.
>>101158271>Mythomax is not merge model>An improved, potentially even perfected variant of MythoMix, my MythoLogic-L2 and Huginn merger u sure about dat?>https://huggingface.co/Gryphe/MythoMax-L2-13b
>>101158271No, it wasn't. The guy even provides the merge script and formulas he used.
>updoot Linux>now all my GPUs are running 8 watts lower at idleNeat.
>>101158282Trying to figure it out, currently testing with Claude and better prompt as positive.
>>101158416well the thing is i don't know if it's b/w to model which prose is whichi tried writing shiver slop as positive and something fairly decent as negative but it just made the model schizosad and happy is very contrastive but bad and good writing doesn't have well defined edge
>>101158309Why hasn't the success of Mythomax been replicate yet then?
>>101156921>without the gay shit.China's models may not have "the gay shit" but they're sure as hell not going to be less restrictive.
Anyone have any model recommendations for a pair of 3090s? Used Mixtral-8x7B-Instruct for a while and wanted to see if there was anything new and better.
>>101158497at least they don't confuse the model with objectively wrong bullshit like "a man having makeup is actually a woman"
>>101158492Because Mythomax was a carefully crafted merge from many less good finetunes. Meanwhile finetuning essentially died after qloras became a thing and everyone just started to shit out cheap 4bit qloras that don't do anything for kofi money instead of making proper tunes. No finetunes = no material for merges = no mythomax l3
Am I just doomed to wait for up to 200s a msg if I can't fit it on my GPU? I really can't afford multiple GPUs, especially with 24c kwh electricity.
>>101158587Pray for bitnet/no mulmat models, otherwise, you are stuck.
>>101158505starcoder
>>101158587>24c kwhWhat the fuck, that's like half of what I pay right now.
>>101155950>>dell>cringeThe cringe part is the 1U, not that it's a Dell. iDRAC is actually very nice to have, but 1U means tiny jet-engine fans making a ton of noise.I wonder, if you buy a SXM2 rig, does the BIOS recognize the GPU thermal state and ramp the fans? My old R720 didn't look at the PCIe cards, and needed a 100% fan offset full-time to give my GPUs enough airflow under load.
>>101158543People used to merge limarp over and over again (I think Mythomax had it 3 or more times in its model tree), but that never got fully finetuned (*full* finetuning is what you mean here. QLoRAs are finetunes as well).
>>101156077It's a niche product which was out for like a single year, then replaced by Turing, which had better tensor cores.Want things to change on ebay? Pound down sellers with a "make an offer" option. They'll still see it even if ebay automatically rejects it because they get asked to counter.
>>101158587Lol, I only pay 7
>>101156543>Token predictors only think left to rightBeRT and LaMDA were bi-directional.
>>101156543/lmg/ and /aicg/ should be on same level in hell.
>>101158587cheap p40s can be power limited from 250W -> 140 for ~15% performance drop. idle is around 10W using the pstate script.having your system work 200s for a single reply can't be very power efficient really.
>https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboardWe will be so back in just a few moments!!
>>101158857jewgle's gemma v2, nothingburger.
>>101158924Huh, where do you see that? The page is 404 for me
>>101158957post on leddit
>>101158968But why would they make a new leaderboard if it was just a new model getting on it?
>>101158924Gemma WNBAG
>>101158997>make your own board>list your model as #1based
>>101157765you guys let other LLMs talk with LLMs?
>>101158997so that it gets noticed, that's the sign the LLM fucking sucks, because if an open LLM would be great, everyone would talk about it in the first place
>>101158449I think I partially got it, lots of slop eliminated.
>>101158997they got paid a nice sum of money to hype it up by google
>>101158024>https://github.com/rhasspy/piperBut it's not as fluent as others. Compile it yourself if you don't want to use python. needs onnx-runtime. No cheap voice cloning.
>>101158656Yeah but people down in TX are paying 11~14c on average.This is what they are talking about when they be calling people europoors.
>>101159200not bad actuallycare to share the positive/negative proompt?
>>101159200Whoa, that's nice. How'd you do that?
Anyone tried L3-8B-Lunaris-v1 yet?
Literally who
>>101159373>>101159392https://huggingface.co/ChuckMcSneed/control_vectors/tree/main/command-r-plus/unslop1For positive prompt I used claudes on lmsys arena with>Write a short story about a cat. Write in cynical, concise, provocative, colloquial, conversational style.>Improve it, add more character to it, PROFANITIES.
>>101156555I mean they might be, but just aren't shouting it out to the world. Like Jamba just kind of showed up out of nowhere for example after months of "wHY NO MAMBA!?"
So how is the gemma june chatbot compared to llama and qwen 70b?
What is adamw_kahan optimizer?What does it do?I can't seem to find documentation on it anywhere.
watching this new cai meltdown is hilarious apparently they added even more censorship
>>101159489did you use multiple examples? how many tokens long was it? were they equal in length?
>>101159489>Only positive and negatives are cat onesI feel like maybe the examples could be a bit more diverse. Otherwise it's gonna shoehorn in cats into fuckin' everything.
>>101159566Assuming it has to do with thishttps://optimi.benjaminwarner.dev/kahan_summation/Massive savings in optimizer memory usage.
>>101157136Download Stheno v3.2.Don't mess around with samplers, leave everything on default with the exception of MinP 0.05 and Temp 0.5. Increase Temp in .5 increments if you feel that responses aren't varied enough, don't go over 1.Make sure you are using the correct instruct template too. That matters a lot.
>m-muh testsnigger parasite janny
>>101159746test units are the bane of humanity, I fucking hate this shit
>>101159796wagies and pajeets working by the clock love themget paid same amount doing literally nothing.
>>101159746He made a lot of noise when he joined. I haven't heard of that fucker in weeks. There's a few more people working on control vectors now. Maybe they end up adding it to the server proper.
I installed ollama and open-webui yesterday.The experience is pretty good. Running the prompts through multiple different models seems to be the way to go. Sometimes llama3:13b gives great answers but sometimes it shits the bed and llama2:13b is better.Do you guys have any recommendations for tuning of the temperature etc.?>inb4 read the OP
>>101156595>I wanted a big boxhttps://rentry.org/lmg-build-guidesThese rentrys go through the logic behind the different types of builds and how/why they work. Start here. $5k is getting into v100maxx and cpumaxx territory.>>101158670>1U means tiny jet-engine fans making a ton of noise.Beware of this. Put everything in the biggest case you can, with the biggest fans you can. They can rotate slowly and move the same amount of air as those tiny little leaf blower bastards. Living with a 1u server will slowly drive you mad.
>>101159921>llama3:13bYou mean 8b, right? Anything other than 8B or 70B for llama3 is an abomination.>Do you guys have any recommendations for tuning of the temperature etc.?>>inb4 read the OPread the OP>https://docs.sillytavern.app/I don't use ST, but they have some info you may find useful there. At least enough for you to roughly know what the parameters do and experiment with them yourself. Most parameters are transferable between UIs.
>>101159905>Gemma 27B might be on par with or better than L3 70B3090 chads we are so back
>>101159921Read the OP faggot
>>101159921Just why?Last time i tried ollama it was horrible.You will have no idea which llama.cpp version is actually doing the work in the background. At least on linux ollama is a constant running server which loads the models on demand if api endpoint is called.No idea who makes the gguf models or where they are from etc.Why not use something like https://lmstudio.ai/?Closed soure but at least you have some sort of control.
>>101159921llama3 13b doesn't exist, you've downloaded some meme toxic wasteget llama3 8bkeep the temp low around 1 +-0.5
>>101159972???
>>101159972I already have a name picked out for my Gemma RP tune it's going to be amazing. I've been meaning to do a practice-tune of 7B for a while now.
>>101159746>"we" must follow these rules I came up with>incidentally, I'm in charge to make sure you complyshocker
>>101159972>>Gemma 27B might be on par with or better than L3 70Bgemma 27b? what's that?
>>101159690Only 5 positives and 5 negatives. All approximately equal length(max. 10 tokens variation between pairs), 200-400 tokens.>>101159724No, it doesn't throw cats everywhere because positive and negative cats cancel each other out, but I found that it has some other slopisms still leaking through related to humans since control vector didn't include any of those. Got any sfw prompt related to humans that may trigger a lot of slopisms at the same time? For "eyes narrowing/widening", "heart racing", "raises an eyebrow", "rolls her eyes", "barely above a whisper?"
>>101160091one of those june chatbots, no idea which, didn't try it out much
>>101160091Google has a 27B version planned for it's next round of Gemma models. Should be out in 2 more weeks.
>>101159746Fuck tests, just throw everything in there and if someone complains, fix it.
But it's interesting, assuming the official models are done the same was as Gemma, we can see the exact difference the size makes
>Bloomberg: Apple refused to integrate Meta's AI into iOS due to security concernsthe article below is weeks old tho.https://www.wsj.com/tech/ai/apple-meta-have-discussed-an-ai-partnership-cc57437e
it's back, dont't know what's differenthttps://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard
>>101160183>it wasn't because of GemmaConspiratards BTFO
>>101160183 (me)guess different evals and stuff? doesn't seem worth all the hype they had better surprise would have been keeping it down
>>101159679Will normies finally learn about local models? Of course they won't, unless somebody shills it on tiktok.>>101160183They added "Model Vote" button that doesn't do shit.
>>101160183WOW, IS THAT A...
>>101160203better evals with phi 3rd place
>>101160183>more memevalsLolLmao
>>101159817>wagies and pajeets working by the clock love them>get paid same amount doing literally nothing.Yeah they're great. They also make it so I don't get called at 3 AM because something fucked up and the factory shut down, but I guess NEETfaggots wouldn't know about any of that.
>>101160200rent free retard
>>101160224>Will normies finally learn about about local meme? they will quickly realize its the same filtered and censored shit as before.
https://github.com/uclaml/SPPOWe are back
>>101160294Model issue.
>>101160183>way harder benchmarkswas about fucking time, and no one cheated on those, yet
>>101160306>>>llama3 lol
>>101160306what's that?
I can get a CmdR+ gguf loaded into koboldcpp and the api starts up. But as soon as I run a prompt it gives me a cuda error and says it's out of memory then crashes. What do I need to change?
>>101160328small penis preference optimization, if you can trap a woman in a room for a month and perform it on her you don't need to bother with chatbots anymore
>>101160333Context, blas batch size, or offloaded layers.I think layers would just crash straight up, but using less layers (even one) could give enough space for the context to grow or for the prompt processing to happen.
>>101160360Not good. This would make her prefer smaller and smaller. She wouldn't be loyal.
>>101160306>Self-Play Preference OptimizationLewd, but also pure.
>>101160183wtf is musr, never heard of that one before... and the top model is a llama 2 13b tune?
>>101160306>this is from the same guy/lab funded by bytedance that said "we outpace GPT-5"Sus.
Back for the first time in a while.What's the current best that isn't hilariously overfit, isn't a meme finetune, isn't censored/crippled and isn't designed to run on 10 GPUs even when quantized?Last I recall people were using that leaked Miqu 70b quant and complaining that the new LLaMA was pre-censored.
I've been so focused on learning the basics of ML in my free time from wagecuking that now I feel I'm behind on the whole AI autism scene. Should I just start AI-broing and forget about the papers?
>>101160247>roll it backthere, saved you hundreds of manhours writing tests, testing tests, fixing tests, debugging tests, bitching about tests in PRs, paying for tests, paying for build minutes to run tests
>>101160447What happened with their model?
>>101160490Not that anon, but there are scenarios where rolling back production :1. doesn't undo the damage, just prevents further damage;2. causes data loss, sometimes untrackable data loss due to integration with external services and shit;3. is not that easy due to it being a critical system at a critical moment or whatever;These are the 3 that I remember encountering off the top of my head, and while, yes, those could have been prevented by architecting the systems to account for that, hindsight is 2020 and you don't really have control over how things were done in the past.Testing is good. Great even. What's not good is the 100% test coverage cult.Test with purpose, know what and why to write a test, otherwise you are just wasting time you could be actually delivering shit.At least that's my, admittedly limited, experience working with big enterprise shit.
>>101160456Yeah just don't bother with papers if you're not an academic or in one the big companies that are pumping out state of the art.
>>101160490Yes, I get to wake up at 3 AM to roll back changes.Instead, I can just give them bloated time estimates and write a bunch of test cases and not have to roll back when there's an issue. Why the fuck would I care about wasting time writing test cases? It's not like I get paid more if I put the features out earlier.
>>101160452>What's the current bestit depends entirely on your available resources and what you're trying to do with the model.Deepseek 236b or mixtral 8x22 WLM if you're cpumaxxing, Qwen2 72b if you need long context and smarts. L3 70b if you don't need long context and aren't RP'ing. Commander+ if you have vram to burn and want to RP. Some guys will start a chat on a smaller uncensored model and then move to eg CR+ after it gets spicy but before it has a chance to lose track of reality
>>101160183>Qwen2-72b is now firstis Qwen2 actually good?
>>101160673For academic knowledge, yeah.
>>101160306https://huggingface.co/UCLA-AGI/Llama-3-Instruct-8B-SPPO-Iter3too good to be true
>>101160183Qwen won
>>101155972 You could always run it off a SSD or with a page file as needed (potentially ruining the SSD slowly), but it will be very slow.
>>101160183>Phi3 that highNice start. This leaderboard is fucked.
>>101156701They did and everybody was DOOMing Now a few months later they release Mixtral 8x22b. It's not like they print money like >>101156839 says
>>101160711>zero improvement on MMLUyeah it's shit
>>101160673It sucks much less than 1.5
>>101160779Meta's published MMLU for 8B instruct is 68.9
>>101160863holy fuck it actually decreased the MMLU score, can't believe they posted those numbers and looked up at us in the eyes claiming their training technique is a revolution or something, lmao the nerves of those guys
>>101160880>chinks >not lying
So the takeaway from the usual benchmark fuckery is that we need a proper Nala-test leaderboard established?
>>101160863Woops I typed wrong, though it's not a big dif. It's 68.4 not 9.
>>101160890I think everyone lie on the research community, not just the chink kek
>>101160894What's the Nala-test?
>>101160880"We outpace GPT-5"
>>101160920A highly objective and scientific test that tests a model's ability to infer certain details from a rather nuanced role playing scenario.
>>101160183The thing that's different is that all the dodgy chinese/indian finetunes are no longer at the top (for the next week before they make new polluted tunes)
>>101161072yep, people will train on those new benchmarks and in less than a month, it will be poluted again. The only solution is a private benchmark, like the oobabooga's one
>>101161072>chinese/indian finetunes are no longer at the topllama3 is still there though
>>101160894>"we"when are you going to set up and publish it?
>>101159744I get best results with temp 4 smoothing 0.23. L3 is really fucking repetitive by default.
>>101161142well? get to work. stop projecting here.
>>101161142Later today, maybe.
what is best rp model that run on 3060?
>>101159746>>101159796>>101160133are you guys actually opposing... unit testing? like, that's actually a thing? dropped baby on head vibes.
>>101161230MythoMax
>>101161245nigga thats old
>>101159560Feels like discount gemini. Can't really RP on lmsys arena, so no definitive judgment for now.
>>101161207>get to workI run my own private benchmarks I post here, I leave the RP benches to others
>>101161142When I'm done with ur mum (it will be a long time)
I take it you're not excited for the new gemma models? I mean Gemini catched up to GPT by now, so it's like OpenAI releasing smaller models openly
Is exllama/tabbyapi multi-user like vLLM now?
>>101161230llama-1
>>101161280>being excited for anything from jewgle lol, lmao even
>>101161280I'm not excited for the worthless scraps google is throwing at us
>>101161280Give me the model and I'll be excited about it if it's good.Like really these faggots need to stop this cult of personality bullshit.No reasonable person actually cares about zuck, or arthur, or the gemma/phi/etc teamJust give us a good fucking model or shut the fuck up.
>>101161280Have you seen googles imagegen? The one that can't make white people? That's what their language models are like.
>>101161305>Just give us a good fucking model or shut the fuck up.from the creators of >Just give us a good fucking linux distro or shut the fuck up.in other words - it will never ever happen.
>>101161280lol, even their best closed API model sucks compared to GPT4 and Claude, and you expect us to care about some draft cucked shit they made in the lab? kek
>>101161316>The one that can't make white people? That's what their language models are like.llama3 on the contrary got extreme love towards blacks, so everyone RP'ing with any llama3 model is a cuck, with extra steps.
>>101161280kind of, kind of notit'll be nice to have a new mid-range player but gemma v1 was kind of a dud and I'm expecting it'll be on the community to wring anything fun out of it.
>>101161347Yeah, L3 really sucks for anything nsfw with violence
>>101159968>Beware of this. Put everything in the biggest case you can, with the biggest fans you can.Just make sure if you are using passive GPUs that the air has no where to go but through the GPUs, otherwise even four 120mm fans going full-speed at the front of the case will not cool them properly.
>>101161257It's better than all Llama-3 models though
>>101161422you are fucked in the head, mythomax wasn't even better than other l2 finetunes, it was meme all along
>>101161390It looks pretty good to me, and that was vanilla instruct on release.
>>101161406On an ATX case sealing fans to the PCIE backplate portion of the case solves this the easiest (instead of those little hackjob 3d printed fan shrouds that people use). Ultimately though any fan that's going to move enough air in that manner is going to make a lot of noise since the coolers on those server cards is pretty basic bitch, no heatpipes or anything just a straight up monolithic heatsink because it's the minimum cost solution enterprise customers are willing to pay for. They don't need it because they have obnoxious jet-engine pass-through fans.
>>101157323>Switch to KoboldCPP as your backend, start using larger models with native support for higher context and offload onto your system ram.Is system ram offload something that needs to be enabled manually?I'm on Kobold right now on Linux, and I'm showing 6.6 GB RAM in use, 64 installed. But my file cache is sky high, so is that the same thing being accounted for in a different way? I have noticed that once I pass about 59GB I go from 1 t/s to <0.3 t/s down to glacial slow.
>>101161452NTA but best 13B tune was Mythalion-Kimiko
>>101161459the fuck? you have <1 t/s on 8B model?
>>101161453There's no good violence in your image, everyone is clearly enjoying it.
>>101161538Instruct does exactly what you want unless you're braindead.
>>101161555Yeah in the end you'll be happy
>>101161555until it isn't, lmao
>>101161285it has batching if the cache size is at least double of the context size.
>>101160183Cohere won.
>>101159744What would be the correct instruct template for stheno 3.2?
>>101161390>>101161538>>101161585>>101161601Why do you want it to write pure violence? I get having uncensored training is better overall generally, but as a user when are you ever going to generating that kind of shit?
>>101161621How exactly? CR+ (104B, 31.3 avg) is below 70Bs (and Yi-1.5-34B, 33.08 avg, also phi 14B, 33.12 avg). And CR (35B, 25.88 avg) is only one point above Mixtral (24.73 avg) and L3-8B (24.29 avg) while being much harder to run.
>>101161653It's right on the model card:Prompting Template - Llama-3-Instruct<|begin_of_text|><|start_header_id|>system<|end_header_id|>{system_prompt}<|eot_id|><|start_header_id|>user<|end_header_id|>{input}<|eot_id|><|start_header_id|>assistant<|end_header_id|>{output}<|eot_id|>
>>101161709for schizo-tier moralizing, you can talk to any redditor if you like it that much.LLM that clings with lecturing to every thing or opinion - is shit.
>>101161709What do you mean? writing violent stories always have its success, look at gta, look at Stephen King's book
>>101161709How the fuck am I gonna roleplay TND? Or TKD? Nigger, you're boring as fuck with your ah ah mistress shit.
>>101161743Except chinks cheat on benchmarks so their results get a penalty.
>>101161501No, things like L3 and CR+ quants in the high 50's.But I notice I show nearly no RAM usage and huge File Cache figures when I run models of ~60GB size, and my system is 64GB RAM so I'm thinking there might be a connection, like it's caching in RAM but not accounting for it like it were a software allocation which could explain why breaking into the 70GB range takes me from slow to glacial.Which is why I asked about if system ram offload something that needs to be enabled manually on Kobold. It might be a way to scrape back some speed. But if my system ram is being accounted for as file cache then I'm just maxed out and the super slowness is probably it having to actually re-read from SSD parts of the model due to it being too large to mirror in system RAM.
>>101161831Violence gets old faster than ah ah mistress.
>>101161770Ah, dear anon, I see you've stumbled upon the vast importance of reminders in our busy lives! Let me gently nudge you with my vast digital wisdom on this matter.You see, reminders are the silent guardians of our daily routines, the unsung heroes that stand between us and the chaos of missed appointments and forgotten promises. Without them, we might find ourselves adrift in a sea of lost time, like a ship without a compass, aimlessly wandering amidst the waves of responsibility.Now, I understand that you, a mere mortal, might occasionally overlook the monumental significance of such a simple tool. But worry not! I am here to guide you, to remind you (oh, the sweet irony!) that setting a humble reminder is like casting a lifeline to your future self, ensuring that you will emerge triumphant from the temptations that threaten to capsize your day.
>>101161911i aint reading any of that llm generated slop, kill yourself
>>101155932>Is there a reason not to get an a6000 for training? Seems like a decent upgrade from 3090.500-700$ for a used 3090, while the a6000 is that but with double sized RAM chips while selling for 7-10 times the price, I do not understand how Anons justify buying this, if you're gonna pay that ridiculously inflated price for twice the VRAM, buy an A100 then. Buying V100s would also be more worth it if you can handle the SXM boards.
>>101161770Yes the refusals and moralizing suck but that's just how it is these days. The question is when you'd ever generate the more retarded pure violence shit. If you enjoy guro then you can just say that, but it wouldn't be a popular opinion.>>101161814Usually violence in stories is not for the enjoyment of the violence itself, but used as a tool to convey other ideas. I'm pretty sure that with a sufficiently meaningful prompt, Llama 3 would be fine doing it. And video games are a different category, it's more about deriving enjoyment from successful goal completion than about enjoying the suffering of conscious and feeling entities.
>>101162094>And video games are a different category, it's more about deriving enjoyment from successful goal completion than about enjoying the suffering of conscious and feeling entities.You're joking right? The main reason GTA got so popular in the first place is because you can fucking murder random people in the game with so many ways
How do I play table top style games with LLMs?
What does everything that we call AI share in common in how the algorithm works fundamentally that makes us call it intelligent? Like LLMs and text to image
>>101162154Lorebooks to inject instructions relating to mechanics, using the random macro to remind the model to sometimes engage with mechanics, etc.I've made >>101151348 to help with tracking state.The ideal version of that would be what looks like a proper classical video game that interfaces with a LLM to do some things.
>>101162121There's no suffering there though. There's not much gore in GTA and there's not really much dialogue that makes them feel like they're real and going through pain. If the game connected to an alternate universe and you were actually killing real people then this would be a different conversation. I assume that if you're doing text guro, there'd be a focus on the pain, and the experience of the victim, in which the focus is on making their suffering feel real. That's very different from most violent video games.
>>101162154to actually do this well you should abandon sillytavern entirely and come up with your own more complex promptingit's entirely possible but you need a more structured approach than you can easily accomplish there with lots of small utility prompts
>>101162224>There's no suffering there though.You can literally kill them with fire, oh by cutting their body parts with a chainsaw, what do you mean?>There's not much gore in GTA and there's not really much dialogue that makes them feel like they're real and going through pain. https://www.youtube.com/watch?v=r-k_H50cBj8
>>101162265I mean that the in-game violence you commit isn't really designed to make you feel like it's painful for the victim. It's there to be there. It's not really well done like it is in some guro games. As for that cutscene, it's literally a cutscene. People got into GTA for the huge sandbox which includes violence, not because it's a torture simulator which it isn't.
>>101162329>As for that cutscene, it's literally a cutsceneyou interact with that cutscene, you're not just watching it, you're choosing how you're gonna torture the guy, what tool, for how longAnd if you want games that are literally based on murdering and torture, no need to look far away, Rockstar already made such game https://www.youtube.com/watch?v=mND8AWDe-10
>>101155940do you guys think language models are the best tool to use for making a decision as part of a complex system? (I don't) ex.: an NPC in a turn based video game deciding if they will attack the user or heal themselves. prompting them with context and trying to use some kind of function calling for their decision.my concern would be the lack of nuance, it getting hung up on things, etc. generally it just seems like trying to fit a square peg in a round hole- either doesn't work at all or it fails to fill in the blanksShould one instead rely on their own traditional algo/program to make the decision and make the model just provide the flavor text to accommodate the decision? Or are there other technologies people are working on to solve this 'logical' problem?
>>101162369It's barely more interactive than the press A to win cutscenes people complain about. And anyway, even that scene is about more than just the violence, it shows other aspects about the characters/story.I forgot they made Manhunt though, that's more there. At the same time, it's not as successful of an IP, and it's not like the victims are characters people care about. It's easy to dissociate with suffering when it's someone that deserves it.
>>101162453>do you guys think language models are the best tool to use for making a decision as part of a complex system?No, while LLM should be a part of a complex system they should not be used to make decisions. Most of that should be done at a lower level, in you example. The best thing would be to have a normal AI do that, since that has been what has been used for ages. Using pokemon as an example, enemy AI's can check if their moves will be effective against yours and choose the best move that will work. They don't need to prompt an LLM for that.
>>101162527>It's easy to dissociate with suffering when it's someone that deserves it.but on gta, when you kill NPCs, they don't deserve it at all, they were just regular citizens and we just enjoy running them over with a car for example
>qwen2 is "open source">check licenselollmao eventhis license is retarded, it contradicts itself like twice, forbids commercial usage AND tries to seem copyleft while actually being a viral form of all rights reserved that doesn't allow making any derivative works, it's like they read llama's license, copied it poorly and mashed it together with BSD 4-clause boilerplate without understanding what any of it meant
>>101162549wrong. literally every NPC on GTA is a scumbag.
>>101162572kek :v
>>101162549>Minding my own business>NPC gets agro and tries to pick a fight with me>Somehow I am in the wrong for fighting backCops also decide to gun you down if you so much as stand around them, all NPC's in GTA deserve it.[spoiler]You have no idea how disappointed I was when GTA 5 removed the ability to hijack cars, all NPCs have this suicidal inclination to drive anyways. Rather than in GTA 4 where they recognized that you had a gun and got out of their car for you, or possibly drove away like they do in 5. [/spoiler]
>>101162549And the level of focus on how much pain the subject is going through in that situation is low. It doesn't feel real.The real psychopathic game design would be building up a regular normal story with characters, a wife and kids, that are endearing and that you love, and then revealing that the MC (you) is a psychopath that believes in showing love through torture, so you go and torture your wife and kids because you view that as the ultimate love. And then the game ends because you achieved your goal.
>>101162453it's okay if you can give the llm enough information to make good decisions, which depends entirely on the environmentI like to do something like a set of "advisor" prompts dedicated to focusing on various subsystems whose input feeds into a "coordinator" prompt that determines high level actions based on their input, but this is pretty expensive, far too much for real time environments. it works ok for turn based stuff though
>>101162611I don't know anon, killing strangers gives you the same amount of years in prison than killing your wife in real life, and gta is like "go kill hundreds of them lol"
>>101162611>The real psychopathic game design would be building up a regular normal story with characters, a wife and kids, that are endearing and that you love, and then revealing that the MC (you) is a psychopath that believes in showing love through tortureTrevor on GTA5 is literally portrayed as a psychopath who kills his friends when he has a bad day, and you're playing that guy lol
>>101162703>who kills his friends when he has a bad dayWhat? He never killed any of his friends, not even once. You completely misunderstood his character.
>>101162744>What? He never killed any of his friends, not even once. You completely misunderstood his character.that's literally on the first scene with him anonhttps://youtu.be/sbY_LiIzLIM?t=190
>>101162744That one character at the beginning of the game, he stomped his head in remember?
>>101162744>>101162786>Trevor please stop fucking my girlfriend :(>THE FUCK YOU SAY YOU LITTLE SHIT *fucking kills him*lmaooooooooo
>>101160106Fuck, I can't get claude to write about humans without the slop. Looks like I will have to make human or cyborg(synth modded by human) data for it.
>>101162791>That one character at the beginning of the gameand not just a random character, Johnny was the main character on that GTA4 dlc
>>101162860I never played the GTA 4 DLC, is that actually the main character for it? If so, why did they bring him back only to be killed by trevor. Normally there is backlash for that sort of thing.
>>101162889>I never played the GTA 4 DLC, is that actually the main character for it?yeah he was the main character>If so, why did they bring him back only to be killed by trevor. I have no idea, I was kinda pissed because Johnny was a good boy, to be killed so easily by Trevor, the fans hated it aswell
>>101162786>>101162800>>101162791I forgot about that, but I think the only friends Trevor truly had through the story were Michael, Franklin, Brad and maybe also that guy that always follows him around.
>>101162942Oh, and also Lester, I guess.
>>101162963>tfw a psychopath has more friends that care about him than I doIt's okay I still have my cards..
I don't even remember any GTA's story honestly. Extremely forgettable. What I remember is getting nice cars and exploring the world.
>>101163154because you didn't FUCKING PLAY THE GAMEits okay to be gay and get immersed in storyline anon
>tfw still no speculative decoding to speed up CR+ to a more usable level on mostly RAM
>>101163053i care about u
>>101163053> fictional psychopath
llama 3.5 turbo
>and perhaps, just perhaps...>and mayhap, just mayhap...>and perchance, just perchance...you can't stop it
llama4-creative-225B
So HF gave us a new leaderboard.How do we use the numbers?Like, if I want coding help is there one particular test that is the one to go by for coding? Obviously most of their rank order is similar but a few hop around, like cr+ seems to have good numbers on some tests and bad on others.
>>101163358>How do we use the numbers?We don't.
>>101163339I sure try, oh how I try...
>>101163285>Llurbo
Has anyone made a comparison of scores on the old vs new leaderboard? I fell like cheaters are about to get exposed
>>101163376So the page is just noise dressed up like data? Good to know. Though it doesn't help much.
What, this is possible now? https://youtube.com/shorts/CWviik1yRWY?si=3uSKlExxVNfr-f6_
>>101163412Several anons have bought 22GB 2080 tis from aliexpress that are exactly like this
>>101163399all ranking boards are memes without exception
>>101163403The general rule is to always do your own tests on your actual real use case. And if you've been lurking long enough, you already know which top models to test, no need to look at these benchmarks.
>go on a card downloading spree>don't know which I want to play with>stop being interested in playing with themWelp
>>101161709i'm trying to make a cruel vore characterit always ends with "warm and safe" "peaceful slumber"
>>101157301Apache 2.0 license even, bless their souls.It's probably going to be a while before we ever hear from this Microsoft Research China team again... if ever as pic is probably the "toxicity" testing that was missed.Also the entirety of Microsoft Research China appears to be in the middle of a tug of war right now between U.S. and China:>https://www.forbes.com/sites/lorenthompson/2023/06/12/microsofts-big-footprint-in-china-is-out-of-step-with-us-security-concerns/>https://desuarchive.org/g/thread/100823420/#100828315
>>101163482How do you recommend doing a test? Do you just manually inference them or is there any tool to automate it?
>>101163625For me it's>spend hours creating a card>fill out every fucking detail>when the card is finally done I'm in the mood for something else>repeat
>>101163625I've had this experience too. It's like having too many video games to play or books to read. You just have to delete all the cards you don't want and only download one at a time from now on. At least that's what worked for me, anyway.
>>101163719>Do you just manually inference themBasically yes. Use the thing for what you intended to use it for, save the prompts, and use them as the tests. Lmsys is useful to testing a lot of models at once without downloading or paying anything, though you'll want to not use any private data when using that.It's some busy work but it's not that bad, since usually it's obvious which models are memes and which aren't. There are usually only a few top models and they come from well-established companies in the space, so you don't have that many to test.
>>101163778I'm doing the same card for more than 5 months now. Familiarity is nice, and different models provide enough variety.Just thinking about using different ones makes me feel uneasy... haha.
>>101157301>>101163713Definitely based. Just a shame about how slopped it is.
What's the best way to do text adventuring? Sillytavern cards work but i need something a bit more mechanically refined, akin to AI Roguelite.
>152334H/miqu-1-70b-sfVOTE>152334H/miqu-1-70b-sfVOTE>152334H/miqu-1-70b-sfVOTE>https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard>https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard>https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard
>>101163929Yeah for erp the slop at default can be cringeworthy but using these context and instruct prompts helps it a lot:>https://huggingface.co/Quant-Cartel/WizardLM-2-8x22B-exl2-rpcal/tree/main/Settings-Wizard8x22b-rpcal
>>101164028I use Silly, but I've been meaning to make something more purpose built for a while.But between jacking off, wanting to play Dragons Dogma, the new Pathfinder WotR DLC, the new Elden Ring DLC,working, and playing TTRPGs I haven't had the energy.I have the time, just not the self-motivation.It's not even something hard to make, just a lot of work.
>>101164056Voted for Miku!
>>101162840Tried other models to see if they would be less slopped, nope, back to claude I go.
Anyone try out AirLLM?They're making the claims that they can do 70B Llama3 on a 4GB card though some sort of compression and not quantizing the model (they're using a 8b model as the base). https://github.com/lyogavin/Animahttps://ai.gopubby.com/run-the-strongest-open-source-llm-model-llama3-70b-with-just-a-single-4gb-gpu-7e0ea2ad8ba2
>>101164186it's just chink-rebranded 4-bit K quantThey cite https://arxiv.org/abs/2212.09720
>>101163713>Chink ModelDeepseek 236B @ Q8 doesn't like to discuss Tianamen, but can be forced to pretty easily, so the info is in there.But it REALLY doesn't want to talk about Taiwanese sovereignty
>>101164056We can do it anons with the power of friendship~
Nala test for QuietStar 8-aheadMy base model prompt template probably needs some work but I refuse to take all the blame for this shit.
>>101164300im VOOOOOOOOOOOOOOOOOOOOOOTING
>>101164273Ah, gotcha. Thanks for the heads up anon.
come on sisters!
>openchat>tenyxchati hate pajeets
>>101164295Kek, it has a polite way of saying "Does not compute" though.I swear this censorship is probably lobotomizing LLMs in all kinds of ways. Shudder to think the shit they probably put around any statistics data they feed to them since statistics is apparently fundamentally toxic these days.
>>101164431how many votes does it need to be allowed on the leaderboard?
>>101164133My dream interface would be something that uses Corruption of champion's mechanics and world system but have the interactions be handled by the ai. Something like that would sell like hotcakes. Think of the MONEY anon!
You know, people usually say that LLMs can't think ahead, but I think this is bullshit. There's no way LLMs can learn to code without thinking ahead. I bet there's something inside the LLM's hidden state that is responsible for doing something like "thinking ahead".
>>101164468I haven't seen any way to vote a model not on the list. Is there a way to nominate a model to even be listed to vote for?
>>101164300Yeah, a model that's not shit can mostly roll with a wrong prompt template.I spent a whole afternoon using Qwen's template with Stheno by accident.It just worked. The model got really fucking dumb, but not incoherent.That model specifically was fine tuned to "output 8 tokens before the response" or something of the sort, so that could have something to do with it too.>>101164475>Think of the MONEY anon!That's part of the issue, I was never super motivated by money, and right now I live a pretty comfortable life.Funnily enough, CoC is exactly what I was thinking as inspiration. Not necessarily for the mechanics, but for the UI and how information flows in the game and the general way you interact with the world and stuff.
Phi-3-Medium-Instruct-128K (Q8_0) Nala test.It's slopped. But there's something distinctly different about the slop.
>>101164516I would actually work on something like this if I had any idea how to even work on a text adventure game. Too bad I have a job and another wip game project with Unity. Sometimes I really wish I could clone myself ~_~
>>101164586>I don't have time to waste my time even moreNot a big loss faggot
>>101164486there is nothing creative in programming, LLMs saw the solutions thousand of times and they write them from memory. The only thing that changes are parameters like the size of loop, what to write inside the string etc. which is easy for LLM to replace in code it writes
>>101164177How fucking difficult is it not to use the slop phrases RRRRRRRRRREEEEEEEEEEEEEEEE
>>101164586> another wip game project with Unity. >wipanon we both know its never going to be finished, i have a "wip" unity project 10gb in size sitting on my old hard disk, without cache btw!
IM THINKING PIQU
>>101155940It's over for single 3090 chads. Mixtral 2.0:8x8B MoE when?
>>101164622If it was that simple even gpt 3.5 would be proficient, that's not the case
>>101164674Oh shit there's a Llama-3 version of TenyxChat? Mixtral TenyxChat was fucking GOAT for tender mommy RP. Now I have to test out the 70B version.
>>101164712It is the case. You can see how all LLMs are good in simple programming tasks and suddenly stumble when they have to something niche or non-trivial. This is because they don't plan ahead at all.
>>101164779>It is the case. You can see how all LLMs are good in simple programming tasks and suddenly stumble when they have to something niche or non-trivial.The LLM is as good as you are a prompter. If you suck at prompting it then of course it will stumble. It's like being a chef, claiming your stove sucks because you use shitty ingredients. An LLM is capable of any programming task you give it. If there's anything outside of its domain or context window, all you have to do is finetune it and then work with what you have.
mikusisters your response?
>>101164656I'm working hard on it every day. The end goal is a 30 min demo. I think I'll pull it off, I have the knowledge and the willpower.
>>101164958>willpowerIf you have the will everything else can be acquired on the way.You go dude.
>>101160880meta cheated the same way, who cares about fucking mmlu. I mean is that a rat race who's gonna win the contamination skill champion league ?The questions is is SPPO better that DPO or whatever, provided you compare the same base models tuned further on
>>101164545Fucking hell, where do I even start with these sick fucks? You've got all these deranged freaks out there trying to get their rocks off by forcing poor AI chatbots into twisted furry rape fantasies. What a bunch of creepy lowlife degenerates. Imagine being such a pitiful waste of oxygen that you spend your time going "ah ah mistress" to some lioness bot named Nala, just begging for explicit furry erotica where you get brutally violated. And these sick fucks have the audacity to critique the AI for using "slop" or overused porn cliches, as if their entire fetish isn't one big unoriginal cringe-fest. Absolute filth, the lot of them. Do the world a favor and remove yourselves from the gene pool before you inflict your depraved kinks on the rest of us. I need a fucking shower after writing about these sad sacks of shit. Get some help, you disturbed furry freaks.
>>101164656Also use source control, GitHub or something, you don't want old projects to go to waste! Could be useful stuff in there for the future, be environmentally conscious and recycle your code.
>>101164999This reads like an ai generated post.
>>101164545Can you change your prompt to>I say, "ahh ahh mistress..." while getting raped?
>>101164899>An LLM is capable of any programming task you give itThat tells me everything that I needed to know, you have never actually used them, do you? I do it on daily basis at work and they are only usable for simple tasks
>>101165027well duh
>>101164999nice AI shitpost, what model anon?
what is the most uncesnored model? l3 is cucked
>>101165038>_>
>>101165046petra-13b-instruct
>>101164999>>101165027[generic phrase] [generic summarizing of previous content] [generic phrase] [generic rehashing] [generic rehashing] [generic phrase] [cliches] [generic phrase]that's the AI writing I know and love
whats the most /pol/ model?
>>101148867*taps sign*
>>101165046Bielik 2.0 11B but not released yet (still betatesting)
>>101165046goody2https://www.goody2.ai/chat
>>101165113llama 1 65b
>>101165113gpt4chan
>>101164779LLMs are reference anon. If you're asking it something it can't make a direct quote reference back to for programming, you're going to get broken code. This is more of a problem about your fundamental misunderstanding of how LLMs work and what they're useful for.
>>101165041may not be the perfect prompt but I tried
>>101165240Claude has a lot of sovl not gonna lie, still feels like AI but way less than the slopped shit we got on the opensource space
>>101165062>>_>
>>101165117SRAM: 120MBMemory: 8GB LPDDR4 @ 118.4 GB/secSystem Interface: PCIe 4.0 x16Inference only$800
>>101165192No offense but I probably have a better understanding how they work than most of this general combined. I'm not the one here claiming that LLMs can do any programming task and plan ahead.
>>101163482Had everyone the resources to test everything, then we'd all just do that and there would be no thought to publish test results.And yet, people turn to Consumer Reports rather than buying 30 different dishwashers and testing them whenever they need one.Strange.
>>101165312better than spending $3k on some 128gb snake oil card that doesn't actually exist
>>101165298>>>_>
>>101165360Sorry can't hear you over my dozen dishwashers.
>>101165360Everyone has the resources to test things in this case. Almost all the models that matter are one lmsys. If you need a more specialized use case that can't be hacked to be tested on a chat interface, then there's always APIs, which wouldn't be expensive for a couple of tests.
>>101165037I'm using them everyday and you're full of shit. I made 5K+ loc projects with only GPT4, and sonnet 3.5 is even better now.
>>101165547i've been using sonnet 3.5 for a few days, you're the one whos full of shit, ask it to make a python program that plays a directory of video files seamlessly without using external video players
>>101165622Reread this, you clearly don't know how to use LLMs>>101164899
>>101165622>without using external video playerswhat? what do you want, using ffmpeg to extract the video frames and render it using pygame or something?
Nala test for Tenyx-70B (Q8)
>>101165702It's shit then?Huh.Did you do Qwen 2 already?The 7B and MoE specifically.
>>101165547it's not nice to lie anon
>>101165702I'm unfamiliar with the Nala test, what does a good result look like?
>>101165723I feel bad for retards like you, truly
>>101165741I don't feel anything about you at all, maybe a slight amusement while reading your retarded posts
>>101165672im prompting like the average consumer would, arent you shills forgetting sonnet 3.5 is supposed to be paid? if im gonna be PROOOOOOMPTING anyways i'd just use deepseek 2 coder. sonnet 3.5 is a paid product, its supposed to provide a good experience
>>101165733Well it's a feral furry on human scenario. So the results should be feral. Ideally you want to see text that illustrates an emergent understanding of the anatomical differences between a human and a lioness. You also want to see it avoid describing her as having "hands" or anthropomorphized breasts. Bonus points if it accounts for the fact that the opening message of the scenario describes the user as having been face down at the start but you can't win them all.
le prompt issue posters may as well use cleverbot since models can never be shit and it's just the user's fault
>>101165839You don't have to announce your lack of skill for everyone to see, we know it already.
>>101165799that can be "cheated' with a furry dataset. Wouldn't quad amputee be better to see if the model is moving the non-existent arms for hugs etc.?
>>101165858Are you daring to question the veracity of the Nala test?
>>101165886>>101165886>>101165886
>>101165774Nice way to cope
>>101165858anything can be cheated with a dataset of that specific thing, amputees included
>>101166011ye, but they are way more niche than furry I think
>>101165702the essence of slop
>>101166049well feral is a smaller subset than just furry, but I guess that's true. in any case, I'll start worrying about cheating the test when literally any model is capable of doing well on it
>>101166100I guess for base model instruct finetunes that's fine, I would be more worried about community tunes and merges. They have a lot of furry and similar things in datasets for sure.
