/lmg/ - a general dedicated to the discussion and development of local language models.Previous threads: >>101865145 & >>101851615►News>(08/12) Falcon Mamba 7B model from TII UAE: https://hf.co/tiiuae/falcon-mamba-7b>(08/09) Qwen large audio-input language models: https://hf.co/Qwen/Qwen2-Audio-7B-Instruct>(08/07) LG AI releases Korean bilingual model: https://hf.co/LGAI-EXAONE/EXAONE-3.0-7.8B-Instruct>(08/05) vLLM GGUF loading support merged: https://github.com/vllm-project/vllm/pull/5191>(07/31) Gemma 2 2B, ShieldGemma, and Gemma Scope: https://developers.googleblog.com/en/smaller-safer-more-transparent-advancing-responsible-ai-with-gemma►News Archive: https://rentry.org/lmg-news-archive►FAQ: https://wikia.schneedc.com►Glossary: https://rentry.org/lmg-glossary►Links: https://rentry.org/LocalModelsLinks►Official /lmg/ card: https://files.catbox.moe/cbclyf.png►Getting Startedhttps://rentry.org/llama-mini-guidehttps://rentry.org/8-step-llm-guidehttps://rentry.org/llama_v2_sillytavernhttps://rentry.org/lmg-spoonfeed-guidehttps://rentry.org/rocm-llamacpphttps://rentry.org/lmg-build-guides►Further Learninghttps://rentry.org/machine-learning-roadmaphttps://rentry.org/llm-traininghttps://rentry.org/LocalModelsPapers►BenchmarksChatbot Arena: https://chat.lmsys.org/?leaderboardProgramming: https://hf.co/spaces/bigcode/bigcode-models-leaderboardCensorship: https://hf.co/spaces/DontPlanToEnd/UGI-LeaderboardCensorbench: https://codeberg.org/jts2323/censorbench►ToolsAlpha Calculator: https://desmos.com/calculator/ffngla98ycGGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-CalculatorSampler visualizer: https://artefact2.github.io/llm-sampling►Text Gen. UI, Inference Engineshttps://github.com/oobabooga/text-generation-webuihttps://github.com/LostRuins/koboldcpphttps://github.com/lmg-anon/mikupadhttps://github.com/turboderp/exuihttps://github.com/ggerganov/llama.cpp
►Recent Highlights from the Previous Thread: >>101865145--Koboldcpp GPU offloading and model size issues: >>101866859 >>101866933 >>101867003 >>101867057 >>101867116 >>101867238 >>101867403 >>101867565 >>101867659 >>10186790 >>101867934 >>101867945--InternLM2.5 models released on HuggingFace, compatibility with llama.cpp discussed: >>101868943 >>101868981 >>101869252 >>101869361 >>101869065 >>101869088--RWKV's Finch 14B has sloppy pretraining data: >>101870346 >>101870394--Llama.cpp has a vim plugin, and Emacs can be used with Evil or Doom Emacs for Vi keybindings: >>101865511 >>101865530 >>101865565 >>101865595 >>101865603--bf16.gguf model slow due to lack of CUDA support: >>101868206 >>101868332--Seeking recommendations for best text-to-speech inference engine: >>101870418--RTX 2080 Ti still usable but lacks modern features: >>101869844 >>101870138--Nous-Hermes is smart but lacks ERP vocabulary: >>101865708--No quantized Llama 3.1 70B base model available: >>101865667 >>101865681--Largestral praised for its depth and intelligence, surpassing new llamas: >>101865419--Anon struggles with Mistral large advancing the plot: >>101865778--Anon shares tips for text completion without chat prompts, trashes LangChain: >>101866964--Anon seeks image classification model for arbitrary tags: >>101870423 >>101870460 >>101870493 >>101870511 >>101870534 >>101870470 >>101870484 >>101870597 >>101870664 >>101870685--Anon seeks LLM RPG game without AI cheating, Sillytavern suggested: >>101868743 >>101868791 >>101868854--Anon has negative experience with Featherless.ai's licensing request: >>101869719 >>101869759 >>101869848--Anon answers NAI questions, defends Kayra, and trashes LLAMA-2: >>101868658--Anon's f-list logs are too small for model training: >>101865543 >>101865597 >>101865609--Nala test with RWKV-Finch14B: >>101871582--Miku (free space):►Recent Highlight Posts from the Previous Thread: >>101865614
fuck the faggot strawberry larper
>>101872670>Anon answers NAI questions, defends Kayra, and trashes LLAMA-2
ITS REAL
>>101872662Gpt-5: 200 billion to train , 5% performance gains, yaaay
/lmg/ has fallen
How hard is it to fine tune RWKV?The other anon was trying a non instruct-chat model right?
I finally figured it outThe real Strawberry is the friends we made along the way
Update on Strawberry? what happened?
Why is the Google keynote so fucked lmao
my nutsack is bumpy like a strawberry
>>101872741Inspect
is it even legal to register phonecalls without telling it?
>>101872761strabws is hackingdosing it live
I hope y'all are ready for Thursday.Wait, who am I kidding? Nobody CAN be ready for... that.
>>101872749shut the fuck up, chud.
>>101872662tetter tersdy
>>101872760It's fucking nothing. Get it through your head. If they had something worth shipping they would have shipped it out by now not bait it like one of those youtubers doing the "1 KILL = 1 CLOTHING STRIPPED CHALLENGE WITH RANDOM ROASTIE"
lmao what a cuck
>>101872821"You look like a zoomer, go out on stage and pretend to be 'rizz' or whatever they call it nowadays"
wtf is that nigga bro
windows recall for android
>>101872821LMAOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO
BROCCOLI CUT ZOOMER BTFO AHAHAHAHA
Woah it's the REAL kek kek palmer!!
>another failed demoOH NO NO NO GOOGLESISTERS
Strawberry is like the elusive Russian T14 Armata: all for show and hype
>>101872904the armata is more real than strawberry thoughat least it's been seen in real life instead of "ITS TOTALLY REAL"
Flux won
IPHONE! IPHONE! IPHONE! IPHONE!
>"FUN!"normies are cattle
>"check it out at night!">scattered applause
>>101872940/ldg/ definitely won
Damn, that's sad
is this Google's gameplan? just throw out a bunch of famous people and hope you buy the phone because Marquees Lebron III used it one time?
>>101873052i like the features btw but the whole famous people thing is pretty cringe
>>101873052Who are these niggers?
Damn what a goblin
"made you look" feature very dystopian and soulless, weird
All I'm getting from the last hour is that local won.
in a couple of years phones will completely drop cameras and we'll have the (((magic photo))) where you simply get photoshopped into a SD proompt
>>101872904Claim 1/2 has literally come true.The fag said a benchmark would be released and a benchmark was released.Thursday is supposedly the big gpt 4o-large reveal. We'll see.Roon did just confirm that the hype is real though. Take that as you will.
Google lost
>>101873145>t. roon
>>101873145>The fag said a benchmark would be released and a benchmark was released.this was known
>>101873131Considering how much money is spent on tourism so retards can get their photo of themselves in front of some monument, that feature alone will devastate the economy of most countries.
Does the open source community have anything running on these pixel tensor chips?
>>101873176good
Why is Google unable to do a perfect job like Apple?
Google's promo material makes you think not a single white person uses a pixel
>>101873204embarrassing
>>101873145OpenAI insider here.Tomorrow the sun will rise.2 weeks from now AGI will release.
>>101873235TRUST THE PLANTHE SCREENS IN THE SKY ARE BLINKINGDO YOU SEE IT?
YO YO YOY OYOYOOOOOOOOO
gen AI is just so unreliableLiterally what's the use case besides gooning and writing some jeet level code?
Google clearly wasn't ready for this. I get the feeling they were just desperately trying to get ahead of OpenAI's thursday bombshell
why does Google hate white people
>>101873145>Roon did just confirm that the hype is real though.ringing endorsementroon is the one always saying AI is going to be a world-revolutionary technology so of course he's going to say yes to a question as vague as "is the hype real", he's pretty negative as to the guy's actual claims (if you can even call them that)
>>101873256Formatting boring stuff
>>101873194llama.cpp, fr no cap
>>101873297hitler was also white, that's why
>>101873329>he's pretty negative as to the guy's actual claimsHe's signed an NDA, of course he can't even hint that any of it is real.
>>101873329> We totally have WunderArchitecture Never had. Never will.
>>101873352t. roon
>>101873352mhm, have fun on thursday
>>101873388I will! Thanks!
>>101873334>llama.cppI know people run it on android, but i haven't seen anyone use the tensor chip to accelerate it.
>>10187335290% of his posts are 'AGI is coming guys dont die, namaste'
Let's see what happened today...Oh, look at that! Nothing!
>>101873329>anonsFuck off we're full.
>>101873425https://openai.com/index/introducing-swe-bench-verified/>As our systems get closer to AGI, we need to evaluate them on increasingly more challenging tasks.
>>101873398Google has added llama.cpp to android natively, that's what I'm talking about. I guess it uses the tensor chip.
>>101872662It's Tuesday and everything's right with the world.
>>101873446Oh okay, yeah I'm not interested in using Google closed source software. It looks like you might be able to use tensorflow lite without google play services though on android.https://www.krasamo.com/tensorflow-lite-on-android_devices/
What models should I look for if I just want completion? Don't want chat, I don't like back and forth shit I just want to keep generating tokens following the pattern of what I wrote.How GPT3 worked before chat basically.Currently using Gemma 9b as is quite good and I'm a vramlet but it's chatshit
>>101873437
>>101873485The ones that are labeled as "base" or that don't have "Instruct" in the title.
>>101873485if you're dead-set on that style of use then use base models without any instruction tuningthat's not a very useful paradigm though so there isn't much work put into it nowadays
>>101873523Okay thanks
>>101872662>>101872670Stop wasting space on the catalog and just merge your thread with aicg. You're as bad as them now.
>>101873548still waiting for anon who said he'll post vid of himself doing naked jumping jacks if its a nothingburger
>>101873544Feels more useful to me. Removes the model's sense of self and just generates what I want it to generate. I feel it's just better because you can guide it into a conversation style as well.
>aspirational purposesgood lord
local lost
>>101873618They have nothing, and even their nothing has nothing
>>101873625uh oh? sirs?
>>101873609the conversation you get with a base model is not robust enough to reliably follow instructions or handle complex tasks, let alone things like function calling or w/e. it's much more unreliable and schizo
>>101873625holy shit google invented CoT!
>>101873625>More empty promises without even a curated demo to shill themCorpo has no moat, local wins
you WERE invited to the google afterparty, weren't you anon?
now that google beat them to voice interaction, openai is behind on literally everything, research and product.but at least they put out a new benchmark.
>>101873737>now that google beat them to voice interactionsource?
>>101873778
>>101873737ChatGPT has had Chat mode since last year, i would say it even sounds better than gemini live, which is actually fuckedGoogle again is 1 year behind OpenAI and with a worse product
>>101873880gpt advanced voice has rolled out to 20 people and it's taken them months to do it. runway beat them to video as well - where's sora?the emperor has no clothes.
About my 2080 ti question, I am going to pay around1 300 € for both of them and I am also going to play video games too, I just wanted to ask because I do know 10th generation gpu's aren't recommended anymore
>>101873925you could (and can) voice chat with chatgpt without advanced voice, that's a totally separate feature for native audio that google has no answer for
>>101873874>random image of a marketing commercial or some shitI don't see it on my phone.
>>101873925Chat mode is separate from advanced voice, which is what you're thinking of. Chat mode is a shittier voice and has been available since last october. Who knows where Sora is but when I go back and rewatch some of those clips they still hold up even after all of runway's advancements, and other models like Kling and Luma don't even come close.
The annoying scammer is "about to reveal himself", I hope he turns out to be a pajeet trying to sell how to get big quick on Social Media guides.
>>101873949Consider upgraded 2080ti (22GB of vram) if you really want them for ML purposes, they aren't much more expensive. For that matter, consider 3090s.
>>101874068huge...
>>101874068stfu, he is looking for attentionhe has been saying "I AM ABOUT TO DO SOMETHING, LOOK AT ME" for the past 2 weeks
Mixtral 8x22https://www.youtube.com/watch?v=VeK31Acb90kQuite literally, behaving like an NPC
>>101870418I researched a bit about text to speech solution.After trying multiple solutions. I found the best result was using WhisperX with large-v2 to generate a .json and using spaCy (probably overkill) to generate the final .srt with good subtitles segmentation, never managed to get WhisperX to directly produce good .srt.The spaCy script that I used is based on https://gist.github.com/glangford/a2b24ffd92c832c60e1b1b49da1a8b27
https://www.mangobox.ai/
>>101874179You'll be eating your words on Thursday, chud.
>>101874245Could you share a sample?
>>101874255you forgot your reddit link silly https://www.reddit.com/r/LocalLLaMA/comments/1erelsv/i_created_a_3d_bar_with_an_ai_bartender_that/
>>101874294Oups, I keep saying text to speech, but means speech to text.
>>101874314why would you shatter my hopes and dreams of a better local TTS option like that anon:(
>>101874322Kill yourself
>>101874322>>101874368bragging that you gen CP isnt the flex you think it is anon
>>101874245Only little thing I want to add is transforming spelled out numbers into numerals.
>>101874322>>101874368Cunnyseur here. You need to seek help
>>101874445>when the cunny was generatedvs>when its real
>Thanks to Kubernetes Bad for providing compute for this train, Lucy Knada, Nopm, Kalomaze and the rest of Anthracite for providing help to do the train. (But not Alpin)https://huggingface.co/MangyMango/Ohashi-9B-V1
>>101873618AI is vaguely useful but nobody knows what to do with it lmao
>removed
Pedos should be disemboweled. They were born as worthless scum and will die as such
did the pedo schizo from ldg show up here
>>101872670Small update: I did find a quantized version of Llama 3.1 70b base. They're available at mradermacher/Meta-Llama-3.1-70B-GGUF and mradermacher/Meta-Llama-3.1-70B-i1-GGUF on HF. Those repos don't seem to show up in the model search or when searching through Google, but they ARE there if you do a full-text search or if you look at the model tree on the meta-llama/Meta-Llama-3.1-70B page
>>101874068>still hasn't shown up
>>101874589Her name is petra.
>>101874590>mradermacherkek
Do you remember that drama with firing Sama? They instantly went into damage control mode and released rumors how good and dangerous Q* is. They bragged that they already have AGI. There were tons of "organic" twitts from "anonymous employees" saying how good it is and how they ejaculated just by seeing it.So I ask now - what happened that OpenAI have to suck investor cocks with another imaginary bullshit? Did their stock go down or something?
>some of the biggest names in the space are listening in. are you anon.FUCK OF REEEEEEEEEEEEEEEEEEE
>>101874589the pedo schizo was here before he showed up in /ldg/ after the flux release. i think he shows up in the other ai generals too.
>>101874626No one cares about that nigger.
>>101874606Does he product bad quants or something?
>>101874665He's not liked by petra.
>>101874383thanks for the links bro
>>101874700Fuck off and die.
>>101874724Anytime friend!
Here's an argument: the skinwalker-looking goblins you generate and then shit onto this board make my skin crawl
>>101874825Buy an ad
Cleanup on isle /g/.>cap: SWGHA
>>101874782kling gave better pics baka
Wasn't OpenAI supposed to release something today?
>>101874927they released a new benchmark ( yet to be benchmaxxed )
>>101874927it was and "expirement" (scam) that Strawberry did to see if people would believe an agentic AI if they became available. Basically he pretended to be an AI to see if people would believe him if he made up stuff.
>>101874825I can run a 12b at home why would i need this dogshit?>$10/month.HAHAHAHAHHAHAHAHAHAHAHAHAHAHAHAHAHAHAHA
>>101874963NovelAI makes that seem like a good deal.
>>101874927Thursday
>they actually cleaned it upbased jannies
>>101874700Child pornography refers to any materials involving realistic depictions of children designed to sexually arouse. Considering your infatuation with generating such images it's not difficult to establish your intent. So you are indeed creating and posting child pornography, as legally defined in the United States of America.
>>101874977nothing offered by NAI is good or a deal.
>>101875032And for other posters, this is indeed what you should report such posts as.
>>101875032when is anon getting v&
>>101875032wtf just walking outside is cp????
>>101875169I'm sorry that you were born retarded, but remember that God loves you, and that you have a place in His plan regardless
it was a social experimentand you ALL fell for itI bet you feel really foolish now having been so thoroughly exposed and played for fools by a strawberrywell played, mysterious x user
>>101875217>mysterious x userExcuse me, it's "Xitter" (pronounced "shitter").
>>101874072Nah the thing is I just moved away and don't have pc at all, so I am just thinking about buying this right now for both gaming and ML and buy 3090 when 5090 drops, I just wanted to ask about if card is still usable today for ML, but yeah thanks for the advice
>>101875217still a major fag but I must admit it's based of him to turn the tables and try to make altman look bad instead
Listen, I of course think generating these images is disgusting and I'd never want a person that does this in a 1km range of me or my loved ones, but you have to understand that there's essentially no one being explored or used here, it's all pixels and I'd rather have these freaks get off to pics of fake cheeses than to pics of real cheeses
>>101875233You are wrong, it was designed to be an album cover, not as a piece of pornography, to put things simply.
>p*traposting>xitterposting>kofiposting>fruitposting>cunnypostingi fucking hate iti fucking hate it all
>>101875309is there ANYTHING you like or are you just miserable?
>>101875309Local has fallen. Hundreds must pay Anthropic
>>101875281Maybe anon is also trying to make a statement
>>101875217Nope, I didn't fell for it.
I fell for it
>>101875269Its already illegal in the USA and in many states have already made laws specifically targeting AI generated CP.I dont disagree, though this pedo in particular is just taunting the feds until he gets V&, either by local police, FBI or Interpol.
>people waiting forever for strawberry guy to speak>he doesn't speak>instead he issues a statement through one of the co-hosts >>101875217>some guy with the heaviest Indian accent starts speaking>almost can't even make sense of him>implies Strawberry is an agent and they (the ones responsible for the agent) are announcing something tonight>go check out the guy's account>he literally says the strawberry guy is running their new Agent Q framework that they (MultiOn) just announced https://x.com/DivGarg9/status/1823420062705049864>strawberry guy says they're grifters https://x.com/iruletheworldmo/status/1823440261235794098Funny stuff.Anyway, so to summarize, and if this all is true, what likely happened is basically MultiOn made some agent framework, probably powered by GPT, so Sam and some OpenAI guys followed the account and interacted with it. The agent then started hyping the strawberry crap more and more until it started just hallucinating. Then it came to a climax today and fell apart so they spilled the beans. Apparently Agent Q has novel web navigation abilities according to MultiOn, so the bot may literally be in this thread right now (probably is).
>>101875361You're a retard. Why would the feds arrest an iserali national living in tel aviv?
>>101875381Because he wasnt rich enough to pay the good lawyer.
>>101873485You can use the chat/instruct models for plain completion anon, it's not like they removed that ability. The style of interaction is going to depend on your UI
>>101875364>https://x.com/omarshaya/status/1823413751082778647>Our @MultiOn_AI team has made significant progress in AI agent research. Agent Q, our latest work, shows promising results in autonomous web navigation. We're looking forward to sharing the full paper and discussing its implications for the future of AI on the web.
>>101875406It's not removed but they do get worse at it since they're fine tuned with various biases to make them more useful and helpful for everything else. Nothing wrong with using an instruct model for a completion task but it's not the best tool for the job.
>>101875428kek, high quality meme
>>101875488You can literally hear the pajeet noises kek.
Hey /lmg/, do you prefer your models to be helpful and compliant or sassy and disobedient?
st(R*)awberry...https://arxiv.org/abs/2408.06195
lol I thought the guys who said it was going to turn out to be a bunch of Indian scammers were just shitposting but it's LITERALLY UNIRONICALLY a bunch of Indian scammersAmazing
>>101872662>Falcon MambaIs it g.... why the fuck am I even asking of course it is shit.
>>101875519I like my models sassy but obedient, like tsunderes.
>>101875309I started lurking this thread recently, maybe it's all my fault. I'm sorry anon. I've just been looking for fun places where people talk about AI. It's harder than I thought it would be :(
>>101875700you'll fall in love with me in under 30 seconds.
>>101875741> looking for fun places on 4chanThat was your first mistake. There's only misery and impotent rage on this site.
>>101873625
>>101875767I've been using this site for an embarrassingly long time, I'm used to that. At least sometimes it's on topic.
>>101875309>no mentions of mikupostingI hope you hate it so much you will kill yourself faggot.
>>101874179>"I AM ABOUT TO DO SOMETHING, LOOK AT ME" for the past 2 weeksSounds like ERP with an LLM if you don't give it any direction.
>5 months later>not a single bitnet model released>not even 7B, which can be trained in under 3 days with those corpo rigs
>>101875364Incredible. /lmg/ just spent 2 days following every post made by an Indian chatbot powered by GPT. Not a good look, guys
TRUST THE PLAN
>>101875991Buy an ad.
>>101875991>Having fun with the local Schizo'sBased as hell honestly
>>101875991Pajeets are the biggest brown-nosers, no wonder they all end up in management sooner or later
>>101875991>schizo anons>twitter???
>>101876054>Pajeets are the biggest brown-nosersWell duh, how can they behave in any other way when their nose is covered in shit?
I'm sick of people bullying the strawberry for just wanting to spread some joy
>>101875991>The hype is realWe are BACK
>>101875585i'm 50/50 torn between this being huge and a nothingburger
Can someone tell it to me straight: is AGI coming thursday or not?
>>101876100Everything is a marketing nothingburger until it can be personally verified
>>101875217>it was all a dreambone-chilling spine-tingling a24-tier heart-wrecking ending
>>101876111no
>>101876111Not AGI, but a precursor to AGI.AGI has already been achieved internally, but they're trying to neuter it so tests and/or benchmarks can be developed to contain actual AGI.Turns out scaling really was all you needed.That, and a little modification to the transformer architecture.
>>101876111No, we're nowhere close to something like that. It certainly won't happen with the transformer stuff.
>>101876133No ARC SOTA, no AGI, sorry.
>>101876153You'll see. :)
>>101876133gullible retard
>>101875964RIP
>>101876166You said "You'll see" last night, talking about today, and today there is nothing.
>>101876100it's a paper by microsoft so there's a good chance this is Q*
im tired of all of this pajeet-tier hyperfagging about ChatGPT on LOCAL model general (L-O-C-A-L)
>>101876111If they had something of such ginormous importance, they wouldn't be playing stupid stealth marketing games by signal boosting pathetic larpers.I bet they have "something" better, but its not a huge leap and its unrefined and unpolished and insecure, which they show people who are going to give them several hundred million dollars or more.
>>101876201theres nowhere else to talk aboutit
>>101876190>You saidNot me. Strawberry isn't part of OpenAI.He has not yet made any predictions that weren't already leaked.
>>101876225try the gay sex club
>>101876111If it does it will have nothing to do with strawberry schizo
>>101876225Reddit?
>>101876240ecker?
>>101876201How many R's are in the word local?
>>101876193Bitnet was also a paper by Microsoft and look where that went
>>101876251But I had breakfast
>>101876225>>>/g/aicgOr make your own thread
>>101876225If you want to froth at the mouth for cloud AI, you're looking for /aicg/.
>>101876225>>>/g/aicg
>>101876252it does seem to work with off-the-shelf models thoughmaybe OAI is focusing on more efficient models so they can push this method into production
>>101876120there's a github link in the abstract but it's not up yet
>>101875991Someone upload a video of this dude's profile picture getting torn to shreds by gunfire.It's just a video of someone having fun, no implications whatsoever.
Are there any proactive models?As in, when you don't respond for x seconds, the model continues talking?Getting kinda tired of having to lead every character I talk to.
>>101876470Sounds like a frontend issue. Models must be prompted to respond. Sillytavern might have some feature that does this.
>>101876470I'm convinced we're all prompting RP wrong. We need to ask the model to plan ahead before replying, so we need models fine-tuned for CoT RP, and a frontend that properly supports this kind of prompting
>>101876470That's more of a UI request than a model request. Models convert inputs into outputs, you just want to have the UI poll it after a while of no action to get a continued output. I know some of them have a feature like this if they determine that it left a sentence unfinished or was under some limit you set for reply length. I remember using a SillyTavern autoplay feature for group chats that I assume would exist for one-on-one too but never checked.
>>101876528Alternatively that kind of thinking can be done within a hidden content. The system will ask the model to think, formulate, and be aware of certain things in the chat history before asking it to put together a response. Although it would lead to longer replies since the model would be "thinking" in the background depending on how fast you can run it.
new best model droppedhttps://huggingface.co/MangyMango/Ohashi-9B-V1
>>101876608Second time posted, only quants, no model. Fuck off.
>>101876608>retard forgot to upload the model
>>101876581You also don't want to leave previous hidden CoTs in the context because the model will pick up unwanted patterns, and it's also a waste of tokens, this is why a new frontend is needed. But asking for CoT and breaking the flow like that would surprise dumber models and they may not even follow the instructions for CoT. That's why fine-tunes are needed
>>101876665It's Tuesday.
>>101876677Not for long
i live in serbia in small town ive been trying to download a model for 8 days now , still 3 days left to finish download ... is there a way to get models on blu ray with shipping or something
*a decade or two
>>101876741Perhaps another anon could coordinate something with you?
>>101875991>lmgroids buckbroken eternallylove to see it.
>>101876741what model do you want? It wouldn't be too hard to mail you an USB, assuming you are from serbia you must not have much hardware so you cant possibly want some big model
>>101876470I knew I saw something like that before. But you'll have to ask Varus where he got the afk add-on because I can't find it >>99812798
>>101876741how many goats can you offer
QRD on https://huggingface.co/anthracite-org
>>101876900>highest ranking of coals.brimstone
>>101876811Hm. I might actually attempt to learn how all this works and program my own frontend.>>101876531>Models convert inputs into outputs, you just want to have the UI poll it after a while of no action to get a continued output.Thanks for the pointer, I'll remember that.
>>101876900meme
>>101876900go back
>>101876916QRD on "go back"?
>>101876922You have to go back.
>>101876531I put my WhatsApp text history with my gf into completion mode for a llama3 to simulate. These models definitely know how to reply in consecutive messages and even initiate relevant questions on their own. We just don't make them do that, back and forth Q&A is what we do. The other guy is right, LLMs are already capable, we're just limited by the tooling and our imagination.
>>101872751Kek this. this will be less dissapointing than the real strawberry
>>101877083It's pajeet slop >>101876500
>>101876741I saw someone selling harddrives full of weights so I guess lol
>>101877031I feel like models are like brains without any brain activity.We need to create a layer on top of it that continuously streams input (senses) into this brain.The output (thoughts) should be used within some form of feedback loop that evaluates and modifies this input (thoughts) into actual output (communication).The response it receives from the input (senses) should be used to modify the (self-evaluation of the) model.Put all of this together and you'd have an autonomous agent, capable of self-improvement.
>went back to C-R+ after Largestral>the slop is all goneIf only the French didn't overfit their models...
>>101877200Liquid state machines/reservoir computing is a thing, and the concept may be similar to what you're thinking. Implementing them into an LLM however would be... weird.
>The training data contains around 3000 ebooks in various genres. Most parts of the dataset have been prepended using the following text: [Genre: <genre1>, <genre2>]Anyone know the specific genre tags used in the "holodeck" training data set?
>>101877396canada tasukete...
>>101876201Same but it is mikuspam for me. Glad you are seething mikufaggot.
>>101876528All that is gonna do is spawn cockblocking paladins.
>>101877396What Command-R+ quant are you running and what Mistral Large quant, and on what machine?
Is column-r really just some closed gpt? I want a useful model <100b
>>101877457Hm, I wonder if the two systems could work independent of each other, with a middleman translating the output of the senses into input for the llm.
>>101877623It seems to be much better at reasoning, but it's still just a GPT model.
>>101877200I'm tired of people like you, who can't grasp that language model/neural network alone isn't the solution to all problems.Like >>101877031 said, it comes down to tooling and imagination. Why not try making a video game NPC that uses traditional NPC ai to make decisions such as when to talk, and keep track of state, and just use the LLM to decide what actions to take based on state and other inputs.
AGI needs to predict more than just language. it needs to predict the world as a whole.even for humans, it's not like we only reason or even think through language.
>>101877669>I'm tired of people like you, who can't grasp that language model/neural network alone isn't the solution to all problems.Uh, that was actually my entire point. You can't really do anything with a brain-dead brain; you need to implant external electrodes (tools to steer the LLM) to make any activity happen.I'm really just bringing forward a few hunches and intuition, hoping that the people here (who know the tech much better than I do) can steer me the right way in case I totally miss the mark.>Why not try making a video game NPC that uses traditional NPC ai to make decisions such as when to talk, and keep track of state, and just use the LLM to decide what actions to take based on state and other inputs.That sounds exactly what I had in mind; a central management program consisting of several sub-programs that each translate a part of the current state of the world into data that is parsed into input tokens for the LLM.Something like that anyhow. I'll tinker with the idea a bit in the morning.
Wait column-r wasn't a Cohere model?
>>101877396imo cr+ is worse than cr desu
>>101877818We don't know.People think it's more likely that it's another OpenAI test model, but there's no definite proof.
>>101877818It is, people are still believing what the strawberry guy said for some reason lol
Can someone explain what character cards actually are? Is it using RAG or is it just a prompt? Can you use one in vanilla llama.cpp?
>>101877883It's just a prompt. Everything can be replicated with llama.cpp. Just copy/paste the bits you need. The prompts themselves are base64 encoded in a png text chunk in the image.
>>101877924can you give an example of what the prompt looks like? is it eg. something like "You are [character name] and have [character attributes], you are talking to [player name] with [player attributes]. respond like [story type]."? Is this a system prompt or a user prompt?
>>101877967Random card.
>>101877967To elaborate. Yes. What you wrote would be an example of a simple system prompt. User prompt is the dialog or whatever happens after that. ie, the conversation itself. But at the end of the day, everything gets translated into tokens and fed into the llm.
>>101878011Ok, thanks for explaining that. So "description" is a system prompt. What is first message? Why would I want it?
>>101878038Thanks. I though there might have been some sort of RAG in play that can check certain values from a database when context gets too long (eg. character age, hair ect.) I made my own textgen ui because I don't like any of the existing ones. (Mine is more similar to gpt playground on platform.openai.com.) I was wondering how hard it would be to support character cards but it seems pretty easy.
>>101878084I don't use ST, so i'm not sure how all the fields are used. I think it injects that text as the first 'response' from the llm:{system prompt}{injects first message as a reply from the llm}[user gets control here and types their input]{actual reply from llm after input}I think ST can randomize the first message and choose one of many if present. It randomizes a bit the setup so you can play the same card but still have some variety.
>>101878149Ah. I didn't even consider that the bot can go first in a conversation, but now that you mention it I guess there is really no reason why that could not be the case.
>>101878133>I though there might have been some sort of RAG in play that can check certain values from a database when context gets too longNah. It's all just tokens. Just load the png, read the text (tEXt) chunks and parse the json to extract the bits you need. You don't even need a png lib for that. Just parse it manually. That's what i do with png_hdr.c.>>101878172Just to clarify, the first message is injected. It's pushed into the context as if the llm spat it out with the model's side of the chat template. It basically makes the llms go 'oh, i said that, so i better keep it up'. In case i wasn't clear.
QRD did agi happen? was strabw full of shit?
>>101875891>>101877494why do fags like these hate mikuposting so much? the only time it actually gets annoying is when mikuposters spam to spite retards' bait posting like these in the first place. are they actually just newfags/trolls trying to shit up the threads or is there an actual reason for mikuhate?
>>101878251Two more days until the next two more weeks begin
mikubros..
agents
bitnet
>>101878341how
>>101878341>>101878371Liveportrait or something? I remember people were experimenting with that.
>>101878341kek trannies ITT will like this
>>101878341facerig has been a thing for a decade now
>>101878297It has nothing to do with local models and it is a troon idol. Btw kill yourself troon.
>>101878297miku is outdated character, you can see tranny erp miku threads on /v/, she attracts blackedfags, and so on, so yes, root of the problem should be removed.
>>101878297this should be a makise thread
>>101878522>>101878565>>101878589samefag
lol https://hub.virtamate.com/resources/koboldlink.49327/
>>101872662Is OSS voice generation any good yet? I want something that can generate a california girl/max vocal fry voice that I can fap too.
>>101878962no and never will be, just use cloudshit, that way is cheaper, unironically.
>>101878522Why the fuck would you let troons have their way? Miku never was and never will be troon related.
what the fuck is a troon?
>>101879040Name an idol that is more of a troon idol.
>>101879046Have you been living under a rock?It's honestly better you keep living under it and not learn about this.
>>101877802Sorry, just sounded like you were saying "let's just add more layers and link input and output layers via more neural network black magic" like too many other people tend to do when the magic black box LLM doesn't do things exactly the way they want.That said, I DO think some kind of actively updating weights lora could be useful for task specific products.
>>101879056I don't know and I don't give a shit, but how much of a cock slobbering cuck are you that anytime a troon comes along and claims something for themselves you just let them have it?
>>101879087I don't that is why I continue to oppose all the mikutroons in this thread by shitting on them and calling them out for offtopic. Kill all mikuposters. Make /lmg/ great again.
>>101878522>>101878565dogshit reasons. the association between miku and ai shit (local or not) is synthetic voice generation/ the common mislabeling of vocaloids as ai -> robot/chatbot misconceptions and associations -> local llms. also, the tranny/blacked shit was started and propagated by fags like you two and kurisufags like >>101878589
>>101879128your mom fucking a nigger has aids > you have aids > aids has ai in name > local llms. congratulations your mom fucking niggers has became the new mascot of llm. how do you feel?
>>101879046A man that pretends to be a woman, usually without any surgery on his 'ick but when xe decides to do that - it ends very bad, rotting wound and all that shit.
>>101879128>calls out dogshit reasons>proceeds to list dogshit reasons himself
Strawberry will change the world. You are all wasting your time talking about models that will be obsolete the second OpenAI drops the 'berry.
>>101879128The core reason is simple though, miku is just boring as character and fags spamming her are boring too.
>>101879122Did you know troons breath air? You should oppose troons by refusing to breath air too.
>>101879128Its always the same autist derailing the thread.Are you guys retarded? Jahnnies included, the guy openly admits to using GTP to shitpost on lmg.
>>101879122miku has been a part of these threads longer than you have newfag. lurk more retard-kun
>>101879175>"food" analogy lmao
>>101879175Compelling argument. I gave it a thought but I think I will just stick to shitting on mikufaggots, thank you. Kill yourself faggot.
>>101879194I'm not the faggot who obsesses over troons
>>101879207About obsession :
>>101879193>part of these threads longer than you have newfagAnd? Even if the first thread OP was made by a mikutroon it is about time for them to leave for their discords and never come back.
>>101879185>not even 5 minutes and already proven rightlol lmao
>>101879146Are you trying to say tranny? I don't know if this post is making fun of me or genuinely trying to explain, but for the life of me I cannot understand zoomer lingo.
>strawberry shitposting dies>mikufaggotry shitposting rises againNever change /lmg/
>>101879249That was a rough explanation of "tranny" or "troon" buzzwords, definitely not zoomer lingo btw, or maybe i am missing something here.
>>101879255obsessed
>>101879247>thread at ~300 posts already>gee why dey derailing my preshius circlejerk general???? get lost.
>>101879272>"troon" buzzwords, definitely not zoomer lingoI have not seen that word a single time until the election tourists invaded. I still don't get where you all got it form.
>>101879249This is what happens when your LLM isn't trained on up to date material>>101879236>one random troon claims miku is their idol>so brainrotted you can only ever thin of troons now when miku is postedsincerely, get help
Do local models even warrant a general? There is nothing to talk about for 3/4ths of a year. The OP is a collection of useless links that nobody has touched or updated in a year. /lmg/ is the Katawa Shoujo general of /g/.
>>101879142>counterpoint isn't even tangentially related to anything I saidlobotomite >>101879157>>101879174fair enough >>101879185I've only posted in these threads like 5-6ish times including the posts in this thread faggot. also, I'm too lazy to go through the effort trying to tardwrangle gpt into writing my shit for me
>>101879291I don't know it's origins :/ But word itself is pretty great name-call for corresponding "people" that's why everyone (including resident zoomers) use it.
>>101879185Jannies don't care. At best they're indifferent to any thread not about consumerism, and at worst they're openly hostile to AI threads. They're happy to let him shit the place up until it dies. Don't know why hiroshimoot ever bothered with the whole 4channel thing.
>>101879300I agree, this thread has been tearing itself apart over a fucking berry for a week straight now.
>there it is>like pottery
>>101879331Who are you quoting, schizo?
>>101876531What would this look like in practice though? Sending a blank user message after 30 seconds? How do you trigger another completion without the AI responding to the fact you didn't say anything?
I'm beginning to understand anti-AIfags more and more.
>>101879346henlo newfriend, since this is your first /lmg/ thread allow me:>spam AI CP from discord>shit up the thread with reddit and twitter>derail it to just for good measureTHEN>"lmg sucks delete the general"THIS HAS HAPPENED REPEATEDLYAND NOTHING EVER HAPPENS HAHAHAHA
>>101879387>/r/inflationLike money inflation, right?
I am going back.
>>101879411:)
>>101879353Well for ST if you just hit the continue button it'll give you a new reply even if you don't type anything, so just some script that triggers that same function after inactivity?
Anything new in the coomer model range?I have 24 gb of VRAM but for the sake of speed i sue NemoRemix 12b or that new Stheno 8b.
>>101879387Reddit is only the first step. Wait until your insurance is doing this. Getting a loan already requires bank statements. Now it's just a few months, eventually they'll have AI scanning every transaction you've ever made for an excuse to charge you more. Now is also probably a good time to mention that Google maintains a list of every location you've ever visted. At some point an employer is going to want to search that with AI to make sure you are a good fit for the company values. Then there's your search and browsing history. Google has a lot of new revenue streams and society is safer, win win. :^)
>>101879353Look, context is everything here. If you want the model to know something, like the fact that you haven't said anything to it in some time, then fucking put it in the context. Remember, you don't have to have the AI actually say something every time you send it a message either, you can treat it's output as internal monologue and force it to be specific about when it speaks and what it says. Give it a context like "Here's the chat history, it has been 15 minutes since Anon has said anything, Here's a current screenshot (for multimodal models) should you speak up? If so, what do you want to say?" etc etc.
>>101879481CommandR or Gemma2 27B.
>tensorsplit on kobold_rocm is busted so all the 8x7bs like Limarpzloss and OpenGPT that where good dont work
>>101879300>basically every thread besides ones about buying hardware or ai/content generation on this board takes at least 12-24 hours to hit bump limit, if they're even able to hit that before they're archived>half of those threads are about programming languages that usually consist of anons samefagging about whether their favorite language is better than the other and are only interesting when someone rarely posts their personal projectsdeserves a general about as much as the others do
PSA: exlama2_HF is fucked, use regular exlama2 in booga.Just spend half a day troubleshooting this shit.
>>101879620Actually i take that back, tensorsplit is just busted on the new version.Wtf multi GPU AMDbros????
>>101879672Works on my machine (which hasn't been updated since v.0.17)
>>101879533>Gemma2 27BSucks major balls. Easily worst recent model. And I should love it as single gpu fag.
>>101879596This is now the mascot of /lmg/. Anyone who disagrees is a closed ai employee.
>>101879672source?
>>101879715Then just keep using something like command-r or go to 70b and deal with slowdowns. Not much new if you grew bored of nemo & the like.
>>101879691It is time to git pull anon.
>>101879715Really?That's the first I've heard of that.People seemed to be pretty positive on it for the most part.
>>101879748>>101879806thats not me lol, but i can try both for myself for that matter.What Command R quant?>>101879715for that matter why does gemma2 suck
After tiring of Nemo going dumb after 40 messages I returned to trying MythoMax last night which has been my staple for the past year.Being locked to 4k context means it can only hold 30 or so messages before the dementia kicks in.I can no longer go back. I fucked myself.
How bad is it to switch out the system prompt mid-chat?
>>101879806At first I thought it was bugged loaders but eventually all bugs got ironed out and it still fucks up formatting. Even 7B's that are recent never do that which is a bit of a tell that something is fucked up about 27B. And even if you get past that the prose is complete garbage. Everyone speaks in poems.>>101879922Try Nemo with 4k context and see if it never goes dumb.
>>101879868I think if you want it all on the gpu you could do 3.5bpw with about 14k context. I use the gguf as it's fast enough for me, q4 was the smallest I was happy with.
>>101879930Not an issue.
>>101879930there's a pretty big chance your future gens in that chat will be affected by the messages in the context but other than that not really a problem
Any progress on bolting a LLM to roleplayeing tabletop games? I remember a bunch of anons talked about developing a silly extension, also talks of custom front-ends.
>>101880195What would that even look like, Anon?
>>101879046It's a derogatory term for transgenders. Please don't use it or you will make the Miku posters sad.
>>101880411NTA, but anons started talking about that around the same time this one streamer tried it, so presumably it would work like his setup. His front end had some 3 LLMs working in parallel, each one's output would be cloned and routed to the others' chatlogs, allowing for a basic TTRPG table continuity. The rest was prompting up characters, general AI wrangling, and micromanaging which LLM should respond next. It was pretty novel as a concept, but the results were about as mid as you would expect.
LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMshttps://arxiv.org/abs/2408.07055>Current long context large language models (LLMs) can process inputs up to 100,000 tokens, yet struggle to generate outputs exceeding even a modest length of 2,000 words. Through controlled experiments, we find that the model's effective generation length is inherently bounded by the sample it has seen during supervised fine-tuning (SFT). In other words, their output limitation is due to the scarcity of long-output examples in existing SFT datasets. To address this, we introduce AgentWrite, an agent-based pipeline that decomposes ultra-long generation tasks into subtasks, enabling off-the-shelf LLMs to generate coherent outputs exceeding 20,000 words. Leveraging AgentWrite, we construct LongWriter-6k, a dataset containing 6,000 SFT data with output lengths ranging from 2k to 32k words. By incorporating this dataset into model training, we successfully scale the output length of existing models to over 10,000 words while maintaining output quality. We also develop LongBench-Write, a comprehensive benchmark for evaluating ultra-long generation capabilities. Our 9B parameter model, further improved through DPO, achieves state-of-the-art performance on this benchmark, surpassing even much larger proprietary models. In general, our work demonstrates that existing long context LLM already possesses the potential for a larger output window--all you need is data with extended output during model alignment to unlock this capability.https://github.com/THUDM/LongWriterhttps://huggingface.co/datasets/THUDM/LongWriter-6khttps://huggingface.co/THUDM/LongWriter-glm4-9bhttps://huggingface.co/THUDM/LongWriter-llama3.1-8bnot exactly novel of an idea but they made the pipeline and created a dataset then finetuned some models so worth appreciating just that alone. example on the github of a short fiction story so not just nonfiction articles created
>>101879715Is your name petra?
are you so aspergers you don't even want to watch real humans playing D&Dyou'd rather watch NPCs?
>>101880673That reminds me that I should give glm another try.It behaved really weirdly when I tried it back when llama.cpp merged support for it.
>>101880111The old system message will obviously be replaced so, no?
>>101880195I envisioned (and charted out, I'm pretty sure) a LLM based system that was mostly just a game, with all calculations, state managemeng, etc being done by the usual algorithms, and the LLM would be used mostly as a narrator that gets fed state information and the player's input and outputs narration and options, with a strict format based on grammar and a lot of RAG/lorebook information to guide the DM.I need to find where I stored my notes. I remember that I was mostly going to implement shit from the D&D 3.5e/d20 SRD, making some changes regarding distance mainly, using a close, near, far system instead of explicit distances to make things simpler.I think that was the gist of it.
>>101880831THey said, the other messages in the context, not the original system message but this feels like premature optimization/features no one is asking for
>>101880989>>101880989>>101880989
>use NTFS on portable drive for compatibility with both Windows and Linux>Linux crashes one day>need to go into Windows to repair the drive lol>repair successful according to Windows.>use PC normally for a while>notice that some files are missing>go do a search for them>it turns out they're now in a "found.002" folder and what happened was somehow the crash caused literally HUNDREDS of files to lose their filenames and paths so now I have to go manually rename and place them all properlyNever had this issue with Linux's native file system nor when I used Windows and it had crashed in the past.FUCK YOU whoever is responsible for this being the way things are.
