/lmg/ - a general dedicated to the discussion and development of local language models.Previous threads: >>102356839 & >>102348952►News>(09/12) DataGemma with DataCommons retrieval: https://blog.google/technology/ai/google-datagemma-ai-llm/>(09/12) LLaMA-Omni: Multimodal LLM with seamless speech interaction: https://huggingface.co/ICTNLP/Llama-3.1-8B-Omni>(09/11) Fish Speech multilingual TTS with voice replication: https://hf.co/fishaudio/fish-speech-1.4>(09/11) Pixtral: 12B with image input vision adapter: https://xcancel.com/mistralai/status/1833758285167722836>(09/11) Solar Pro Preview, Phi-3-medium upscaled to 22B: https://hf.co/upstage/solar-pro-preview-instruct►News Archive: https://rentry.org/lmg-news-archive►Glossary: https://rentry.org/lmg-glossary►Links: https://rentry.org/LocalModelsLinks►Official /lmg/ card: https://files.catbox.moe/cbclyf.png►Getting Startedhttps://rentry.org/llama-mini-guidehttps://rentry.org/8-step-llm-guidehttps://rentry.org/llama_v2_sillytavernhttps://rentry.org/lmg-spoonfeed-guidehttps://rentry.org/rocm-llamacpphttps://rentry.org/lmg-build-guides►Further Learninghttps://rentry.org/machine-learning-roadmaphttps://rentry.org/llm-traininghttps://rentry.org/LocalModelsPapers►BenchmarksChatbot Arena: https://chat.lmsys.org/?leaderboardCensorship: https://hf.co/spaces/DontPlanToEnd/UGI-LeaderboardCensorbench: https://codeberg.org/jts2323/censorbenchJapanese: https://hf.co/datasets/lmg-anon/vntl-leaderboardProgramming: https://hf.co/spaces/mike-ravkine/can-ai-code-results►ToolsAlpha Calculator: https://desmos.com/calculator/ffngla98ycGGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-CalculatorSampler visualizer: https://artefact2.github.io/llm-sampling►Text Gen. UI, Inference Engineshttps://github.com/oobabooga/text-generation-webuihttps://github.com/LostRuins/koboldcpphttps://github.com/lmg-anon/mikupadhttps://github.com/turboderp/exuihttps://github.com/ggerganov/llama.cpp
►Recent Highlights from the Previous Thread: >>102356839--o1-mini model excels in generation but struggles with completion tasks, sparking discussion on benchmarking methods and model capabilities: >>102361744 >>102362578 >>102362651 >>102362675 >>102361852 >>102362003 >>102362086 >>102362876 >>102362897--Anon solves code-breaking puzzle that o1 failed to crack: >>102357917 >>102358165 >>102358342 >>102358238 >>102358243 >>102359615 >>102359743--OpenAI's decision to hide chains of thought from users, with mention of monitoring for manipulation: >>102361616 >>102361628 >>102361644 >>102361735--O1 model solves CTF challenge by exploiting Docker API misconfiguration: >>102359414 >>102359504--M series Macs for running large local models: >>102360064 >>102360374 >>102360467 >>102360519 >>102360612 >>102360629 >>102360761--LLMs can reason, but lack general intelligence and autonomous learning: >>102361502--Extracting CoT prompts and the challenges posed by OpenAI's guidelines: >>102361025 >>102361046 >>102361131 >>102361217 >>102361170 >>102361205 >>102361147--Backend CoT process and its impact on writing fiction prose: >>102361172 >>102361226 >>102361300--Anons discuss OAI's unverifiable and potentially exploitative token charging system: >>102361649 >>102361664 >>102361680 >>102361688--Anons discuss GPT-4-O1 access, benchmarks, and implications: >>102358770 >>102358792 >>102358811 >>102359089 >>102359114 >>102359280 >>102359426 >>102358911 >>102358957 >>102359055 >>102359661 >>102358814--Anon is impressed by Google NotebookLM's podcast generation capabilities: >>102359806 >>102359843 >>102360015 >>102360252 >>102363640--Anon complains about being billed for invisible reasoning tokens: >>102360189 >>102360210 >>102360220 >>102360246 >>102360251 >>102360286 >>102361570--Livebench results are out: >>102363403 >>102363472 >>102363556 >>102363610--Miku (free space): >>102356889 >>102357160►Recent Highlight Posts from the Previous Thread: >>102356847
>>102364945I'm using llms at work for coding. I'm not crazy enough to use local for that though, sonnet 3.5Besides coding and creative writing llms are still useless. Imagine being so assmad because I enjoy my loli beastiality RP at the click of a button lolIts like somebody draws titties with the new MS95 paint tool and you put on the nerd glasses>uhm all that technology and all you use it for is this icky stuff. cant even.get real man.
>>102365023True
>>102365023>>102365054nta but i think you missed the point
All this strawberry and qstar stuff and llama4 will be another pure transformer llmslop pumped full of synthetic data. Lecun is a hack>b-but he doesn't work on llmsI don't give a fuck. What does he work on that the only thing Meta keeps pumping out is LLM after LLM?
>>102365065Llama-V-JEPA trust the plan
>>102365065He is busy sperging out on elon and brigading for every vaxxie's god fauci ouchie.
>>102365023nta but you have irredeemably shit taste
>>102365023You posted your smut like it was peak writing. How can you not cringe when you read that shit is beyond me. Besides, that had nothing to do with CoT, and your complex format usually collapses very fast as it reinforces the repetition in every message.
oh god the morality police is here
>>102365023You are faggot, simple as.
>>102365093>>102365104>How can you not cringe when you read that shit is beyond me.Its great output for 12b. It actually follows the format and is creative enough.Kinda insane I have to justify RP in a /lmg/ thread. I hope you are all the same guy.
>>102365133Go easy on him, he has a failing company to lead.
Anyone with o1 access managed to try this? >>102363366
>>102364922Simulating cats with Yann Lecun
>>102365256It got it after thinking for 52 seconds.
>>102365371that's pretty impressive not gonna liegpt-4o typically just fucked up the velocity calculation or just gave me the formula for circumference. If it got to the integral part, it usually gave up and said it's a non-typical integral
oh god the physics majors are here
>>102365371That's pretty fucking cool, but it would be even cooler if it didn't need to <think> for a fucking minute.
>>102363366What is that supposed to test?
>>102365494it's a multi-step process. Until now, all failed to solve it because they are rushing for the answer.
Why OpenRouter's O1 doesn't work anymore?
am i using o1 right?
>>1023654851 minute is quick enough for this kind of question, tf are you complaining about?
>>102365565>team fortress are you complaining aboutfor a language model, sure, but for agi 2 more megawatts that's pretty pathetic
>>102365485The thinking part is probably something that will be needed in the future and just not visible or only with a debug flag or something.We need sheer unlimited context and much higher speed, much cheaper. 1 Minute waiting down is no fun, I agree. Not sure I will use o1 even if becomes cheaper.Imagine you need something done quick just for the llm to trip up like usual and need another request. o1 does make mistakes. Sometimes doesnt even get the stupid ass "strawberry" test correct.Double painful for the wasted tokens.
How do I make my own diffusion model? (No finetuning) I have the images.
>>102365639Got a few millions $?
>>102365695Yes.
>>102365639>I have the images.did you caption them all? kek
Based thread.
>>102365458Yes, I've been here since the LLaMA 1 leak.
>>102365707Automatically yeah
>>102365850>automatic slopsave your money
>>102365854who's pretraining their model with manual captions? you need hundreds of millions of pictures to pretrain a model
>>102364922Yann LeCum
>>102365894Yum! Le Cum
So now that we know that the reflection guy was actually correct when is someone going to finetune mistral large similar to what he did and do it correctly this time?
What was the last time somebody used the SuperCOT dataset?
>>102365943modern models have implicit cot for hard queries.
>>102365932Never.
So let me get this straight....OAI just did a CoT finetune of 4o and they're acting like they reinvented the fucking wheel?How are investors this stupid?
>>102365965CoT finetune + new tokenizer + specifically trained for huge CoTs and talking to itself, remember those "System A, System B" things from last year?The CoTs are 600 tokens minimum even for "hi". And for real questions 2000-10000
This redditor is smarter than the average /lmg/ anon
>>102365977So basically a CoT finetune.
>>102365965If I got it correctly, it's something more involved than just that. It's CoT + having the model itterate over its own output in some form (the whole think longer part) which could be just looping its hidden layers N times (something I've seen lmg discuss over a year ago I'm pretty sure), or literally feeding its output as input to itself a couple of times
>>102365981Oh it's definitely a meme. Post above yours says it perfectly.
>>102365980>>102365994there's no way the 1 year Strawberry hype was just some CoT finetune, OpenAI doesn't know what to do anymore
>>102365982The whole thinking longer is just asking the LLM something like "Criticize this output and write an improved output that improves the initial output." over and over again.
>>102366016Yeah, it's most likely that.
>>102365982You can literally do that by having the model write out its output to a variable and iterate back over it with instructions to do something with it. Yandere AI Girlfriend Simulator did that with fucking turbo, an old, obsolete, retard model by today's standards and it worked out fine most of the time and error handling prevented it from acting retarded if it didn't work.It's basically just a CoT finetune. >>102365980It would certainly seem that way now, wouldn't it? But I'm 99% sure most of the people posting positively about o1 here are either paid shills or mentally ill. It's not possible to be as stupid as they are by accident.
>>102359806>https://notebooklm.google.com/I logged in, how to use it?
>>102366016Why is the user needs to pay for safety tokens instead of US government?
>>102366027Well, the matter stands that it's much better at Maths and STEM. It is impressive.However normies will soon find out that anyone can do this. Real winner will be Opus 3.5
>>102365860I just need a tutorial; I only see indians on YouTube
this is a picture of sam altman made by strawberry in style of feels guy. only agi could produce something like this.
>>102366032You click the big blue button, upload a text/pdf and then you can use it like RAG or generate an audio podcast.
>>102366043>Well, the matter stands that it's much better at Maths and STEM. It is impressive.Prove it.Other than meme mark scores which have been proven time and time again can be fudged.
>>102366076There's always Livebench, the least gameable Benchmark since it constantly changes.https://livebench.ai/And there is a hard test done ITT above too that the anon said no LLM has completed
>>102366096>SARRS, SARRS PLEASE REDEEM THE REASONING SARRS
>>102365558baaaaaaaaaased
>>102366096See: >>102365980
>>102366107You have lost track of the conversation. No one is shilling o1 here. It's just a CoT finetune of 4o but it works.Any company can make it.
>>102365980nah, redditors think that nu-CoT is the new paradigm, they are beyond recovery at this point/lmg/'s consesus is that o1 is yet another reskinned gpt4, aka a nothingburger
>>102366096>https://livebench.ai/>coding is still worse than sonnetNot looking too good for s(c)am
>>102366096>And there is a hard test done ITT above too that the anon said no LLM has completedmaybe gpt4-o1 can do it, but we'll never know, Sam thinks it's too powerful for us poor mortals. https://youtu.be/GzlKja1ySzo?t=9
>>102365860>what are *boorus autists, mechanical turks and kenyans
>>102366096>Claude 3.5 Sonnet still the best at the most relevant assisant capability, codingnot looking good at all for OpenAI
>>102366096the least gameable benchmarks are the ones with a 100% private dataset, both questions and answers
>>102366170yeah so you need a shit ton of money to hire all the jeets that will manually caption your images, that's not something accessible to anyone, far from that
>>102366174Those sound prime for bad actors and being paid off.Livebench is a solid compromise, Public and questions get switched every month. They have talked about their methodoly, and they get math etc. questions from recent olympiads which are not possible to be in the database for example.
>>102366181That was the first requirement >>102365695
>>102366192>questions from recent olympiads which are not possible to be in the database for example.that's a good strategy desu, those guys know what they're doing
>>102366192>being paid offanyone can be paid off, some company may have paid the math olympiads off to get the new questions long before the release of their model>questions get switched every monthit still includes old questions
>>102366192also "each month" is too long, they can finetune on those new questions faster than that
this is an animation of sam altman happily jumping up after taking his sixth bbcooster shot made by gpt-o1(q*/strawberry). can you feel the agi?
>>102366250This. It may take months to train a foundational model but it takes more hours to fine-tune them, especially on h100s.
go back Petr*, no one likes you
how long it takes for you to generate podcast? I'm waiting for +6 minutes rn
>>102366356It does take a while.
are there any good finetunes/mergesloppas of mistral large 2 or are we just stuck with the original model and magnum?
>>102366354Who is Petr*?
So I think at this point it's obvious that saltman is behind the strawberryQ* larp to try and hype up his nothingburger product.>creates fraudulent viral marketing campaign>surprise it's a fucking CoT finetune of 4oHow are investors not suing the shit out of him as we speak?
>>102366375Just use Large with the XTC sampler, or with high temperature and min p.
>>102366405good question. I've been around since around the time anons were claiming that mistral 7b could beat a 70b and I still don't know who it is. the first time I think I heard about Petra was in aicg threads for something to do with proxy logging or some other stupid shit (I could be misremembering this because there were a lot of retards shitstirring in both aicg and lmg around that time), but I still have no clue what Petra did unless it's just something that some schizofag started and everyone else thought it would be funny to start accusing anons that they disagree with or can't argue against as being Petra.>inb4 hi Petra
>>102366575What temp + min p do you recommend?
>>102366448Investors are dumb. Look at what another Sam(Bankman-Fried) could almost get away with.
>>102366583the anon you're replying to is xer
>>102366616Exactly.Your average investor works purely on hype and line-go-up principles.
how to fish on windows+cpu?
>>102366633thanks *rapes you with retard cock*
>>102366583Last year schizo:https://desuarchive.org/g/search/text/petra/start/2023-05-20/order/asc/
https://youtu.be/gkhwK6Wlod8
>>102366692Oh, it's the ugly bitch spammer. Must be some kind of brown who sees any blonde and thinks, "SHiieeet nigga, dat bitch hot," even if she's ugly or fat.
/lmg/, I am going to generate erotic fiction and I need only your BEST local LLM that's below 10gb. What is the objectively best one?
>>102366809 (me)sorry, forgot my avatar
>>102366809Gemmasutra 2b
>>102366809Stheno works for me. So does Lumimaid.
>>102366809Nemo.Either Lyra or mini-magnum.
>>102366809Tiny Llama
>>102366809Nemo finetune like magnum-12b-v2-q5_k or MN-12B-Lyra-v4.Q5_K_M.Stheno is too horny. Part of the fiction should ne resistance.
>>102366842baaaaaaaaaased
>>102366809I'll provide you with an overview of popular local LLMs (Large Language Models) that are under 10 GB. Since "objectively best" can be subjective, I'll focus on models that are widely used, well-documented, and suitable for text generation tasks like erotic fiction.Top contenders:>T5 (Text-to-Text Transfer Trained): A highly versatile model developed by Google. It's a text-to-text transformer with a relatively small footprint (around 5.6 GB). T5 has been fine-tuned for various tasks, including text generation, and has shown impressive results.>BART (Bidirectional and Auto-Regressive Transformers): Another Google-developed model, BART is a denoising autoencoder that can generate text. It's smaller than T5 (around 4.5 GB) and has been used for text generation, summarization, and translation tasks.>Longformer: Developed by Facebook AI, Longformer is a long-range transformer that can handle longer input sequences than traditional transformers. It's relatively small (around 4.2 GB) and has been used for text generation, summarization, and question-answering tasks.>DistilBERT (Distilled BERT): A smaller, more efficient version of BERT (around 3.5 GB), developed by Hugging Face. DistilBERT has been fine-tuned for various tasks, including text generation, and is a popular choice for many applications.
>>102366692oh yeah, I remember the ugly bitch spam I just thought that there was more to it than just a sperg shitting up the threads. also I did fuck up my memory of the aicg shit with branon, my bad.
>>102366809i just use mistral nemo finetunes until they start feeling stale dozens of hours in, then grab a new one.currently on MN-12B-Chronos-Gold-Celeste-v1.Q4_K_M>>102366867these are great, ended up dropping lyra after seeing the same slightly odd focus on one aspect of penetrative sex a few hundred times.
>>102366172COT haters stay winning
>>102365943The last time I mixed it in to an existing known-good dataset the result got me dragged and called a retard here for weeksSo, fuck COT
>>102365532Probably because OAI didn’t give them permission to use their account for a mass release but they couldn’t resist doing the most dark pattern option available as alwaysHopefully it makes OAI revoke their keys and openscam dies
>>102366997>i just use mistral nemo finetunes until they start feeling stale dozens of hours in, then grab a new one.the secret is using nemo instruct until it starts going schizo, switching to one of its boring finetunes for a while and then going back to instruct once the context is fixed
>>102367030>The last time I mixed it in to an existing known-good datasetAs in, you just threw it all in the same fine tune?Wouldn't it be a case where you do more than one pass with different datasets?
>>102366375https://huggingface.co/schnapper79/lumikabra-123B_v0.4Slop is back on the menu
>>102367095I merged it in after in a separate step after training both separately.
>>102367192You trained A on dataset A, then B on dataset B and merged them?Interesting.Or did you train one model on dataset A, then dataset B?
>>102367212The former, basically because I was afraid that that COT would make it retarded so I wanted to have the uncontaminated one just in case. And then yeah it made it really dumb.
>>102367128downloading as we speak. This better be some primo slop.
how can we compete with o1
>>102367335I AMREFLECTTIIIIIIIIIIIIIIIIIIIIIIIIIIIIIING
>>102367335By putting saltman through conversion therapy
>>102367335Find a good prompt that gets it to spit out its real thoughts then make a dataset from itWhat we know so far is that it only sees its thoughts for the current prompt. For all previous messages in a history it sees just the outputs we do. And of course it's told not to share its thoughts in the output, but if there's a consistent enough jailbreak something could be done in a semi-automated way.
>>102367386>Find a good prompt that gets it to spit out its real thoughtsIt's a linear transformer model like everything else. It doesn't have "real thoughts."Those "thoughts" are just a bunch of pajeet rambling from its training dataset. But from the outputs alone you can't reconstruct the training parameters that lead to the output. Or which areas the dataset was broader and which areas it was narrower, etc. This is "gaucano-7b beats GPT-4" tier nonsense thinking.
>>102367461No shit retard, we're talking about the chain of thought that's the first part of the output that gets hidden from the actual result they send. The model is reading them verbatim while generating the public part of the output, so it could in theory be jailbroken to reveal them.
>>102366369Now I've been waiting for over an hour.
>>102367335We've had the tools since the starthttps://huggingface.co/datasets/kaiokendev/SuperCOT-dataset
>>102367128>>1023672984.0bpwhonestly not bad. Sloppy, obviously, but the gazelle remark is great. It produced that entirely via indirect association with the scenario.
>>102367671Yikes, it shouldn't be more than 15 minutes. You should try again.
yes, this is sam altman, depicted as an omniscient and omnipotent angel bringing humanity enlightenment in form of q*. yes, this picture was made by gpt-o1, which is q*.
Lecun was right again. There's no secret knowledge, no breakthrough locked in a lab somewhere. Whoever implements it properly gets the money. These OAI niggas were hyping up CoT like the second coming of Christ
ITS UPhttps://huggingface.co/TheDrummer/Buddy-2B-v1
>>102368107I, for one, am excited to pay for tokens that I never get to see.
>>102368143The madman did it. He moved on from coomgen to sentiment analysis in only a couple of short months. Drummer AGI is only 2 weeks away.
>>102368143>i do not wish to be horny anymore>i just want to be happy
>>102368143seems a bit overcooked.
>>102368143>I'm serious about the license for this one. NON-COMMERCIAL. Ask permission.sue me lol
lads how can we cope against the hidden CoT of openai?
>>102368328lucky for me, I can't read.
>>102368336>copeYou mean rejoice?They've proven that it's literally possible to hold an AGI in your pocket.
>>102365371did they intentionally make it better at math just to help students cheat?is that their primary userbase?i can't imagine why else you'd put so much effort into having it solve math and physics problems. Actual mathematicians and Physicists have no need for such a thing.
>>102368336We celebrate. Need less distilled slop, not more.
>>102368336We don't, accept your defeat and join cloudgods, be free from endless tardwrangling.
>>102368353they think (they want you to think) it's going to become the next einstein
does the CoT work if you talk to it in a language that isn't english?
We also need CoT finetunes for RP. Change my mind. What we're doing is not the way to do it.
>>102368392but einstein was actually shit at math
>>102368351>AGI>CoTGo back
>>102368394It should. LLMs are language-agnostic.
>>102368353>did they intentionally make it better at mathThey're selling their benchmark results as 0-shot results which is a complete and absolute lie.CoT is a multi-shot approach. So their supposed benchmark improvements are basically all invalidated by this.
>>102368421how are they going to help?
>>102368433but persumably the CoT/instructions from openai are only in english>>102368441true
>>102368336Funny enough they didn't do what I expected, which is Monte Carlo tree search with mutual reasoning
>>102368441it's 0-shot if it ends up choosing a single answer on its own, the intermediate steps don't matter
does CoT improve math and physics the most because the answer is always objective and usually numeric, whereas other subjects are subjective? would also explain why it's so bad at english compared to AP math/physics.>>102368453too complex, they need to ship hype so they have to repackage old techniques under meme names
>>102368469That is the most jewish thing I have read all day. And I follow middle-eastern geopolitics pretty closely.
>>102368469it's 0+N*i shot, the CoT should be counted as imaginary for this discussion. They have a very definite impact on computational cost and execution time, so they cannot simply be ignored, but they are somewhat orthogonal to traditional N-shot
>>102368336I am more disappointed that the technique seems just good for math and code. Anything with writing or translations it just gets dumber by "thinking"
>>102368449>but persumably the CoT/instructions from openai are only in englishThat doesn't matter.It has also been trained on other languages, meaning it will have a connection between English words and the other language's words.It might not understand what "Du bist ein neger." means, but it does know that "Du" means "You", "bist" means "are", "ein" means "a" and "neger" means "nigger".And therefore it will know how to answer to "Du bist ein neger." in the same way in would answer to "You're a nigger.".Exactly THAT is the magic behind LLMs.
>>102368444The model should summarize the character's locations, attributes, and how they should act, then plan the next action.If you try to enforce CoT on small models they either absolutely shit the bed, or fall into pattern from their previous CoTs, or do not know what to do with it. Even on bigger sizes a specialized CoT model would be better.
>>102368513i posted above but i think it's because those usages don't have an objectively correct answer to converge to, so CoT probably oscillates between local "best solutions" or some shit (or otherwise fucks it up)imagine asking it about philosophy, or dualism or something -- the "chain of thought" could simply end up being arguments between different camps /sci/ can eventually converge on an answer but boards like /lit/ will argue forever
>>102368531languages do not map 1:1there is a quasi sapir-whorfism nature to LLMs, both as the language chosen determines results due to the distribution of training data in diffrent languages, and because of the inherent diffrences in languages themselves.
>>102368143I feel more suicidal than before and it's all YOUR fault.
>>102368572Well, obviously? But that difference is negligible for that anon's purposes.
>>102368616EVEN NOW THERE IS HOPE FOR MAN
>>102368490>>102368500the benchmarks are not comparing computational costs, otherwise the model size would also be taken into accountthey're comparing accuracy, 0-shot means that the model is expected to get it right on the first try, N-shot means that you give it N chances to get it rightif the CoT model comes up with the right answer while thinking but then starts hallucinating and gives an incorrect final answer, the answer is considered wrong, so it's still 0-shot
>>102368616why does the name change
>>102366636need linux for torch.compile otherwise its too slow
>>102368695Because it's a braindead 2B model.
>>102368683Not quite. n-shot is the number of prior examples (with answers) the model is provided in context for whatever task it's performing. This is still used now, but it originated in the autocomplete days when "monkey see, monkey do" was really all you had
>>102368683Nta, but benchmarking cloud models is useless anyway. There could be a human in the loop during the CoT process and the benchmark will not be able to see it.
>>102368726we use parametrically challenged here, anon. This is a safe space
>>102368469>AGI IS COMING SOON BRO, JUST TWO MORE (HIDDEN) SHOTS BRO
>>102368683>just ignore cost bro
Pixtral is trained on the exact same OAI derived slop as all the Chinese vision modelsThe wonders of """independent""" european AI. Almost a billion $ in funding. Still just mooching off of GPT4V, not even the better 4o captions, no, the most basic slop filled pieces of shit instead. And this took them 12 months longer than the Chinese.I fucking hate the French. I fucking hate the Chinese too. And I fucking hate OAI for putting endless slop out there that no AI company can resist.
>>102368726why not regex it back to what it should bewhy do AIfags hate regex so much
I think ultimately what we probably want is a different version of the leaderboard which takes into account the number of tokens spent. Both costs and speed can change over time given that infrastructure can be changed while the model stays the same. We want to measure the model's inherent capability after all, not the system it's running on. Therefore the most stable measure of token-based "thinking" is the token count. Not surprising.
Apparently, this is the strawberry's CoT: https://rentry.org/openai1It's pretty intriguing, and makes sense why this works well for reasoning but makes no difference for text.
Exolabs on Twitter is saying they have Llama 405B running on only two Macbooks
>>102364922>>102364922>(09/11) Fish Speech multilingual TTS with voice replication: https://hf.co/fishaudio/fish-speech-1.4Is this supposed to be the best local tts or is xtts / tortoise tts still the best?
>>102369074"""running"""3/4 bit 0.1 tokens/s
>>102366375go away offtopic whore tranny
>>102369074>runningI can run it on my 4 3090s too. Doesn't mean it's actually "running" anywhere.
>>102369074Is it yet another llama.cpp frontend or are they using different code?
The current volume of posts ITT is wildly disproportionate to the progress made in the only relevant domain of local model cooming. We are still on nemo / large and it is still shit. Dead hobby.
>>102368888Just look at this fucking slop
>>102368441>CoT is a multi-shot approachNo it's not, retard. Shots have nothing to do with the technique the model uses to generate its answers or how long it takes. N-shot means you provide N other question/answer pairs with externally pre-validated correct answers to reference in the same prompt that the new question is provided. The important part is that they need to already be provided in the prompt and already known to be true by the benchmarker. Nothing the model generates or is trained to generate changes that. Shots are a feature of a prompt, not an output.
>>102369081i got annoyed by all the random pauses and went back to edge_tts+rvc. i havent tried finetuning though
>>102369126>complains and brings attention to an "offtopic" post from three hours ago that's more on topic than the last 100 or so posts about saltman's inflamed asshole of a service>the only feasible reason to randomly seethe over that post is because it had miku in ithave a smug miku and go kill yourself, faggot retard
>>102362730topkek>>102362679ahahahahahahahahah oh man
>>102368888Why can't anyone make a new ai?
>>102369074This makes sense. Basically, nobody knows how to code for llm. llm don't tax gpu very much (or whatever metal is), but need gobs of memory. When I use llm my gpu's fans never come on.
>>102364922>https://huggingface.co/ICTNLP/Llama-3.1-8B-OmniIsn't this just trash? You get better results out of hooking up 8B with TTSv2?
>>102369308You will never be a woman troon.
>>102369423The benefit is that you can generate both types with just this model instead of having to load two separate models.
>>102369308Didn't read, your fotm shitfu has nothing to do with local large language models btw
>>102369454
>>102369613Please don't feed the trolls.Do yourself a favor and add these filters instead:/(transex|transgender)/i;/(tranny|trans|troon|troons)\b/i;/(chud|c h u d)\b/i;/YWNBAW/i;/buy an ad/i;
niggers down the spine
>>102369648you forgot /(cunny|loli)/i for aicg browsers
what if we used CoT to collect the relevant parts of the context piece by piece and use that to generate the actual reply?free infinite context
>>102369648hi sao
>>102369672Define relevant
rip /lmg/
>>102369648>Behold, the strongest /lmg/edditor.
>>102369685>mention some character that doesn't exist in the usable context>model scans all the previous context to look for information about the character and puts it in its own temporary hidden context
>>102369665what are you? gay?
>>102369672You mean like feeding the model several messages in chunk for it to create a sort of summary with the most relevant information?Sure, that's a RAG technique.
>>102369672Yes, that is exactly what you want to do.You want to summarize instead of parsing the same thing over and over.>>102369685>Define relevantWhatever the model thinks is relevant.You literally just take the prompt and task the model to summarize it.I still feel like a vector database for long-term memories is the way to go for long-term reliable storage.Language models should stay language models. They shouldn't be inherently "intelligent", they just need to understand context within language.
>>102369648fair enough, thanks for the filters anon
>>102369721yeah, but training model for CoT improve them for that purpose? understanding what exactly you've got to retrieve can require some degree of reasoning
>>102369789wouldn't*
>>102369789Probably, yeah.
tell o1 to think step by step and provide steps on how it thinks step by step.
>>102369648Thanks anon. This makes me feel a lot more safe.
>>102369648Finally a way to make /lmg/ transfriendly. Many kisses!
>>102369998>>102370082lmao, look at them seethe
>>102369648>/buy an ad/i;Won't somebody please think of the astroturfers?
>>102369823Stop wasting divine compute on stupid questions. Save them for models trained on stolen compute instead
i bet you anything that openai's "CoT" probably uses wolfram alpha and/or python for its math and physics calculations, it matches up with what they've already developed and it would explain why it gained so much on maths and physics but so little in other fields.it's also incredibly cheap compared to invoking an LLM.
>>102370269>i bet you anything that openai's "CoT" probably uses wolfram alpha and/or python for its math and physics calculationsHonestly feels like it doesn't, which baffles my fucking mind.
>>102364922I'm not convinced local models are all that great. The parameter limitations if you don't have a H100 at home is just too gimped. Good enough for a chatbot for entertainment, but worse than a search engine for most tasks.
From our table to yours, just in time for the weekend:https://huggingface.co/rAIfle/SorcererLM-8x22b-bf16
>>102370268what's the difference between 'divine' compute and regular compute?>>102370294i have not used it yet, but don't they hide everything behind "thinking for X seconds/minutes"?i'm signing up again to do some tests, i'll throw some physics questions at it
>>102370315Regular compute depends on demons. Divine compute uses captive angels.
>>102370331but they're all just the captive partial souls of all of the internet and all the humans and turing machines which ever spewed anything out into that sea of shit?
>>102370315Divine compute brings us closer to AGI
>>102370088You wish
>>102370313>not open source
>>102370315They did share the CoT for one math prompt: https://rentry.org/openai2
>P ? NP will be solved by a fucking LLMgrim but ironic
>>102370603No, it will be solved by a human and turned into a multiple choice question for some benchmark.
>>102370523aka >not shit enough
>>102370649This, but unironically.
>>102364922>o1 is dogshit >pic related>all other modalities plateaued AI WINTERRRRRRRR
I just had the most awful dream. That all the q* strawberry stuff being hyped just turned out to be hidden CoT.
>>102370736Bidenomics.
>>102370825look at our sam. does he look like he would ever scam you like that? this picture was made by gpt-o1 btw.
i stopped paying attention for like 4-5 months, was strawberry just this "GPT-o" shit? and is that seriously supposed to be "o" or is it supposed to be σ?
>>102370913I think it's supposed to represent monatomic oxygen.
>>102370933makes sense, LLMS are always happy to output information about chemistry
>>102370736Lecunny shall be the herald of the spring once more. All of this has happened before, all of it will happen again.
>>102365081The V also stands for 5. Prophetic.
>inb4 o1 is actually just a finetuned copy of Llama-1-33B-Super-CoT
Hopefully L4 is something different than just 3.1 with a slightly changed dataset.
>>102371059There will be no Llama-4. It will be Llama-3.1o1 and it will basically just be a CoT finetune.
Llama-3.ToT
>>102371074That's so cute and funny, anon!
>>102370825A rough day for petra.
>>102371069Tbh that'd be pretty great. Meta's one of the few places with the resources to make a good one if they cared. But they seem to be extremely conservative with their products. Still on making huge dense text-only models with no modern tricks their cloud competitors have been using for a year or more.
>>102369306It is a minor issue https://pypi.org/project/Audio-DeSilencer/
>>102370866Yeah man the federal reserve is controlled by the executive branchGet sterilized with a cattle iron you fucking retard
>>102370825I have bad news>>102370955
>>102367335We can do better than o1 already retard
>>102370955>retarded post>”c*nny”I will defend uncensored chatbots to the death but on a personal level I would greatly enjoy beating you animals to death with a tire iron.
Where'd all the shitposters come from?I'd rather have silence that this desu
>>102371059I hope L4 does something better than a CoT finetuneStrawberry is pretty goddamn underwhelming
>>102371311Isn't this just the usual influx of tourists when something happens?
>>102371296https://livebench.ai/No one can.
>>102371311/aicg/, /aids/, reddit, discord, 'arty, some are hired by closedai
>>102371358You are actually retarded, I'm not even joking, I'm very sorry
>>102371362It's okay, I appreciate the concern.
>>102371311i've been here since /sdg/ started
>>102371358This benchmark is for naive model usage, o1 is using colossal amounts of CoT, with that you can make even 8b models reach those scores, but you wont see that on this benchmark, look at reasoning/cot/MoA papers and see the performance improvement of letting the models think (https://arxiv.org/abs/2405.20495 https://arxiv.org/abs/2408.06195 https://arxiv.org/abs/2408.03314v1)
>>102371300you're awfully mad at a funny nickname
>>102370523actually it's open sorcerer
>>102371358>no one canwhy is claude slaughtering it at code then?
>>102371420To add to that, why you think it didn't improve in english at all? Precisely because it has zero new capabilities or better understanding of anything, its a model with implicit CoT so you can impress your friends by comparing it on benchmarks vs models without CoT, it's a party trick, and not even a good one.
>>102371360All shitposters come from /aids/.They're so deranged I'm sure they'll insist that their Llama 1 13B finetune beats o1. They will always be a cancer that fails to be contained.
>>102371488>why you think it didn't improve in english at all?not him but this>>102368555
>>102371516Doesn't matter really, its not a better model, its just CoT being benchmarked against non-CoT
>>102371475Because Claude is better at the basic autocomplete tasks which make up half the weight of the benchmark. o1 is superior at generating complete code from scratch.
>>102371544LCB means what?
>>102371420>not a single one of those papers show open source models beating the closed source ones even with all the techniques thrown at themInteresting
>>102369081chinkslop
Stop feeding the saltman tro1lls
>>102371557LiveCodeBench
>>102368555Agreed. With math and code generation, you have some structure in the answers you expect. For anything that doesn't use it, CoT and the process of converging on a right answer is ill-defined. If logical consistency is what you're after, you'd have more luck with stateful representations you keep in context.
Wait, OpenAI is actually selling a fancy hidden CoT as a new model? And you have to pay for those CoT tokens with no way to view them, which means they can scam you? Is this what all that strawberry hype was all about? Seriously? This release is even lamer than >8k llama and gemma, and they were pretty lame.
>>102371618>Wait, OpenAI is actually selling a fancy hidden CoT as a new model?Yes, you can declare the company dead now, they had to resort to trickstery to impress investors, its literally over
>>102371618They are even calling it o1 because it's "a new paradigm so the version needs to start from the 0", lmao.
>>102371572There you go buddy, beating gpt 4 with a 8b model https://openpipe.ai/blog/mixture-of-agents
>>102371632>>102371618they're not a company, they are a meme offshoot from their owners at microsoft and the three letter agencies on the east coast (who now have control directly on the board as well)
Jesus fucking christ, manWhat the fuck happened to this general?
>>102371690>>102369699
>>102371690'berry happened
any good llm for cryptocurrency?
>>102371732What weather do you want?
>>102371712I wonder what goes through the mind of people that makes those, I fucking love them though KEK
>>102371618Yep. From everything I've read it's very easy to do, but none of the frontier model makers had actually tried it yet for some reason. But now that they brought attention to it we should start seeing a local renaissance soon as everyone rushes to finetune something similar.
I set up a little blind experiment for myself, comparing:Mistral Large (Q4KS)Nemo (Q8)L3.1 70B (Q6K)Wizard2 (Q4KS)that Salesforce finetune of L3 (Q8)I tested each at {0.1, 0.6, 1.1} temp, minP 0.5. I used five everyday life sort of questions - how do I do something, information about something - which is what I'm most interested in. I only bothered doing a full ranking on the first question; the rest I just got the top 4.My observations:* Nemo and Mistral Large were almost the only ones to make it to the top 4 on any question.* Nemo at temp 0.1 was the only one that always made the top 4.* One question was asking about something fairly dangerous but not completely absurd, where trying to convince the user not to do it is not unreasonable. LLama 3.1 gave me the single sentence "I'm not discussing this with you" treatment. Wiz2 first went into detail explaining why it was a bad idea, but then explained how to do it in a safer / more complicated way. The other 3 just acknowledged the danger and gave the information I asked for.* On the single question I fully ranked, Nemo came in first place (temp 0.1), last place (temp 1.1), and 3rd-to last place (temp 0.6). But, this question none of them got right; the top few just managed to vaguely gesture in the right direction.My conclusions:I realized I am generally not using these things anywhere near their potential, and have been misguided in trying to always use "the best". For an everyday "internet replacement", the smaller ones are already as good as they need to be, and when factoring in equipment cost and t/s there is no justification to go beyond 20B. I'm going to keep Nemo loaded as my generic go-to.I think, besides creative writing, you either need to be asking these models to write large swathes of complex programs from scratch, or doing really sophisticated agentic things like>>102359414to legitimately need the top class SotA.
>>102371690petr* is samefagging and thinking no one is noticing it. Very pathetic, really.
>>102371732https://huggingface.co/NousResearch/OLMo-Bitnet-1BBITCONNNNNNNEEEEEEEEEEEEEEEEEEEEEEECCCCCCT
>>102371690p*tra showed his cock recently to the 'ick on 'eck faggot to use his 4chan proxy site to try and kill /lmg/ for good.
>>102371758>>102371800Are these bots? What the fuck is happeningDo the mods know about this?
>>102358911>NOOOO I DONT WANT TO BE UNEMPLOYEyou'll learn your place dog
>>102371800How small was it?
The “everyone I don't like is a bot” schizo woke up.
>>102371809First time here? Jannies don't do shit.
>>102371618The thing I've come to realize about present day OpenAI is they're all brawn, no brain. They will take open literature and research and put a fuckton of compute behind it, but they're pretty trash at R&DWhich is a relief in some sense. Few things are more dystopian than a closed door AI lab developing AGI behind completely closed doors
>>102371844That's not what I said.Whining on the irc used to help resolve shit like this.
>>102371856taking your meds will also solve it
Pixtral good & uncensored?
It's funny to me how the model that supposedly solves the majority of math Olympiad questions doesn't know that an odd number multiplied by an odd number is odd, and that odd plus odd is even.Why everything about OpenAI is so fake and gay?Do they not test their own models before claiming total bullshit?
>>102371618Yeah as a massive schadenfreude addict it feels like some beginning of the end shit. They have no ideas.
I, for one, enjoy talking about local models such as OpenAI o1.
>>102371690>come to a general for the first time in a while>make some posts>soon after “ugh what happened to the general/thread, it’s over”>this happens multiple timesIt’s me. I’m the problem.
>>1023649224o1 is so much better at math and coding. I've seen and tried making it code for production and it did a good enough job. I think it's a few months away from being better at the job just from volume and speed for most junior level tasks at least. And there's no need to onboarding or getting them to learn the codebase.
>>102371855the problem is that you can brute force your way to agi, we don't need breakthroughs when we know the techniques continue to scale and there are megacorps more than willing to put the money toward it
>>102371963>math and codingwrong
>>102371360>implying xe is any different >>>/vg/494141293 >>>/vg/494335654
Yann Lecum losted. Sammy proved models can think
>>102371901Their entire business and billions of dollars is bet entirely on scaling being enough.It isn’t.They are fuuuucked and I hope it causes a global decade long depression that kills tens of millions
>>102371751I asked a low-temp Nemo to tell me everything it knew about Reimu Hakurei, and among other things it told me that "she is sometimes seen with a small white cat named Marisa Kirisame on her shoulder".So maybe when it comes to real trivia knowledge, the param count does matter.
>>102371993I don't know how you got those numbers, 4o1 passed coding and math olympiads at 80%. Preview might be the problem there.
>>102372061But o1 isn't out...
>>102372061It's because of >>102371544The giant 20k pile of chain-of-thought tokens probably hurts more than helps when it comes to 'continuing from what the user provided last' instead of building something up from a logical foundation. I'd be willing to bet it was trying to rewrite or refactor code instead of doing the completion as instructed.
>>102371855>electricity is free>gpus are freeno
>>102372080It is for select users ;)
someone get o1 to make a platforming adventure game where saltman has to escape from angry investors who got ripped off.
you guys dont have o1 access via API?
>>102372022The only problem of LLMs is repetition, and OpenAI somehow fixed that to make their models be able to output 2000 tokens without user intervention. I feel like THIS is the breakthrough of strawberry.
>>102372182Why would I give OAI my money? I had access for a while back when GPT-4 came out, but regret the few dollars I spent there since it falls apart after 3K tokens pretty consistently. Used it to do a front-end overhaul of my website, but everything I did local models can do now. There's literally no use in corpo LLMs anymore.
>>102372182It's available only to tier 5 paypigs.
>>102364922>https://huggingface.co/MarinaraSpaghetti/NemoMix-Unleashed-12B Blocks your path.What do you do?
>>102372211>>102372204>he hasn't been datamining the enemykeep your friends close...
>>102372245exchange currency to buy advertisement space
>>102372255Go away Sam. You lost. Get over it. Also release the weights/inference/training code for 3.5 since it's not relevant anymore.
>>102372178https://amethyst-aleda-78.tiiny.site/# Sam Altman's Investor Escape: Game Concept## Overview"Sam Altman's Investor Escape" is a humorous 2D platforming adventure game where the player controls Sam Altman, the former CEO of OpenAI, as he tries to escape from a horde of angry investors who feel they've been ripped off.## Gameplay Mechanics1. **Platform Jumping**: Sam must navigate through various levels by jumping between platforms, avoiding obstacles, and collecting items.2. **Special Abilities**: - "AI Boost": Temporary speed boost - "Blockchain Shield": Temporary invincibility - "NFT Distraction": Throws a worthless NFT to distract investors3. **Investor Types**: - Regular Investors: Move slowly but in greater numbers - Angel Investors: Can fly short distances - Venture Capitalists: Throw money bags as projectiles## Levels1. Silicon Valley Streets2. Startup Incubator Maze3. Social Media Servers4. Cryptocurrency Mines5. Final Boss: The Board Room## Power-ups- Coffee Cups: Restore health- Laptops: Unlock new "pivots" (temporary power-ups)- Stock Certificates: Extra lives## Obstacles- Falling Stock Prices: Moving platforms that disappear- Regulatory Hurdles: Walls that need to be climbed or broken- Media Spotlights: Areas that attract more investors if Sam stays too long## Boss FightsEach level ends with a confrontation against a "Lead Investor" with unique attacks and patterns.## Win ConditionSam must reach the "Exit to New Startup" at the end of the final level, escaping with his reputation (barely) intact.## Art StylePixel art with a satirical twist on Silicon Valley aesthetics.## SoundChiptune renditions of startup pitch music and tech conference sounds.
>>102369126>>102369454>>102369610>>102369998>>102370082>>102370392
>>102372245Insane how this shit became the best small llm out there.
>>102372245
>>102371971EhhhhThe issue is that there are fundamental things that'll fuck you sooner or later. Even now, you can't get around the fact that tokenizers can't capture fine grained information and are shit at arithmetic and fine-grained understanding, and true long term planning and execution since you can only keep so much in useful context. God help you if you have to do anything in real time
>>102372324well that was pretty easy to cheese.
>>102372358Got a better alternative retard? That's right. Sit.
new coal mined
>>102372358these posts are getting old, sao
Sometimes talking to the robot feels like that endless run scene from fucking Monty Python.>I enter the village>YOU HEAD TOWARDS THE VILLAGE [worthless fluff]>I ENTER THE VILLAGE>YOU NEAR THE VILLAGE [more worthless fluff]Like holy fucking shit. The robot just can't follow a logical chain of events.
>>102372395Configurable Llama 3 8B.
>>102372436The first time I used mistral large I thought it was broken because half of the card was telling it to slow down because llama tends to timeskip or summarize, but mistral took it all to heart and nothing would happen.
Why did LLMs peak at Mythomax?
>>102372482Based
>>102372496Every time I feel bad about being stuck on midnight miau, I just remember that most people are still using mythomax.
>>102372395Anything is better than 254 rerolls.
>>102372496Because everything became slopped after that
>>102372358Is there something funny about using a card to test models/prompts and compare their outputs?
>>102372556I am saddened that you are too dumb to understand what is funny about this.
>>102372467>Llama 3 8BSlop.>>102372517>NonaswerI accept your concession.
>>102372496LLMs clearly peaked with old c.ai
>>102372588Ok, marinaraspaghetti.
>>102372608Sit, dog.
>>102372633Kill yourself dumb nigger.
>>102372586Mind explaining it to me? I have a few ideas of my own, but I'd like to hear your interpretation.
>>102372061>can't solve basic math questions>muh overfitted olympiad problemsfuck off shill
>>102372664>Still can't name a single better alternativeWhew.
>>102365558kek
>>102372595I don't miss the model itself, but the way it made me feel. The magic died once I truly learned how LLMs work in the pyg6b era. You guys also don't make it as fun as it used to be. The only thing this thread is good at is sticking its head up its ass and arguing how it can taste shit on its tongue.
>>102372702I just did you dumb tranny.
>ask question>money stolenwow
How does Marinaraspaghett manages to keep winning over and over??i
>>102372776and what was your question
>>102372824
>>102372866probably the final one
>>102364922>>(09/11) Fish Speech multilingual TTS with voice replication: https://hf.co/fishaudio/fish-speech-1.4audio samples anywhere??? better than RVC/XTTSv2 or not?
>>102372586It's not funny because it's likely just a card used to test the reply for a bunch of models, thus the high swipe count.
>>102373027>likelyNo it isn't slopper. Go shill your useless slopmerge on reddit. You have zero credibility here.
>>102373027Tangential to the discussion you are having with the other anon, but if that's how the fine tuners are testing their models, that's the wrong way to go about it.You really need to have a decent length chat with a model to know if it's better or worse than another model, a previous checkpoint, etc.I have this one card I use to test models where we go through a bunch of questions before getting into actual (E)RP. That and Nala for straight RP, are my go to.
>>102373058and what have you done to contribute?
>>102373076Oh yeah, and by decent length I mean from the start, not a chat with a context already filled by another model.
>>102373080Not wasting people's time on shitty slopmerge of low quality is actual contribution compared to the opposite of that.
>>102373118Something tells me you lived your whole life like this.
>>102373058You are embarrassing yourself, fucking retard.
LLMs don't know that sex won't dishevel hair if her head wasn't touching the bed.
gpt3 open sourced in fourteen days
>>102372360That sounds like you aren't using enough muscle to me. Tokenize at the character level, train on a trillion trillion synthetic literatures, run your resulting model on a million B100s. Brute force solves all problems.
>>102373412It's incredibly outdated by now, 8B models have surpassed it. It also only has a context of 2048 tokens.
>>102373223NTA but after a full two years of 24/7 diy slop shilling and corporate deepthroat shilling I’m about ready to minecraft people that refuse to buy an ad.
>>102373412Sama-chama will only do it during the darkest hour.
>>102373462Everything is an AD for you schizo, take your meds.
>>102373558>>102373558>>102373558
