>Lama-3.1; 8b, 70b and 405 is now released on huggingface!405b is beating chatgpt-4o, while being just half the parameter count, possibly because of not suffering the crippling effects of wokeness infusionWhile the new 8 billion parameter models seems affected in reasoning by being forced to learn thirdie langues compared to the old 3.0 version.
>>101535820
>>101535820What's the VRAM requirement for 405b?
>>101536000for 4q about 200gb, then more for context
>>101536000one Mac Pro with M2 Ultrasay itTHANK YOU BASED APPLE
>>101536000Literally won't run regardless of what you change go for the smaller models
>>101535820>he thinks the top llama model is uncensored>he can't even run the foundation model on runpod without $200+ a month on runpod
>>101535820>open source llm and nn are the path forwardSecond that. Just don't tell any retards. Work or wait for or both. It will come and it will be the best.
>>101536329>one Mac Pro with M2 Ultra405B model, 192GB Mac...you're going to run it at q3? Plus, even the absolute best M2 barely touches at 3070 in terms of performance.
>>101535820Mark looking extra zesty here
>>101538815thats just his new cool look
Whats the cheapest Hardware that can run the bigger models? Doesnt need to be fast just borderline usable. Do i have to become an itoddler?
>>101535820>405bwhat kind of specs do you realistic need to run something like this at home? definitely not something a desktop can handle on its own but how much hardware does a home server g tard need?
>>101541866something like 160GB of VRAM however you can manage that
>>101543013H100 dgx has 640 GB of vram
>>101535820Is that the one that was leaked ages ago?Did anyone make anything with it?
>>101535820nothing released so far is even remotely cool. the dalle3 shit was KINDA NEAT for like 3 months and it died out with how woke it fucking got after that first week of it being kino. the whole ai thing is so shit. lama 3 405 just came out. it is at best as good as gpt4 lol like where the fuck are the robots and the fucking ai that will take over peoples jobs?!?!?
>>101535820I trust this mother fucker as far as I can throw him. What's the catch Zuck? You want me to believe your releasing a multi million dollar piece of software for purely altruistic reasons..
>>101536189>>101538988>>101541866Could I run something like this on say 128 lanes of avx512 at a tolerable token rate? Because I have access to a rig with nearly a tb of system ram. >>101545395He's doing it to fuck over Microsoft. He wants a model for internal use, has the hardware to train one because he was expecting to need it for the VR bullshit and ticktock knockoff. It looses him very little to release it to everyone.
>>101545395To bring down Microsoft/OpenAI.Which I suppose counts as altruistic reasons.
But can it produce the filthiest smut imaginable with no content restriction?
>>101545536>>101545589Nah, I think there's more to it than that. Here's a few theories:Get it integrated into everything then hold everyone hostage. Facebook is on the decline, so he's all in on anything that will bring users.And lastly, long term goals. If his company can steer or control responses, he can subtly influence users for political or financial gains.
>>101535820How are the 'scores' generated for rating these models?>>101536000Someone had a 4090 and used the 405B option and it took him 30 mins for it to generate the word 'The'.I am currently trying to install the 8B version but im a noob and only messed around with stable diffusion before this, so im just trying random shit the meta AI site tells me when I try to use it for tech support. Right now I got an error because I need the llama library but it cant install the llama library due to some error and its suggesting I download an older version of python but idk how to do that. and i tried updating pip and installing the other dependencies too.But I need to finish this challenge of getting it to run locally I refuse to always use a website for it
>>101545703what os are you on?
>>101545770Windows of course, since I am a humble gamer dabbling in these things. Im using git bash as a terminal I guess although I tried to use powershell to see if that would magically fix any errors but it did not.
>>101545536>fuck over Microsoft.Dare I say it, but, based?
>>101535820So it's "unsafe" and will have the hammer brought down on it behind the scenes.
>>101545395Free testing
>>101535820Sauce on 4o being 800B?
>>101535820Open source so that retards can provide cuckerberg with free work and data. Great idea.
>>101535820>405b is beating chatgpt-4ono it's doesn't, it's barely better than the now decrepit chatgpt3
>>101535820zammmmmmmn zucky look like dat??
>>101546056nobody works on Llamo for free thoonly Stable diffusion corp managed that
The Llama-3.1-8b is pretty damn good.Like turbo level at least which is great for local
>>101547271Did they up the context?
>>101537268>he thinks the top llama model is uncensoredYikes. Fucking silicon valley cucks still complying with DEI.
>>101538815crazy what a good haircut can do.>>101545395"The enemy of my enemy is my friend" kind of reasoning.>You want me to believe your releasing a multi million dollar piece of software for purely altruistic reasons..To be fair, they already released a multi million dollar piece of software as open source (Pytorch), so MAYBE he's not so evil...
>>101544098Nigga that shit costs almost HALF A MILLION DOLLARS
>>101545536define tolerable token rateeither way, it'll be way under 1 tokens/sec
>>101535820>405bMan, I only get about 35tk/s with 70b on a 4090... I can't imagine there'd be a person out there with the hardware to run the big one that fast.>>101545703>Someone had a 4090 and used the 405B option and it took him 30 mins for it to generate the word 'The'.lol
>>101545395If I'd have to take a guess, then cheap AI probably makes Facebooks core buisiness more valuable. Maybe they're trying to boost the ecosystem to eventually use AI to offer professional shilling services, hellban people or "reanimate" dead loved ones on facebook for a fee
>>101547287I don't know, I only got a quickie out before I had to leave. I'll put it through it's paces tomorrow though.It's very coherent and seems to be uncensored.
>>101536000wouldn't a dual socket cpu be reasonably fast and cheaper at that size? I doubt strapping x10 p80 together will be fast.
>>101545703just follow https://github.com/Mozilla-Ocho/llamafile?tab=readme-ov-file#using-llamafile-with-external-weights its the best way to test instead of running all these shitty python venvs that pull half the ecosystem.
>>101547287I think so
why
it sucks dick refuses to write anything useful because muh violence when there is nothing like that. I told it to capture analyze a poem and it shits itself about muh safe guards.
>>101536189>>101543013So this is unreachable for consumers? I assume consumers access models running on data center GPUs remotely. But the cost and energy use must be enormous. What the fuck?
>>101545395To reduce value for OpenAI/Microsoft dominance. OpenAI is setting themselves up to swallow the government contracts whole.
>>101549008Yeah corporate trannyism is in place. Surely its not your first time using a corporate model
>>101547486what are you, poor?
>>101549220no, just saving up for a catgirl loli local llama life size silicone stretable reusable pleasure AI sex robot
>>101549310oh nevermind, go on then king
>>101549120it is. They advertise 128k context and after a couple prompts it starts spamming ##########"""""""""""
>>101535820'berg be bussin right now, brah.
>>101535820Zucky M did it
>>101547486Just borrow one from work
